Commit Graph

650 Commits

Author SHA1 Message Date
Cosmin Băieș
c1863f8887 Add missing validation in IInputMethodWrapper
The case handling DO_SET_STYLUS_WINDOW_IDLE_TIMEOUT was missing
the validation check, which could lead to NullPointerExceptions.

Test: n/a
Bug: 278853086
Change-Id: I998e20df74b618396cfe3b1fa5d7d4e7d13e054c
2023-04-19 14:14:50 +02:00
Prabir Pradhan
c4574fc01f InkWindow: Provide a copy of the motion event to view root
When a motion event is enqueued into the view root, the view root
expects to own the event, which means it will finish and recycle the
event if necessary.

In the handwriting pipeline, this event actually comes from an
InputEventReceiver that handles finishing the event, so there is a
chance that the event is finished or recycled twice. Make sure the view
root owns the event that is enqueued into it by providing it with a copy
of the event.

Bug: 277809922
Test: manual, HandwritingIme
Change-Id: I65968d075640c1dd585d5d7b9eadb78e3dff9aa2
2023-04-12 00:16:04 +00:00
Prabir Pradhan
0cb1ad008a Integrate PointerIcon with Scribe
Here, we do two things:

1. To account for the case where the changed the stylus's PointerIcon,
   reset the PointerIcon when handwriting is successfully initiated.

2. Inject the handwriting events directly into the InkWindow's view root
   instead of dispatching it using the view's dispatchTouchEvent()
   method so that things that are handled by the InkView's root will
   work as expected, such as setting the pointer icon. This will allow
   the InkView to change the PointerIcon using onResolvePointerIcon.

Bug: 275851541
Test: Manual
Change-Id: I97d8d4293a5440a6d6531f279ed7811370671111
2023-03-31 19:24:51 +00:00
Adrian Roos
ff35ca73ac Use CancellationSignalBeamer in InsertMode, preview
Implement CancellationSignalBeamer in InsertModeGesture and preview
HandwritingGesture

Bug: 254727073
Test: atest InputConnectionEndToEndTest

Change-Id: I0ddcac59cfb6d200f6eab71180106f3545a7d608
2023-03-16 22:03:51 +00:00
Ahan Wu
f6862470f7 Instrument latency and jank for IME
This CL relands CL[1] and fix NPE as well.

[1]:Iaccb6021cb081a71d7b8819113eb1cfde19b4af5

Bug: 231534374
Bug: 265630720
Test: atest FrameTrackerTest InteractionJankMonitorTest
Test: atest LatencyTrackerTest
Test: atest FrameworksCoreTests:android.view
Test: atest SpaLibTests
Test: atest CtsInputMethodTestCases
Test: Capture trace locally
Change-Id: I99cbf06c39cb7d405f62ef647b179ac98b4f00f7
2023-02-03 06:06:34 +00:00
TreeHugger Robot
f9d4c062b4 Merge "IMF: Factor out OnceResultReceiver to fix TextBoundsInfoResultReceiver" 2023-01-24 18:44:16 +00:00
TreeHugger Robot
1eb47c7f40 Merge "IntResultReceiver: fix clearing on dispatch" 2023-01-24 18:43:23 +00:00
Adrian Roos
953d9873f4 IMF: Factor out OnceResultReceiver to fix TextBoundsInfoResultReceiver
Fixes an issue with TextBoundsInfoResultReceiver, where any Executor
that does not run the Runnable directly crashes, because the mConsumer
is cleared before it is run. Also avoids calling into the executor
with internal locks held.

Bug: 265457058
Test: atest InputConnectionEndToEndTest
Change-Id: I4b229494fb2451135ae1075272f04d865a9b509e
2023-01-20 15:27:54 +01:00
Adrian Roos
687a05d319 IntResultReceiver: fix clearing on dispatch
Fixes an issue where mConsumer was cleared too early for any executor
other than Runnable::run, leading to an NPE when the Runnable is invoked
on the Executor.

Fixes: 265457058
Test: atest InputConnectionEndToEndTest
Change-Id: Ifc8e435e02520eb29c9631427daa5a1a1ea0ccf0
2023-01-18 22:12:13 +01:00
Aurimas Liutikas
44bac83ac2 Revert "Instrument latency and jank for IME"
Revert submission 20190403-ime_jank_latency

Reason for revert: causing runtime crashes in tests
Bug: 265630720
Reverted changes: /q/submissionid:20190403-ime_jank_latency

Change-Id: I79b3aa51828e5e4af34ddc29a012eb9abac49c10
2023-01-18 02:09:28 +00:00
Ahan Wu
31bee097b9 Instrument latency and jank for IME
Bug: 231534374
Test: atest FrameTrackerTest InteractionJankMonitorTest
Test: atest LatencyTrackerTest
Test: atest FrameworksCoreTests:android.view
Test: Capture trace locally
Change-Id: Iaccb6021cb081a71d7b8819113eb1cfde19b4af5
2023-01-13 08:22:32 +00:00
Haoyu Zhang
3b47b24850 Address API feedbacks for TextBoundsInfo
Bug: 262452997
Bug: 265068032
Test: atest TextBoundsInfoTest
Test: atest TextBoundsInfoMethodTest
Test: atest TextViewTextBoundsInfoTest
Change-Id: I2105efc8a90fee7f11d5c534c18e659d821abf1c
2023-01-10 16:41:59 -08:00
TreeHugger Robot
439e94fa09 Merge "Initialize ImsConfigurationTracker when IME#onCreate" 2022-12-01 03:05:12 +00:00
Ming-Shin Lu
9d42c3324a Initialize ImsConfigurationTracker when IME#onCreate
To parse InputMethod_configChanges locally without waiting system
callback with initializeInternal, in case IME#mInputView may not
able to call resetStateForNewConfiguration in time before
IME#initializeInternal when it's required.

Fix: 260198823
Test: atest InputMethodServiceTest
Change-Id: I0268360f1b60ebc1f2676d3edab1910a9ce0f050
2022-12-01 00:52:17 +08:00
czq
9e3642a928 Fix bug of keyboard hidden after rotating to landscape mode.
Add logic in IMS#onShowInputRequested: if IMS receives a Implicit show request and the keyboard is already shown in full screen mode, there is no need to hide it.

Otherwise it would hide the keyboard after rotating to landscape mode, because the rotation causes recreation of the activity and triggers auto-show behavior of Ime.

Bug: 256739702

Test: manually test

Change-Id: I195c2c6bd6f6815806dc90f0a386b7c0bccee7a1
2022-11-23 12:00:35 +08:00
Cosmin Băieș
44ff0dd9eb Revert^2 "Input latency & reliability tracking - hide flow"
d638a60aae

Test: atest CtsSystemApiSignatureTestCases:android.signature.cts.api.system.SignatureTest#testRuntimeCompatibilityWithCurrentApi
Change-Id: I1ee7224b40fa3f4b744311eb4145fa13a87d835a
2022-11-11 12:22:55 +01:00
Cosmin Băieș
1619482379 Revert^2 "Input latency & reliability tracking"
1f4fd2d21e

Change-Id: I096456e9cd7449836b373ff392f2e9df58b3e661
2022-11-10 17:53:07 +01:00
Pascal Mütschard
581fdec050 Merge changes from topic "revert-20313761-IME tracking - hide-HZXGDCCPOZ"
* changes:
  Revert "Input latency & reliability tracking"
  Revert "Input latency & reliability tracking - hide flow"
2022-11-10 13:22:35 +00:00
Pascal Mütschard
1f4fd2d21e Revert "Input latency & reliability tracking"
Revert submission 20313761-IME tracking - hide

Bug: b/258637227
Reason for revert: Breaks CtsSystemApiSignatureTestCases android.signature.cts.api.system.SignatureTest#testRuntimeCompatibilityWithCurrentApi
Reverted Changes:
Icf9ab3fc0:Input latency & reliability tracking - hide flow
I1096564d5:Input latency & reliability tracking - hide flow
I14e009250:Input latency & reliability tracking - hide flow
Ic32e36b2d:Input latency & reliability tracking
I56f96dc6b:Input latency & reliability tracking
I73dec2e9e:Input latency & reliability tracking

Change-Id: Ia376a1624fb6abf937eb2a335aa75cb3ca9831d8
2022-11-10 13:11:27 +00:00
Pascal Mütschard
d638a60aae Revert "Input latency & reliability tracking - hide flow"
Revert submission 20313761-IME tracking - hide

Bug: b/258637227
Reason for revert: Breaks CtsSystemApiSignatureTestCases android.signature.cts.api.system.SignatureTest#testRuntimeCompatibilityWithCurrentApi
Reverted Changes:
Icf9ab3fc0:Input latency & reliability tracking - hide flow
I1096564d5:Input latency & reliability tracking - hide flow
I14e009250:Input latency & reliability tracking - hide flow
Ic32e36b2d:Input latency & reliability tracking
I56f96dc6b:Input latency & reliability tracking
I73dec2e9e:Input latency & reliability tracking

Change-Id: I0d0ce41e3a4f0d19ff482c2f5ab4884b5f758134
2022-11-10 13:11:27 +00:00
Cosmin Băieș
36d3884041 Merge changes from topics "IME tracking - hide", "IME tracking"
* changes:
  Input latency & reliability tracking - hide flow
  Input latency & reliability tracking
2022-11-10 10:19:00 +00:00
Lan Wei
54716fa8e3 Merge "Improve javadoc for InputConnection#replaceText" 2022-11-10 04:35:24 +00:00
Cosmin Băieș
553cb171de Input latency & reliability tracking - hide flow
Test: filter logcat for ImeTracker tag, open some apps with IME connection (e.g. Dialer, Messaging, etc.) and close the soft keyboard, check that tracking reflects observed behaviour
Change-Id: Icf9ab3fc0f7a7116456c13328d22f8b8428a0c79
2022-11-07 17:49:47 +01:00
Cosmin Băieș
6756327922 Input latency & reliability tracking
Test: filter logcat for ImeTracker tag, open some apps with IME connection (e.g. Dialer, Messaging, etc.), check that tracking reflects observed behaviour
Change-Id: I73dec2e9e2d5f919c7a006828ee0b12a998fc7d9
2022-11-07 17:49:41 +01:00
Taran Singh
48cba7c99a Merge "API for realtime Scribe gesture preview" 2022-11-04 18:12:07 +00:00
Rhed Jao
bedef9c26f Merge "Enforce package visibility rules on switch input method apis" 2022-11-04 10:47:38 +00:00
Rhed Jao
567dafb881 Enforce package visibility rules on switch input method apis
Bug: 227207653
Test: atest InputMethodSubtypeTest
Test: atest InputMethodServiceLifecycleTest
Change-Id: Ib5ba67f708b8ebe12b5120c126022eb84ce720cb
2022-11-04 14:16:34 +08:00
Taran Singh
95baa907de API for realtime Scribe gesture preview
Introduce new API to support gesture preview as user is operating over
text.

Bug: 254726230
Bug: 254727073
Test: InputConnectionEndToEndTest

Change-Id: I6c2ac5a5aaeaeab78411d24870f573670304e898
2022-11-03 18:04:00 +00:00
Wilson Wu
7e18e28108 Merge "Refine InputMethodService java doc" 2022-11-03 02:40:39 +00:00
Lan Wei
947fd9335e Improve javadoc for InputConnection#replaceText
Bug: 254278543
Test: atest FrameworksCoreTests:BaseInputConnectionTest
Test: atest CtsInputMethodTestCases:BaseInputConnectionTest

Change-Id: I3e63532eae8349aaff21cfa72ed65f01779103df
2022-11-03 10:24:29 +08:00
Wilson Wu
53df6b0874 Refine InputMethodService java doc
Add more description to discourage IME developers
not use deprecated onCreateInputMethodInterface and
onCreateInputMethodSessionInterface.

Also move paragraph tag since Block tags should be
put after the main documentation.

Bug: 254278652
Test: presubmit
Change-Id: I729fbe2e1544fc72580a08ccb763ab69c58af92d
2022-11-02 11:29:58 +08:00
Haoyu Zhang
167691ce12 Merge "Introduce TextBoundsInfo" 2022-11-01 05:00:28 +00:00
Yohei Yukawa
e9490612d5 Introduce @hide ParcelableHandwritingGesture
This is a follow up CL to our previous CL [1], which introduced
HandwritingGesture and several subclasses.

One of the challenge is that we only wanted to make child classes
Parcelable, while keeping the abstract base class HandwritingGesture
non-Parcelable.  Although InputConnection has only one method

  InputConnection#performHandwritingGesture(
          HandwritingGesture, ResultReceiver),

in the IPC layer IRemoteInputConnection needed to define methods for
each subclass, e.g.

 * performHandwritingSelectGesture(in InputConnectionCommandHeader,
       in SelectGesture, ...)
 * performHandwritingInsertGesture(in InputConnectionCommandHeader,
       in InsertGesture, ...)
 * performHandwritingDeleteGesture(in InputConnectionCommandHeader,
       in DeleteGesture, ...)

because their abstract class HandwritingGesture is not Parcelable.

To avoid such method definitions (and lots of type-based dispatching /
merging logic), this CL introduces

  ParcelableHandwritingGesture

as a generic Parcelable container of HandwritingGesture subclasses.
You can freely use ParcelableHandwritingGesture in AIDL method
definitions as follows.

  performHandwritingGesture(in InputConnectionCommandHeader,
        in ParcelableHandwritingGesture, ...)

In the Java side, you can easily wrap objects as follows.

  var parcelableGesture = ParcelableHandwritingGesture.of(gesture);
  var unparceledGesture = parcelableGesture.get();

Note that this CL is still an internal cleanup. There must be no
developer observable behavior change.

 [1]: I53bcb62e03ac1c371feb60d1385c88c921754092
      3e3ff1a3d2

Bug: 234882948
Bug: 239783077
Test: presubmit
Test: atest FrameworksCoreTests:ParcelableHandwritingGestureTest
Change-Id: I158026087653f6772a78cc2394e678ae9741fb00
2022-10-31 08:08:54 -07:00
Haoyu Zhang
43aae6ff6d Introduce TextBoundsInfo
This is an API for IME to request text bounds information from editor to
perform handwriting rich gestures. It's useful for IMEs to create custom
handwriting gesture that's not supported the HandwritingGesture. And
it can also be used to provide default rich gesture implementation for
handwriting gesture compatibility.

Bug: 239843501
Test: atest TextBoundsInfoTest
Change-Id: Ia7522371976a4e1fcdff4cc0d244ea427ad3d7c9
2022-10-26 13:51:45 -07:00
Lan Wei
08811aeb33 Fix javadoc for replaceText
Bug: 254278543
Test: atest FrameworksCoreTests:BaseInputConnectionTest
Test: atest CtsInputMethodTestCases:BaseInputConnectionTest
Change-Id: I7429d2b77d9f45c99f0b76808271ba61ad1612ae
2022-10-19 18:10:05 +08:00
Wilson Wu
44d2a22a0e Merge "Mark disallow IME interface compat ID as TestApi" 2022-10-17 07:53:53 +00:00
Taran Singh
4270121357 Merge "Disable InkWindow default animation" 2022-10-13 21:07:40 +00:00
Taran Singh
8b1240b7cf Disable InkWindow default animation
By default all PhoneWindows use Theme_windowAnimationStyle which
animates window up and down when gaining and losing focus respectively.
For InkWindow we don't need any animation.

Fix: 237331144
Fix: 245441579
Bug: 253477462

Test: atest StylusHandwritingTest
Change-Id: I209b03dfbed386a2863921b49d43858037d380b8
2022-10-13 19:03:48 +00:00
Taran Singh
2ae0de31d3 Merge "Remove inkWindow after stylus idle-timeout" 2022-10-11 16:12:14 +00:00
Wilson Wu
9f72b7db1b Mark disallow IME interface compat ID as TestApi
Mark DISALLOW_INPUT_METHOD_INTERFACE_OVERRIDE
comapt change ID as TestApi for CTS testing.

Bug: 148086656
Test: presubmit
Change-Id: I7b56d68cb5429c7076943f3d12b1d19cdf54d26a
2022-10-06 18:11:44 +08:00
TreeHugger Robot
21da3fc4ee Merge "Make RemoteInputConnectionImpl package-private" 2022-09-30 03:45:20 +00:00
Taran Singh
539176516a Merge "Handle non-initalized InkWindow" 2022-09-29 23:58:30 +00:00
Yohei Yukawa
f3023dade4 Make RemoteInputConnectionImpl package-private
This CL mechanically moves RemoteInputConnectionImpl from

  com.android.internal.inputmethod

to

  android.view.inputmethod

just to make it a package-private class, as it is (and should be) used
only by InputMethodManager.

This is a mechanical refactoring.  There must be no developer
observable behavior change in this CL.

Bug: 192412909
Test: presubmit
Change-Id: I2d9bd4a5cb39d506a70331c9c930a74580cdbfa0
2022-09-29 15:38:14 -07:00
Wilson Wu
c14d7e459f Merge "Disallow onCreateInputMethodSessionInterface overriden" 2022-09-29 01:51:10 +00:00
Taran Singh
cfef0232ca Remove inkWindow after stylus idle-timeout
After stylus hasnt been used for handwriting in last X minutes,
remove the the stylus handwriting window.

Bug: 243571274
Test: StylusHandwritingTest
Change-Id: I641be63e028ea9cbfd882ce318d30aad71fd8a4e
2022-09-28 18:53:13 +00:00
Taran Singh
b4fa71513c Handle non-initalized InkWindow
With [1] we changed the lifecycle of InkWindow to be short lived and
missed certain cases.

This also fixes CTS testStylusSession_fingerTriggersNavbarGestures

Bug: 243571274
Test: atest StylusHandwritingTest

[1]: Icd3eea91fe144cff7100d3ecf19191c064c0d196

This is a retry for I1593ff10a5520aadb5ed86a4a09f2405ee65c321

Change-Id: Ibf54599f41ac13121215c5344eb92ecca3259db1
2022-09-28 18:53:10 +00:00
Wilson Wu
32950c05c3 Merge "Fix potential failures in InputMethodService (2nd try)" 2022-09-27 07:59:47 +00:00
Wilson Wu
4bcdf9b201 Fix potential failures in InputMethodService (2nd try)
It had the chance for calling into IMS but IMS
already destroyed.

Ideally we should suppress any callbacks after
InputMethodService#onDestroy() and log them.

First try is CL[1], but it wrongly add the API to
IInputMethod. Using the InputMethodServiceInternal
is make more sense for IInputMethodWrapper to
communicate with InputMethodService.

[1]: I10f26a92e739b60c5b8010e8476f8f5a403e0a7b

Bug: 121269861
Bug: 211062619
Bug: 148086656
Test: presubmit
Change-Id: I7b4f34eb38f44dea6872eb46866c9ad7dc1c185c
2022-09-27 12:19:33 +08:00
Lan Wei
d5d530ad63 IMA API InputConnection#replaceText()
Add InputConnecton#replaceText(int, int, CharSequence, int, TextAttribute) API to support replacing the specific range in the editor with suggested text.

Add default implementation in InputConnection which delegates to call
finishComposingText()+setSelection()+commitText().

Add implementation in BaseInputConnection which will address the issue
of selection flash.

Bug: 241503197

Test: atest FrameworksCoreTests:BaseInputConnectionTest
Test: atest CtsInputMethodTestCases:BaseInputConnectionTest

Change-Id: Ia0e42270cfae112fcf19205a7151e4ab4ac4afdd
2022-09-23 09:37:00 +08:00
Wilson Wu
3ddfe673e6 Disallow onCreateInputMethodSessionInterface overriden
Add a compatibility change to forbit IMEs to override
the deprecated onCreateInputMethodSessionInterface
method from U.

Bug: 148086656
Test: presubmit
Change-Id: I002c895ce977cf7ed4ef40019c831ccbd9b6450d
2022-09-22 16:53:31 +08:00