Commit Graph

621 Commits

Author SHA1 Message Date
Wilson Wu
7e18e28108 Merge "Refine InputMethodService java doc" 2022-11-03 02:40:39 +00:00
Wilson Wu
53df6b0874 Refine InputMethodService java doc
Add more description to discourage IME developers
not use deprecated onCreateInputMethodInterface and
onCreateInputMethodSessionInterface.

Also move paragraph tag since Block tags should be
put after the main documentation.

Bug: 254278652
Test: presubmit
Change-Id: I729fbe2e1544fc72580a08ccb763ab69c58af92d
2022-11-02 11:29:58 +08:00
Haoyu Zhang
167691ce12 Merge "Introduce TextBoundsInfo" 2022-11-01 05:00:28 +00:00
Yohei Yukawa
e9490612d5 Introduce @hide ParcelableHandwritingGesture
This is a follow up CL to our previous CL [1], which introduced
HandwritingGesture and several subclasses.

One of the challenge is that we only wanted to make child classes
Parcelable, while keeping the abstract base class HandwritingGesture
non-Parcelable.  Although InputConnection has only one method

  InputConnection#performHandwritingGesture(
          HandwritingGesture, ResultReceiver),

in the IPC layer IRemoteInputConnection needed to define methods for
each subclass, e.g.

 * performHandwritingSelectGesture(in InputConnectionCommandHeader,
       in SelectGesture, ...)
 * performHandwritingInsertGesture(in InputConnectionCommandHeader,
       in InsertGesture, ...)
 * performHandwritingDeleteGesture(in InputConnectionCommandHeader,
       in DeleteGesture, ...)

because their abstract class HandwritingGesture is not Parcelable.

To avoid such method definitions (and lots of type-based dispatching /
merging logic), this CL introduces

  ParcelableHandwritingGesture

as a generic Parcelable container of HandwritingGesture subclasses.
You can freely use ParcelableHandwritingGesture in AIDL method
definitions as follows.

  performHandwritingGesture(in InputConnectionCommandHeader,
        in ParcelableHandwritingGesture, ...)

In the Java side, you can easily wrap objects as follows.

  var parcelableGesture = ParcelableHandwritingGesture.of(gesture);
  var unparceledGesture = parcelableGesture.get();

Note that this CL is still an internal cleanup. There must be no
developer observable behavior change.

 [1]: I53bcb62e03ac1c371feb60d1385c88c921754092
      3e3ff1a3d2

Bug: 234882948
Bug: 239783077
Test: presubmit
Test: atest FrameworksCoreTests:ParcelableHandwritingGestureTest
Change-Id: I158026087653f6772a78cc2394e678ae9741fb00
2022-10-31 08:08:54 -07:00
Haoyu Zhang
43aae6ff6d Introduce TextBoundsInfo
This is an API for IME to request text bounds information from editor to
perform handwriting rich gestures. It's useful for IMEs to create custom
handwriting gesture that's not supported the HandwritingGesture. And
it can also be used to provide default rich gesture implementation for
handwriting gesture compatibility.

Bug: 239843501
Test: atest TextBoundsInfoTest
Change-Id: Ia7522371976a4e1fcdff4cc0d244ea427ad3d7c9
2022-10-26 13:51:45 -07:00
Lan Wei
08811aeb33 Fix javadoc for replaceText
Bug: 254278543
Test: atest FrameworksCoreTests:BaseInputConnectionTest
Test: atest CtsInputMethodTestCases:BaseInputConnectionTest
Change-Id: I7429d2b77d9f45c99f0b76808271ba61ad1612ae
2022-10-19 18:10:05 +08:00
Wilson Wu
44d2a22a0e Merge "Mark disallow IME interface compat ID as TestApi" 2022-10-17 07:53:53 +00:00
Taran Singh
4270121357 Merge "Disable InkWindow default animation" 2022-10-13 21:07:40 +00:00
Taran Singh
8b1240b7cf Disable InkWindow default animation
By default all PhoneWindows use Theme_windowAnimationStyle which
animates window up and down when gaining and losing focus respectively.
For InkWindow we don't need any animation.

Fix: 237331144
Fix: 245441579
Bug: 253477462

Test: atest StylusHandwritingTest
Change-Id: I209b03dfbed386a2863921b49d43858037d380b8
2022-10-13 19:03:48 +00:00
Taran Singh
2ae0de31d3 Merge "Remove inkWindow after stylus idle-timeout" 2022-10-11 16:12:14 +00:00
Wilson Wu
9f72b7db1b Mark disallow IME interface compat ID as TestApi
Mark DISALLOW_INPUT_METHOD_INTERFACE_OVERRIDE
comapt change ID as TestApi for CTS testing.

Bug: 148086656
Test: presubmit
Change-Id: I7b56d68cb5429c7076943f3d12b1d19cdf54d26a
2022-10-06 18:11:44 +08:00
TreeHugger Robot
21da3fc4ee Merge "Make RemoteInputConnectionImpl package-private" 2022-09-30 03:45:20 +00:00
Taran Singh
539176516a Merge "Handle non-initalized InkWindow" 2022-09-29 23:58:30 +00:00
Yohei Yukawa
f3023dade4 Make RemoteInputConnectionImpl package-private
This CL mechanically moves RemoteInputConnectionImpl from

  com.android.internal.inputmethod

to

  android.view.inputmethod

just to make it a package-private class, as it is (and should be) used
only by InputMethodManager.

This is a mechanical refactoring.  There must be no developer
observable behavior change in this CL.

Bug: 192412909
Test: presubmit
Change-Id: I2d9bd4a5cb39d506a70331c9c930a74580cdbfa0
2022-09-29 15:38:14 -07:00
Wilson Wu
c14d7e459f Merge "Disallow onCreateInputMethodSessionInterface overriden" 2022-09-29 01:51:10 +00:00
Taran Singh
cfef0232ca Remove inkWindow after stylus idle-timeout
After stylus hasnt been used for handwriting in last X minutes,
remove the the stylus handwriting window.

Bug: 243571274
Test: StylusHandwritingTest
Change-Id: I641be63e028ea9cbfd882ce318d30aad71fd8a4e
2022-09-28 18:53:13 +00:00
Taran Singh
b4fa71513c Handle non-initalized InkWindow
With [1] we changed the lifecycle of InkWindow to be short lived and
missed certain cases.

This also fixes CTS testStylusSession_fingerTriggersNavbarGestures

Bug: 243571274
Test: atest StylusHandwritingTest

[1]: Icd3eea91fe144cff7100d3ecf19191c064c0d196

This is a retry for I1593ff10a5520aadb5ed86a4a09f2405ee65c321

Change-Id: Ibf54599f41ac13121215c5344eb92ecca3259db1
2022-09-28 18:53:10 +00:00
Wilson Wu
32950c05c3 Merge "Fix potential failures in InputMethodService (2nd try)" 2022-09-27 07:59:47 +00:00
Wilson Wu
4bcdf9b201 Fix potential failures in InputMethodService (2nd try)
It had the chance for calling into IMS but IMS
already destroyed.

Ideally we should suppress any callbacks after
InputMethodService#onDestroy() and log them.

First try is CL[1], but it wrongly add the API to
IInputMethod. Using the InputMethodServiceInternal
is make more sense for IInputMethodWrapper to
communicate with InputMethodService.

[1]: I10f26a92e739b60c5b8010e8476f8f5a403e0a7b

Bug: 121269861
Bug: 211062619
Bug: 148086656
Test: presubmit
Change-Id: I7b4f34eb38f44dea6872eb46866c9ad7dc1c185c
2022-09-27 12:19:33 +08:00
Lan Wei
d5d530ad63 IMA API InputConnection#replaceText()
Add InputConnecton#replaceText(int, int, CharSequence, int, TextAttribute) API to support replacing the specific range in the editor with suggested text.

Add default implementation in InputConnection which delegates to call
finishComposingText()+setSelection()+commitText().

Add implementation in BaseInputConnection which will address the issue
of selection flash.

Bug: 241503197

Test: atest FrameworksCoreTests:BaseInputConnectionTest
Test: atest CtsInputMethodTestCases:BaseInputConnectionTest

Change-Id: Ia0e42270cfae112fcf19205a7151e4ab4ac4afdd
2022-09-23 09:37:00 +08:00
Wilson Wu
3ddfe673e6 Disallow onCreateInputMethodSessionInterface overriden
Add a compatibility change to forbit IMEs to override
the deprecated onCreateInputMethodSessionInterface
method from U.

Bug: 148086656
Test: presubmit
Change-Id: I002c895ce977cf7ed4ef40019c831ccbd9b6450d
2022-09-22 16:53:31 +08:00
Wilson Wu
fe8bf190f9 Merge "Revert "Fix potential failures in InputMethodService"" 2022-09-19 09:13:38 +00:00
Wilson Wu
1e5f9b6bfd Revert "Fix potential failures in InputMethodService"
This reverts commit ce98507bf3.

Reason for revert: b/247426953

Change-Id: If7b7a258150f8f57a7b4a88e4926d412f948fd9c
2022-09-19 08:55:19 +00:00
Wilson Wu
5f931ecef4 Merge "Deprecate InputMethodInterface methods" 2022-09-19 07:25:56 +00:00
TreeHugger Robot
306f2a61cb Merge "Fix potential failures in InputMethodService" 2022-09-19 01:58:35 +00:00
Wilson Wu
0899552556 Deprecate InputMethodInterface methods
IME developers have no strong reason to override
InputMethodImpl and InputMethodSessionImpl methods.

Add deprecated annotation to these methods with
suggested alternatives.

Bug: 148086656
Test: presubmit
Change-Id: I4e970404a1cbf15d6b40760b7808825c0eb94736
2022-09-16 19:01:10 +08:00
Taran Singh
85a57bc143 Introduce Scribe Select/DeleteRangeGesture
Add new APIs for select and delete with two rectangles.

Test: atest InputConnectionEndToEndTest
Test: atest StylusHandwritingest
Bug: 239783077

Change-Id: Ia7240d3eaac8610c774c3a4aad074a4b84b71521
2022-09-15 00:36:40 +00:00
Wilson Wu
ce98507bf3 Fix potential failures in InputMethodService
It had the chance for calling into IMS but IMS
already destroyed.

Ideally we should suppress any callbacks after
InputMethodService#onDestroy() and log them.

Bug: 121269861
Bug: 211062619
Bug: 148086656
Test: presubmit
Change-Id: I10f26a92e739b60c5b8010e8476f8f5a403e0a7b
2022-09-14 12:49:26 +08:00
Justin Ghan
d12fdde387 APIs for remove space gesture, and join or split gesture
Bug: 240436909
Test: atest InputConnectionEndToEndTest
Change-Id: I2f8c511748c9de3a25bd6a17fcf33cc5017dd0e8
2022-09-12 11:13:11 -07:00
Taran Singh
ca10f8bb95 Merge "Incorporate API feedback for Companion widget" 2022-08-29 19:26:58 +00:00
Taran Singh
4b5d696935 Add consumer callback in performHandwritingGesture()
performHandwritingGesture should take an optional IntConsumer
& executor to provide result of Gesture operation back to IME.

Bug: 210039666
Bug: 239783077
Test: atest InputConnectionEndToEndTests

Change-Id: I2ca7eaa182159b7ffef0812d970512ffcf817ce2
2022-08-25 13:58:44 -07:00
Taran Singh
28dd0b1b5a Incorporate API feedback for Companion widget
API feedback:
1. use IntDef for toolType
2. Add javadoc in onUpdateEditorTooltip describing its called after
   onStartInput.

Fix: 241206150
Bug: 233108064
Test: atest CtsInputMethodTestCases
Change-Id: I95bccd992690af5c160277b88a854d8f032278aa
2022-08-25 20:36:31 +00:00
Taran Singh
7a94646a65 Merge "Improve handwriting Ink window lifetime" 2022-08-25 04:37:22 +00:00
Taran Singh
f092a7cb23 Improve handwriting Ink window lifetime
1. When the system removes the last known stylus device
and has no other stylus available, remove the Ink window.

2. Create InkWindow only if the device+IME combination supports
handwriting and the stylus is being used.

Bug: 243571274
Bug: 239121683
Bug: 210039666

Test: atest StylusHandwritingTest
Change-Id: Icd3eea91fe144cff7100d3ecf19191c064c0d196
2022-08-24 00:24:04 +00:00
Taran Singh
3e3ff1a3d2 Introduce Scribe Rich gestures API
During a stylus handwriting session, user can perform a
stylus gesture operation like  Select, delete, insert on an
area of text. IME needs APIs to perform these gestures on text.
This CL introduces API signature for Select, delete, insert gestures.

Design doc: go/scribe-gestures-api

Bug: 239783077
Bug: 210039666
Test: Manually build & compile HandwritingIme

Change-Id: I53bcb62e03ac1c371feb60d1385c88c921754092
2022-08-22 14:22:47 -07:00
Taran Singh
e5f499176d Introduce API to support Stylus Companion widget
Introduce APIs necessary for IMEs to be able to distinguish when to show
virtual keyboard vs companion widget based on toolType.

IME receives click toolType in:
1. IMS#onUpdateEditorToolType(int toolType) for every showSoftInput()
   called from IMM.
2. Initial value of toolType in EditorInfo#getInitialToolType().

Fix: 233108064
Bug: 210039666

Test: atest StylusHandwritingTest

Change-Id: I91ea9204ef62697b8942b0f350169382f18b3278
2022-08-03 01:19:22 +00:00
Shan Huang
0ac31bf658 Merge "Remove TODOs for adding CTS tests." 2022-07-23 19:10:52 +00:00
Kohsuke Yatoh
6bf7692737 Fix typo in getCandidatesHiddenVisibility Javadoc.
It always return GONE or *IN*VISIBLE, not VISIBLE.

Change-Id: Id2e51b4525ca087f5c50ceb7575f3270e97b0138
Test: n/a
2022-07-07 20:04:43 +00:00
Yohei Yukawa
45225d29e1 Introduce IInputMethodManager.InitParams
IInputMethod#initializeInternal() is a hot-spot where we keep adding
new parameters recently.  Introducing a dedicated parameter object
would allow us to add further parameters with fewer lines of changes.

This is a mechanical refactoring. There should be no developer
observable behavior change.

Bug: 234882948
Test: presubmit
Change-Id: I98a5424385804ec190885ba53973935f9db66b88
2022-06-29 20:54:41 -07:00
Yohei Yukawa
bb00fe959a Introduce IInputMethodManager.StartInputParams
IInputMethod#startInput() is a hot-spot where we keep adding new
parameters recently.  Introducing a dedicated parameter object would
allow us to add further parameters with fewer lines of changes.

This CL also renames

  InputMethod#dispatchStartInputWithToken

to

  InputMethod#dispatchStartInput

for consistency.

This is a mechanical refactoring. There should be no developer
observable behavior change.

Bug: 234882948
Test: presubmit
Change-Id: I4a6af9a47db04335921a7d14efe91bfeff25c56d
2022-06-29 12:09:22 -07:00
Yohei Yukawa
55b543e1e8 Do some actions eagerly for IInputMethod#startInput()
This CL moves some operations from DO_START_INPUT handler to
IInputMethod#startInput() binder callback, which should be safe
because calling EditorInfo#makeCompatible() and instantiating
RemoteInputConnection are both OK to be done eagerly.

There should be no developer observable behavior change in this CL.

Bug: 234882948
Test: presubmit
Change-Id: I69e8bc5515d09824ed19e68f585590447e130a38
2022-06-28 21:32:57 -07:00
Yohei Yukawa
d4bff9066c Use consistent parameter/variable name for EditorInfo
There are several places where EditorInfo is still called "attribute"
in our code base.  Let's call them "editorInfo" or something like that
for better readability.

This is just a mechanical code clean-up.  There should be no behavior
change and compability concern.

Fix: 237008479
Test: presubmit
Change-Id: Idd147940c34ab5fd39b2d11b2acb1554c7d8f26d
2022-06-23 15:57:39 -07:00
Yohei Yukawa
3a10176c60 Fix minor style issues in InputMethodService
This is a minor follow up CL to our previous CL [1], which migrated
IME to handle back with OnBackInvokedDispatcher.

Having SomeArgs within SomeArgs doesn't make sense unless we really
run out of arguments.

This CL also addresses minor style issues.

Basically this is a mechanical clean-up that has no behavior change.

 [1]: Ie207b63b11a56c9b2173f26b734a27b13ebccc60
      a6666f2221

Bug: 228358882
Bug: 234882948
Test: presubmit
Change-Id: Ie14f66be0c9d66d221c5f9fae244df5487d65593
2022-06-22 18:52:04 +00:00
Yohei Yukawa
2cc4677f2c Revert "Don't send the same values to onUpdateSelection repeatedly"
This reverts commit c743cb9477.

Reason for revert:
This workaround is no longer necessary [1][2].

 [1]: I56f3198e54f689adc9541f2c2644ee19bcc6deed
      e345e90a01
 [2]: Ic84fb070e0ff21dffb7e6c9bbdc6320eb5946fd9
      1046a22aae6d6c0ee21aca7446540a9d27902559

Fix: 183474466
Test: atest CtsInputMethodTestCases:InProcessImeTest
Test: atest CtsAccessibilityServiceTestCases:AccessibilityImeTest#testOnUpdateSelectionForInProcessA11yIme
Change-Id: Ie5d00bbc51b66f1d6b50969f03d1b3e3bca5e9da
2022-06-21 04:58:14 +00:00
Shan Huang
223bec74fc Remove TODOs for adding CTS tests.
Bug: 232341407
Bug: 232845902
Test: atest KeyboardVisibilityControlTest#testHideImeAfterBackPressed_ScreenOffOn --no-bazel-mode
Test: atest KeyboardVisibilityControlTest#testNonImeFocusablePopupWindow_onTopOfIme --no-bazel-mode

Change-Id: I36ce1846b6f1cfdbb8a9679306966a205842b3ff
2022-06-17 04:27:31 +00:00
Vadim Caen
aa48dc3c2d Prevent callback registration when back flag is disabled
When the enableOnBackInvokedCallback is set to false (or not set),
registering an OnBackInvokedCallback should be a no-op to avoid
overriding the default compat callback.

Test: Manual testing registering a callback on an app with the flag
disabled and doing a back gesture. Currently we don't have test
executing a back gesture so automated tests are not possible

Bug: 235206960

Change-Id: I54d843f11130a78ed5a68cbe4722e601a2086ee1
2022-06-16 12:41:46 +00:00
Taran Singh
83a3a1f847 Merge "Introduce IME API for adjusting handwriting idle timeout" 2022-06-14 17:44:18 +00:00
Yohei Yukawa
2fddead57d Move IRemoteInputConnectionInvoker to android.inputmethodservice
IRemoteInputConnectionInvoker is used only from RemoteInputConnection,
which is already placed under android.inputmethodservice as a
package-private class.  Let's also move IRemoteInputConnectionInvoker
to android.inputmethodservice as a package-private class to minimize
its visibility.

This is a mechanical refactoring, which should have no user/developer
observable behavior change.

Bug: 234882948
Test: presubmit
Change-Id: Ie946993aeedc31a67e3074dc591245f75143a9f1
2022-06-13 18:28:40 -07:00
Taran Singh
f42da45849 Introduce IME API for adjusting handwriting idle timeout
The API allows IME developers to override default system idle-timeout
for stylus handwriting sessions. The system default may be too long or
too short for certain use-cases and it makes sense to let IME change it.

Bug: 233811329
Test: atest StylusHandwritingTest
Change-Id: I9e869658ad57b2ec91b882ab9e003e7f60f95c7c
2022-06-13 23:06:22 +00:00
Yohei Yukawa
bd8375b35a Move *InlineSuggestions* to com.android.internal.inputmethod
This CL moves 4 files that match *InlineSuggestions* from

  com.android.internal.view

to

  com.android.internal.inputmethod

as they are basically used only by IME classes (and Autofill).

Hopefully this gives better code organization and maintainance.

This is a mechanical refactoring of internal details.  There should be
no user/developer observable behavior change in this CL.

Bug: 234882948
Test: presubmit
Change-Id: Iac696ae501953ab6a90d99112a62221454cd76e4
2022-06-10 11:35:05 -07:00