performHandwritingGesture should take an optional IntConsumer
& executor to provide result of Gesture operation back to IME.
Bug: 210039666
Bug: 239783077
Test: atest InputConnectionEndToEndTests
Change-Id: I2ca7eaa182159b7ffef0812d970512ffcf817ce2
API feedback:
1. use IntDef for toolType
2. Add javadoc in onUpdateEditorTooltip describing its called after
onStartInput.
Fix: 241206150
Bug: 233108064
Test: atest CtsInputMethodTestCases
Change-Id: I95bccd992690af5c160277b88a854d8f032278aa
1. When the system removes the last known stylus device
and has no other stylus available, remove the Ink window.
2. Create InkWindow only if the device+IME combination supports
handwriting and the stylus is being used.
Bug: 243571274
Bug: 239121683
Bug: 210039666
Test: atest StylusHandwritingTest
Change-Id: Icd3eea91fe144cff7100d3ecf19191c064c0d196
During a stylus handwriting session, user can perform a
stylus gesture operation like Select, delete, insert on an
area of text. IME needs APIs to perform these gestures on text.
This CL introduces API signature for Select, delete, insert gestures.
Design doc: go/scribe-gestures-api
Bug: 239783077
Bug: 210039666
Test: Manually build & compile HandwritingIme
Change-Id: I53bcb62e03ac1c371feb60d1385c88c921754092
Introduce APIs necessary for IMEs to be able to distinguish when to show
virtual keyboard vs companion widget based on toolType.
IME receives click toolType in:
1. IMS#onUpdateEditorToolType(int toolType) for every showSoftInput()
called from IMM.
2. Initial value of toolType in EditorInfo#getInitialToolType().
Fix: 233108064
Bug: 210039666
Test: atest StylusHandwritingTest
Change-Id: I91ea9204ef62697b8942b0f350169382f18b3278
IInputMethod#initializeInternal() is a hot-spot where we keep adding
new parameters recently. Introducing a dedicated parameter object
would allow us to add further parameters with fewer lines of changes.
This is a mechanical refactoring. There should be no developer
observable behavior change.
Bug: 234882948
Test: presubmit
Change-Id: I98a5424385804ec190885ba53973935f9db66b88
IInputMethod#startInput() is a hot-spot where we keep adding new
parameters recently. Introducing a dedicated parameter object would
allow us to add further parameters with fewer lines of changes.
This CL also renames
InputMethod#dispatchStartInputWithToken
to
InputMethod#dispatchStartInput
for consistency.
This is a mechanical refactoring. There should be no developer
observable behavior change.
Bug: 234882948
Test: presubmit
Change-Id: I4a6af9a47db04335921a7d14efe91bfeff25c56d
This CL moves some operations from DO_START_INPUT handler to
IInputMethod#startInput() binder callback, which should be safe
because calling EditorInfo#makeCompatible() and instantiating
RemoteInputConnection are both OK to be done eagerly.
There should be no developer observable behavior change in this CL.
Bug: 234882948
Test: presubmit
Change-Id: I69e8bc5515d09824ed19e68f585590447e130a38
There are several places where EditorInfo is still called "attribute"
in our code base. Let's call them "editorInfo" or something like that
for better readability.
This is just a mechanical code clean-up. There should be no behavior
change and compability concern.
Fix: 237008479
Test: presubmit
Change-Id: Idd147940c34ab5fd39b2d11b2acb1554c7d8f26d
This is a minor follow up CL to our previous CL [1], which migrated
IME to handle back with OnBackInvokedDispatcher.
Having SomeArgs within SomeArgs doesn't make sense unless we really
run out of arguments.
This CL also addresses minor style issues.
Basically this is a mechanical clean-up that has no behavior change.
[1]: Ie207b63b11a56c9b2173f26b734a27b13ebccc60
a6666f2221
Bug: 228358882
Bug: 234882948
Test: presubmit
Change-Id: Ie14f66be0c9d66d221c5f9fae244df5487d65593
When the enableOnBackInvokedCallback is set to false (or not set),
registering an OnBackInvokedCallback should be a no-op to avoid
overriding the default compat callback.
Test: Manual testing registering a callback on an app with the flag
disabled and doing a back gesture. Currently we don't have test
executing a back gesture so automated tests are not possible
Bug: 235206960
Change-Id: I54d843f11130a78ed5a68cbe4722e601a2086ee1
IRemoteInputConnectionInvoker is used only from RemoteInputConnection,
which is already placed under android.inputmethodservice as a
package-private class. Let's also move IRemoteInputConnectionInvoker
to android.inputmethodservice as a package-private class to minimize
its visibility.
This is a mechanical refactoring, which should have no user/developer
observable behavior change.
Bug: 234882948
Test: presubmit
Change-Id: Ie946993aeedc31a67e3074dc591245f75143a9f1
The API allows IME developers to override default system idle-timeout
for stylus handwriting sessions. The system default may be too long or
too short for certain use-cases and it makes sense to let IME change it.
Bug: 233811329
Test: atest StylusHandwritingTest
Change-Id: I9e869658ad57b2ec91b882ab9e003e7f60f95c7c
This CL moves 4 files that match *InlineSuggestions* from
com.android.internal.view
to
com.android.internal.inputmethod
as they are basically used only by IME classes (and Autofill).
Hopefully this gives better code organization and maintainance.
This is a mechanical refactoring of internal details. There should be
no user/developer observable behavior change in this CL.
Bug: 234882948
Test: presubmit
Change-Id: Iac696ae501953ab6a90d99112a62221454cd76e4
This CL Moves IInputMethod from
com.android.internal.view
to
com.android.internal.inputmethod
to be consistent with other IME-related internal files.
This is a mechanical refactoring of internal details. There should be
no user/developer observable behavior change in this CL.
Bug: 234882948
Test: presubmit
Change-Id: I1a43c2ad1f932ecb9201b3958becb475fb2d718d
This CL Moves IInputMethodSession from com.android.internal.view to
com.android.internal.inputmethod to be consistent with other
IME-related files.
This CL also moves IInputSessionCallback to
com.android.internal.inputmethod with renaming it to
IInputMethodSessionCallback for consistency.
This is just a renaming CL of implementation details. There should be
no user/developer observable behavior change in this CL.
Bug: 234882948
Test: presubmit
Change-Id: I87495cbd482dd55005ac8d948f57f32bb016c7d6
IInputContext is the IPC interface name used behind InputConnection,
which was named in the initial development of IME API for Android 1.5.
To make it more descriptive and consistent with other classes such as
RemoteInputConnection, let's rename it to IRemoteInputConnection.
This CL also renames
IInputContextInvoker
to
IRemoteInputConnectionInvoker
for consistency.
This is a rename of internal IPC interface. There should be no
user/developer observable impact.
Bug: 234882948
Test: presubmit
Change-Id: Iae82d5a26f0746161c2bf91a421b97f0f238aafd
switch.
This can happen when one navigates from an activity showing IME
to another activity requesting to show IME. The following sequence would
happen:
- IMM of the current activity receives an UNBIND message with reason CLIENT_SWITCH
- A new IMM is created for the new activity requesting input and
requests to bind.
- The new IMM requests to start input, triggering
IMS#dispatchStartInputWithToken with a *new* ImeOnBackInvokedDispatcher instance than the one previously kept by IMS.
Previously, the ime dispatcher instance was updated before the existing
back callback was unregistered and caused the unregister call to fail.
The fix is to update the dispatcher after the other logic in
IMS#dispatcherStartInputWithToken (specifically, to after startInput(),
which calls doFinishInput() if input is already started).
Bug: 234416098
Test: atest CtsInputMethodTestCases
Change-Id: Ic7a5daab8709b808343388b095b5b6ad40adc777
During Handwriting session if Stylus is inactive for X seconds, and IME fails
to finish handwriting session, framework will idle-timeout after few seconds.
Bug: 233811329
Test: Manually
Change-Id: Ice4441819976481d05cf320fb2438c8212a212e9
Use explicit Handler callback removal from handler so we dont remove all
posted methods from Handler.
There should be no behavior change.
Bug: 167948419
Bug: 167948123
Bug: 233811329
Test: atest CtsInputMethodTestCases
Change-Id: I9564f5aa2430e5438b6a3d0a25967ff10661822a
EditorInfo.extras may contain arbitrary user-generated contents,
such as
androidx.core.view.inputmethod.EditorInfoCompat.CONTENT_SURROUNDING_TEXT.
We should not include such data in bug reports.
This CL does not change the behavior of the public API
(EditorInfo#dump()), in case any app depends on it.
Bug: 233405987
Test: atest FrameworksCoreTests:android.view.inputmethod.EditorInfoTest
Test: atest CtsInputMethodTestCases:android.view.inputmethod.cts.DumpTest
Change-Id: If3785e91c1aa5ce624a331a25035b7541d012d48
There exists a configuration value to determine if the nav bar should
be shown when the keyboard is open. The InsetsPolicy should respect this
value. This value is currently used on automotive devices but doesn't
necessarily need to be restricted to a particular device type.
Bug: 229256359
Test: manual
Change-Id: I57ea71193476cb88bba8d406d039fe08e05a6b1c
We currently close the IME by having the target application forward KEYCODE_BACK to the IME process through InputMethodManager#dispatchInputEvent and having the IME handle the keycode in InputMethodService#onKeyDown. When apps opt in to OnBackInvokedDispatcher API, we will not dispatch KEYCODE_BACK to apps anymore. Thus we need to migrate IME to the new API for it to close on back invocation.
This implementation forwards OnBackInvokedCallbacks from the IME process
to the app process. This is necessary because all callbacks need to
exist in the app process for them to be considered by hardware back keys. While back gestures go through WM to resolve callbacks from the focused window, hw keys are directly sent to the focused window's ViewRootImpl, bypassing server side back nav logic.
Bug: 228358882
Test: atest CtsInputMethodTestCases:KeyboardVisibilityControlTest
Test: atest CtsInputMethodTestCases:InputMethodServiceTest
Test: atest CtsInputMethodTestCases
Change-Id: Ie207b63b11a56c9b2173f26b734a27b13ebccc60
Shorten IME surface caching so that we can avoid showing cached IME
surface when IME insets have changed from the last time it was shown.
Fix: 221458652
Bug: 230762351
Test: Manually using steps:
1. Go to gboard settings and enable Emoji bar
2. Open a chat in messages app
3. tap editor and make sure Insets dont change after show
animation completes.
Change-Id: I4c2a9c272ef0e7be7128ca64abebd3a3b7d2d7a4
As ViewTreeObserver.InternelInsetsInfo.touchableRegion with setting
TOUCHABLE_INSETS_REGION requires setting the region that relative
the window position, but CL[1] uses the global screen position to
calculate the touch region, which will affect the end result,
To fix this, we should use getLocationInWindow to get the relative
position of the window.
[1]: I0fe54efac80dd0d55f4ba37cfa7d7188b642abb0
Fix: 229562319
Test: manually verified with the IME mentioned in the bug.
Test: ensure Bug 226566506 is verified.
Change-Id: I9e3fe45dccb263445d378b5ac17b0e450bdb1c24
Currently, IMMS will be notified asynchronously when an IME hide
animation finishes, via message dispatching through IMS
(IMM#notifyImeHidden -> IMS#notifyImeHidden -> IMMS#hideMySoftInput).
This creates a race condition when IMM#showSoftInput or WIC#show is
called around the end of hide animation.
This CL fixes the race condition by synchronously and directly
calling IMMS#hideSoftInput from IMM#notifyImeHidden.
Note that there is still another race condition for IMM#showSoftInput
(not WIC#show) if it's called during an IME hide animation;
IMM#showSoftInput ended up calling WIC#show asynchronously, but at that
time the running IME hide animation may have already been finished
successfully and WIC#show may fail to cancel the hide animation
(then the cleanup IMM#notifyImeHidden hides the IME again disruptively).
I will fix the latter issue in a separate CL.
Bug: 221483132
Bug: 225674038
Test: atest InputMethodStressTest
Test: atest CtsInputMethodTestCases
Test: atest WindowInsetsAnimationControllerTests
Change-Id: I7c71dc5a1d6b61aa79d1666f0e257e6401e4adb2
(cherry picked from commit 9065310f81)
Merged-In: I7c71dc5a1d6b61aa79d1666f0e257e6401e4adb2
Currently, IMMS will be notified asynchronously when an IME hide
animation finishes, via message dispatching through IMS
(IMM#notifyImeHidden -> IMS#notifyImeHidden -> IMMS#hideMySoftInput).
This creates a race condition when IMM#showSoftInput or WIC#show is
called around the end of hide animation.
This CL fixes the race condition by synchronously and directly
calling IMMS#hideSoftInput from IMM#notifyImeHidden.
Note that there is still another race condition for IMM#showSoftInput
(not WIC#show) if it's called during an IME hide animation;
IMM#showSoftInput ended up calling WIC#show asynchronously, but at that
time the running IME hide animation may have already been finished
successfully and WIC#show may fail to cancel the hide animation
(then the cleanup IMM#notifyImeHidden hides the IME again disruptively).
I will fix the latter issue in a separate CL.
Bug: 221483132
Bug: 225674038
Test: atest InputMethodStressTest
Test: atest CtsInputMethodTestCases
Test: atest WindowInsetsAnimationControllerTests
Change-Id: I7c71dc5a1d6b61aa79d1666f0e257e6401e4adb2
This reverts commit 38be9e32f1.
Reason for revert: Caused b/229642813 and b/229355440
Bug:229642813
Bug:229636388
Change-Id: I65e4f66ec2156d4e0812753d71ac82afd965fe3b
This is a follow up CL to our previuos CL [1], which enabled
AccessibilityService to use a subset of InputConnection APIs.
In that CL we have reused existing AIDL interfaces that were designed
and maintained for IMEs for simplicity, where a non trivial amount of
unnecessary IPC endpoints were included.
From the security and maintainability viewpoints, however, exposing
unnecessary IPC endpoints is discouraged in general. To address such
concerns this CL introduces a set of dedicated IPC definitions for
A11yIME so that we do not need to reuse IPCs for IMEs.
This CL also stops passing InputBinding object to A11yIME process as
it contains IInputContext Binder Proxy, which can still be used to
directly invoke fallback InputConnection. This is doable now because
A11yIME no longer relies on fallback InputConnection [2].
This CL is should not have any observable changes in the semantics.
End-to-end CTS tests guarantee that everything is still working as
intended now and in the future.
[1]: Ia651a811093a939d00c081be1961e24ed3ad0356
fb17e5ae7a
[2]: I2af3cd50444d8ddf25aa0f6479238156914e6fff
dc635efb68
Fix: 215633021
Fix: 215636776
Test: atest CtsInputMethodTestCases:AccessibilityInputMethodTest
Test: atest CtsAccessibilityServiceTestCases:AccessibilityInputConnectionTest
Test: atest CtsAccessibilityServiceTestCases:AccessibilityImeTest
Change-Id: I5ff2e804cbcf90828370a0612ff54111130bdff4
