Fixes a flicker that occurs during transitions between windows.
This happens for two reasons:
1.) Control is immediately transferred to the new window, and the
previous window didn't get a chance to play the animation.
This is addressed by adding logic to keep control on the
exiting window for the duration of the transition - similar to
what we do with the target for z-ordering purposes.
2.) Upon the input connection being severed, the InputMethodService
immediately hides its window, preventing any animations whenever
the input connection changes
This is addressed by moving hiding of the surface into the
controlling windows - where upon receiving control, we now
trigger removal of the IME surface if we don't show it.
Additionally:
- Now ensures that any requests from the ImeInsetsSourceConsumer
ensure that they come from the window that is currently served
by IMM.
- Removes the transparancy clause from isImeTargetFromDisplayContentAndImeSame
to match the updated IME target computation in DisplayContent in [1].
[1]: Iedd5f7407926167f4891ce9b7e9a79e22751e668
Fixes: 153145997
Fixes: 150902448
Test: atest WindowInsetsAnimationControllerTests
Test: atest DisplayContentTests InsetsSourceConsumerTest
Test: Open app with IME, press HOME button, verify IME smoothly animates away
Test: Open Messages, open a thread, open IME. Click search icon, verify IME opens in the search activity
Change-Id: I4910c2a06cc67b0470477b245fc1de54b75f10f9
The current implementation has a problem where:
In the autofill side, there can be multiple autofill sessions existed
at the same time. But in the IME side, there is always only one
InlineSuggestionSession at any given time. It will cause the previous
autofill session to fail communication with IME.
It would better change to drop down UI when the autofill inline
suggestions don't be shown in IME.
How to reproduce this issue:
To add an input field with autofillable into the authentication activity
of InlineFillService. To tap on the input field of the authentication
activity during the authentication flow. After completing the
authentication, the inline suggestions won't be shown in IME.
BTW, if the input field is marked as non-autofillable, this issue won't
occur.
Manual verification:
1.Tested this patch with InlineFillService and it worked well.
2.Feng also helped to test this patch with the webview
(sort of randomly), and didn't find any broken case
Bug: 158877106
Test: atest CtsInputMethodTestCases
Test: atest CtsAutoFillServiceTestCases
Test: new CTS test Ie1d9055b0eabfcaa00861869467be8dcee25833e
Test: manual verification with InlineFillService
Test: Feng also helped to test this patch with the webview
Change-Id: Ib06edd823fa4478f34362164f3f7dd3544e51705
* In case where there are multiple input fields in a single WebView,
switching focuses between input fields doesn't trigger an onFinishInput
from the previous field. But it always triggers an onStartInput in
the new field. So it's safer to clear the suggestions onStartInput
Test: atest android.autofillservice.cts.inline
Test: atest CtsInputMethodTestCases
Bug: 159479887
Bug: 157515522
Change-Id: Ie380db855fbc93600635790ef5adc1031d6f0787
* The problem with sending empty response to IME is that the IME
may want to handle the following two cases differently:
a) all suggestions are filtered out due to user typing: ime may
want to immediately delete the existing suggestions to make
place for other types of things, such as IME's own word
completion or next word prediction.
b) the current input connection is finished and a new connection
will be created with the same field or a different field:
ime may want to delay removing the suggestions so that if there
is new inline suggestions coming soon after for the next
connection, the UI transition can be smoothed out by skipping
the gap of deleting the old suggestions and showing the new
suggestions.
* We used to rely on the IME impl to clear the suggestions when input
is finished. That was done to give the IME the flexibility to
smooth out the UI updates. Otherwise in case the input connection
is finished and immediately started again on the same field,
and there is another non-empty suggestion coming after short after,
it would cause UI flicker. Because the suggsetion chips would
disappear for a short moment and then appear again.
* The previously implemented solution was to have the IME impl post a
delayed deletion of the suggestions when onFinishInput is called.
* In this patch, we get around this issue by synchronously clearing
the inline suggestions right before the onFinishInput. Then the
IME impl can post a callback to the main thread to do the actual
delection. And in the callback it can check whether onFinishInput
and onStartInput was called right before to determine whether
it needs to delay the delection or delete immediately.
* Also done in this patch is to clear existing inline suggestions,
if any, before IME creating a new callback connection to the
framework.
Test: atest android.autofillservice.cts.inline
Bug: 157515522
Change-Id: I6fd5d294cf8676a24b8576ea554824608672ce49
Floating IME or fullscreen IME won't cause insets (except the area
overlapped with navigation bar). It doesn't make much sense to let
apps move the IME at these cases.
Fix: 157777145
Test: atest InsetsSourceConsumerTest GlobalActionsImeTest
ImeInsetsControllerTest
Change-Id: Id70f59be7653beedc02d6c8bc3b1bd50a357f4fe
IMS handles configuration and display changes from the server side
and render views to inteact with users. Thus IMS should be marked as
an UI context
fixes: 157027563
Test: atest ContextTest
Change-Id: I0a2307c4764acf8b9fc0254a9ee2fc8a344bb7ef
This metrics now is used for calculating the value reported by
Display#getSize. It could be extended to have more function later.
Also replace getCurrentMetrics usages to make them report
Display#getSize value.
Bug: 148904274
Test: atest WindowMetricsHelperTest
Change-Id: I182b6e63f7c28752eee22839cb9e69f073046c3f
Adds filter to prevent passing binder objects in the extras and styling
APIs for inline suggestions. Avoids these to be used to send over remote
objects for Autofill provider and IME to communicate through private APIs
bypassing the OS.
Bug: 152316540
Test: atest CtsAutoFillServiceTestCases
Test: atest InlineSuggestionsRequestTest
Change-Id: I00940a845ed0e2546cbe09fa36e2439869c2f783
This is a follow up CL to our previous CL [1], which implemented
fail-fast mode for blocking InputConnection APIs based on
IInputMethod.unbindInput() async signal from IMMS to IMS.
What was not implemented in the previous CL was a mechanism to
immediately unblock a sync InputConnection API call that is already
requested to the IME client process and waiting for its response.
With this CL, any blocking InputConnection API fails immediately when
IInputMethod#onUnbindInput() is delivered to the IME process, without
waiting for the full time-out period (MAX_WAIT_TIME_MILLIS == 2 sec)
to pass.
Implementation Note:
The key idea is to use CountDownLatch to compose multiple wait
conditions.composed wait condition. The CountDownLatch is initialized
with 1 then will be decremented when:
A. received a result from the IME client
B. received IInputMethod.unbindInput()
Hence InputConnectionWrapper can simply wait for the CountDownLatch to
become 0 with an existing timeout (MAX_WAIT_TIME_MILLIS) then returns
failure unless the CountDownLatch became 0 because of A.
[1]: I0f816c6ca4c5c0664962432b913f074605fedd27
1d113d041f
Fix: 36897707
Test: atest InputConnectionBlockingMethodTest
Test: InputConnectionBlockingMethodTest#*FailFastAfterUnbindInput()
take shorter time to complete.
Test: Monitor logcat with `adb logcat -s InputConnectionWrapper:*`
while running `atest InputConnectionBlockingMethodTest`
Change-Id: Ic65a95eb5d0fd56f505a02fd9083bcf6694b6734
* In IME side, wait for the input start before calling back to Autofill,
rather than returning inline unsupported immediately.
* Also adds an InlineSuggestionManager to simplify code in the
InputMethodService
Test: atest CtsAutoFillServiceTestCases
Test: atest CtsInputMethodTestCases
Bug: 151123764
Change-Id: I199925d77aa508f259e98a8929120aeb96015b57
And let IME always extend into the status bar area so that the position
of IME can never affected by status bar. This can prevent flicker during
IME animation.
This CL also makes PerDisplay can restart the animation while the insets
source control is changed during animation.
Fix: 151759336
Test: atest WindowStateTests
Change-Id: Ic2a308e6b7ec39b4b8645751e31addd26ddf3735
Use case: Jetpack WM will use them to get the location of windows on
screen and compute the display feature positions in window coordinate
space.
Bug: 150908045
Test: atest FrameworksCoreTests:WindowMetricsTest
Test: atest CtsWindowManagerDeviceTestCases:WindowMetricsTests
Change-Id: Ia08950cd5df35971408e8b17bb27d97d29d0ab9b
Exempt-From-Owner-Approval: API change
Insets API notifies IME when it's hidden. however, IMMS never
really received these state changes. Using requestHideSelf() makes sure
IMMS is in-sync.
Fix: 151980214
Test: manually using the steps in bug
Change-Id: I7f6098a61a5942795ffd33a60329e4dd5fb5d6cb
IME process is notified after client has hidden the IME. At this point
IME should call doHideWindow() so internal lifecycle methods are called
the way they were called earlier.
Also, with insets we don't hide the decorView of IME window till client
is unbound, so to be consistent with previous behavior, call
windowVisibilityChanged(View.GONE) explicitly.
Fix: 151125925
Test: atest KeyboardVisibilityTest InputMethodServiceTest
Change-Id: Ibe9a02b93489370b6ae9b497993e366f88791a62
IME is not big enough to be the background of navigation bar when
navigation bar is on the left or right edge of the screen. Also,
IME can be on top of navigation bar in landscape split-screen mode,
we don't want IME to block touches on navigation bar.
Fix: 151083985
Test: 1. Open Messages.
2. Rotate device to landscape.
3. Click on "Search message" and see if IME extends into
the navigation bar area at the left or right edge.
Change-Id: I0ef3d6379a9ae52b3749154d2fdc54e9aa94a9e0
It's useful for apps to know which inset types they can currently
control, as otherwise they have to poll by calling
controlInsetsAnimation repeatedly.
This can be used when apps want to apply a custom animation
immediately during startup as soon as possible.
Fixes: 150780468
Test: InsetsControllerTest
Test: CTS will be added soon
Test: WindowInsetsActivity
Change-Id: Ic0388c11d759843d3ac9edd8ef23904c9ce05c46
Previous attempt to remove IME surface [1] works when IME animates out
within same app window. However, if IME doesn't animate or window hides
without hiding keyboard, surface would still be around in memory.
With this CL, IME surface will be hidden moment it unbinds from the
window.
[1] I6c2a13dd40b22d3fd62b4763ba369992e5ff1138
Bug: 150163310
Test: Manually:
1. Open any app that has editor and tap on editor to show IME
adb shell dumpsys SurfaceFlinger | grep InputMethod
2. Note down IME Buffer layers that consume memory
using command in #1
3. Hide IME and note down memory again.
It should not have anything other than container layers
4. Use 3 button navbar and retest by pressing home button
Change-Id: I629fa817eb57e8dd6f7e58c1a358487a16363ddb
* autofill will cache the inline suggestions response until it receives
a start input view event from IME
* the data flow from IMS point of view is:
IMS#startViews and IMS#doStartInput (before calling onStartInputView)
->
[async] InlineSuggestionsRequestCallback#onInputMethodStartInputView()
--- process boundary ---
->
IMMS.InlineSuggestionsRequestCallbackDecorator
#onInputMethodStartInputView()
->
InlineSuggestionSession.InlineSuggestionsRequestCallbackImpl
#onInputMethodStartInputView()
* similar data flow for IMS#finishViews()
* this CL should not block IME's UI thread because it's only issuing a
new async IPC from IMS start/finish input view call that's running on
the UI thread.
* there should not be performance impact on IMEs if autofill inline
integration is not active
Test: manual verification, atest EditorInfoTest
Test: atest android.autofillservice.cts.inline, with two failing cases:
InlineAugmentedLoginActivityTest#testAugmentedAutoFill_twoDatasetThenFilledSecond
and InlineAugmentedLoginActivityTest#testAugmentedAutoFill_oneDatasetThenFilled
due to the test itself being broken, I'll fix the test in a separate patch
Bug: 149522488
Bug: 149442582
Change-Id: I2faa3577b9f95a122f26a6d7fa7822a769a51e34
* So it can be checked against the autofillId from autofill manager
service
* Currently not checking due to race condition, will find a fix in
follow up CL
Test: manual verification
Bug: 149522488
Change-Id: I49457e33c7a1acb028023cb70f248805a96c5346
* Pipe the augmented autofill's inline actions through to the
InlineSuggestionsResponse
* Do not send the inline action if the number of inline suggestions is
zero
* Refactor autofill inline suggstion session so that all the calls to
the IME has happens within the class
* Send an empty response to IME to hide the inline suggestion UI when
view exits and the previous response wasn't empty
Test: manual verification, atest InlineLoginActivityTest
Bug: 149522488
Bug: 150312201
Change-Id: I7a0dbf44e9fad6e7da857448c0f2b186e1681d17
Pipe the windowToken of the window requesting hideSoftInput just like we
did it for showSoftInput [1].
This calls hideInsets on the correct display when control target is
different from IME target e.g. ActivityView.
Also hideInsets should be called on InsetsControlTarget instead which was
originally attempted in [2]
[1]: Ia49e23dd077d264a58d28a7b8acffde54b7db187
[2]: I7133e151a1037c42b275b97857936437a7a6725f
Bug: 149870112
Bug: 133381284
Test: Manually using steps mentioned in bug.
Change-Id: Ia596a392eb73ae46debd097151c8c9a7edd59833
When IME is no longer shown, notify IMS to hide the window surface.
This reclaims the memory allocated to IME window surface.
Fix: 150163310
Test: atest CtsInputMethodTests
Manually:
1. Open any app that has editor and run
adb shell dumpsys SurfaceFlinger | grep Total
2. Note down total memory
3. Tap on editor and IME shows. Note down memory
using command in #1. It should go up by ~27MB
4. Hide IME and note down memory again.
It should revert to original value in #1.
Change-Id: I6c2a13dd40b22d3fd62b4763ba369992e5ff1138
* The bundle will be generated/consumed by the support library.
* More API documentation and example usage will be added later once we have the support library impl ready.
* The old style resource name approach doesn't work due to the potential mismatch in the support library version across the host IME and the platform renderer service, and the non-static nature of the public attribute int id in the support lib.
* The Bundle added to the onCreateInlineSuggestionsRequest() is intended for the platform/ExtServices to communicate the UI versions it supports.
* The Bundle added to the InlinePresentationSpec is intended for the IME to communicate the custom styles for the chosen UI versions.
Test: manual verification, and also atest CtsInputMethodTestCases:InlineSuggestionInfoTest CtsInputMethodTestCases:InlineSuggestionTest CtsInputMethodTestCases:InlineSuggestionsRequestTest CtsInputMethodTestCases:InlineSuggestionsResponseTest
Bug: 146454892
Change-Id: Id7fea32a7550fb924fec811b376790474a7b92eb
Added a render service in ExtServices to connect to the renderer in
androidx.autofill for inline suggestion slices.
Cleaned up old UI rendering code that lived in system_server.
Bug: 146453086
Test: atest ExtServicesUnitTests
Change-Id: I25a7ea438afe524683671c850625ae80dacccfaa
Replace the existing usages of now-deprecated API
WindowManager.getDefaultDisplay() with WindowMetrics or
Context.getDisplay() in frameworks/base.
Bug: 128338354
Test: Build, auto test
Change-Id: I02d38a022c5e0e6e9d699f03d35b65d6c8126da9
Since we have a better approach of fitting content instead of
setting them on the layout params
(Window.setContentOnApplyWindowInsetsListener), we can now
simplify forcing of filling the screen:
Instead of clearing the fit types when the private flag is set, we
unset the types and straight out reject any params when the client
would like to fit.
Test: DisplayPolicyTests
Bug: 118118435
Change-Id: I845e6b1c81e29ab66a770891d03c62a32418e8cc
When root-level content containers fit insets, they used to just
apply and consume the entire system insets. However, with the new
Inset APIs, and with deprecating ADJUST_RESIZE IME flag, we want
to give apps an easy way to customize this behavior.
For that, we introduce Window.setOnContentApplyWindowInsetsListener
that returns what kind of margins/padding should be applied and
what should be dispatched to the content views. This is essentially
a replacement for SYSTEM_UI_FLAG_LAYOUT_* as well as
SOFT_INPUT_ADJUST_RESIZE: It allows apps to choose which insets
should be handled on the window level vs view level.
For that, we mark the window decor views as
FRAMEWORK_OPTIONAL_FIT_SYSTEM_WINDOWS, in order to distinguish the
case when support library calls makeOptionalFitSystemWindows(). This
is because of two reasons:
- We don't want the listener to be invoked twice.
- We can not do the compat ping-pong between onApplyWindowInsets
and fitSystemWindows. This is because during the ping-pong, the
result of the OnContentApplyWindowInsetsListener would be lost.
However, we still need to do the compat ping-pong for
ActionBarOverlayLayout in the support library (until that gets
migrated to use onApplyWindowInsets), so we have this separate
dispatching path that only gets used for framework optional
fitting views.
Test: WindowTest
Bug: 118118435
Change-Id: I4b514addd9e094163062d651972f85615b4a35db
It takes time from when IME is requested to the time when IME is ready
to be shown. When its ready to be shown, we need to make sure that
window that requested IME is still the IME target in DisplayContent. The
only realistic way of knowing originating window is passing windowToken
from IMM API.
Bug: 111084606
Test: CtsInputMethodTestCases
Change-Id: Ia49e23dd077d264a58d28a7b8acffde54b7db187
Existing annotations in libcore/ and frameworks/ will deleted after the migration. This also means that any java library that compiles @UnsupportedAppUsage requires a direct dependency on "unsupportedappusage" java_library.
Bug: 145132366
Test: m && diff unsupportedappusage_index.csv
Change-Id: I0c336de56bc4a041dc97ff9b7927f62f0b44b457
1. Only print log when debug is on.
2. Callback be notified when input does not start.
3. Avoid using hard code tag in Log class.
4. onCreateInlineSuggestionsRequest() do nothing in NOP.
5. Add missing javadoc.
Bug: 146525448
Test: manual verification
Change-Id: I41e1de92ffcdb8020aef99acbfec274e0294bad3
