* In IME side, wait for the input start before calling back to Autofill,
rather than returning inline unsupported immediately.
* Also adds an InlineSuggestionManager to simplify code in the
InputMethodService
Test: atest CtsAutoFillServiceTestCases
Test: atest CtsInputMethodTestCases
Bug: 151123764
Change-Id: I199925d77aa508f259e98a8929120aeb96015b57
And let IME always extend into the status bar area so that the position
of IME can never affected by status bar. This can prevent flicker during
IME animation.
This CL also makes PerDisplay can restart the animation while the insets
source control is changed during animation.
Fix: 151759336
Test: atest WindowStateTests
Change-Id: Ic2a308e6b7ec39b4b8645751e31addd26ddf3735
Use case: Jetpack WM will use them to get the location of windows on
screen and compute the display feature positions in window coordinate
space.
Bug: 150908045
Test: atest FrameworksCoreTests:WindowMetricsTest
Test: atest CtsWindowManagerDeviceTestCases:WindowMetricsTests
Change-Id: Ia08950cd5df35971408e8b17bb27d97d29d0ab9b
Exempt-From-Owner-Approval: API change
Insets API notifies IME when it's hidden. however, IMMS never
really received these state changes. Using requestHideSelf() makes sure
IMMS is in-sync.
Fix: 151980214
Test: manually using the steps in bug
Change-Id: I7f6098a61a5942795ffd33a60329e4dd5fb5d6cb
IME process is notified after client has hidden the IME. At this point
IME should call doHideWindow() so internal lifecycle methods are called
the way they were called earlier.
Also, with insets we don't hide the decorView of IME window till client
is unbound, so to be consistent with previous behavior, call
windowVisibilityChanged(View.GONE) explicitly.
Fix: 151125925
Test: atest KeyboardVisibilityTest InputMethodServiceTest
Change-Id: Ibe9a02b93489370b6ae9b497993e366f88791a62
IME is not big enough to be the background of navigation bar when
navigation bar is on the left or right edge of the screen. Also,
IME can be on top of navigation bar in landscape split-screen mode,
we don't want IME to block touches on navigation bar.
Fix: 151083985
Test: 1. Open Messages.
2. Rotate device to landscape.
3. Click on "Search message" and see if IME extends into
the navigation bar area at the left or right edge.
Change-Id: I0ef3d6379a9ae52b3749154d2fdc54e9aa94a9e0
It's useful for apps to know which inset types they can currently
control, as otherwise they have to poll by calling
controlInsetsAnimation repeatedly.
This can be used when apps want to apply a custom animation
immediately during startup as soon as possible.
Fixes: 150780468
Test: InsetsControllerTest
Test: CTS will be added soon
Test: WindowInsetsActivity
Change-Id: Ic0388c11d759843d3ac9edd8ef23904c9ce05c46
Previous attempt to remove IME surface [1] works when IME animates out
within same app window. However, if IME doesn't animate or window hides
without hiding keyboard, surface would still be around in memory.
With this CL, IME surface will be hidden moment it unbinds from the
window.
[1] I6c2a13dd40b22d3fd62b4763ba369992e5ff1138
Bug: 150163310
Test: Manually:
1. Open any app that has editor and tap on editor to show IME
adb shell dumpsys SurfaceFlinger | grep InputMethod
2. Note down IME Buffer layers that consume memory
using command in #1
3. Hide IME and note down memory again.
It should not have anything other than container layers
4. Use 3 button navbar and retest by pressing home button
Change-Id: I629fa817eb57e8dd6f7e58c1a358487a16363ddb
* autofill will cache the inline suggestions response until it receives
a start input view event from IME
* the data flow from IMS point of view is:
IMS#startViews and IMS#doStartInput (before calling onStartInputView)
->
[async] InlineSuggestionsRequestCallback#onInputMethodStartInputView()
--- process boundary ---
->
IMMS.InlineSuggestionsRequestCallbackDecorator
#onInputMethodStartInputView()
->
InlineSuggestionSession.InlineSuggestionsRequestCallbackImpl
#onInputMethodStartInputView()
* similar data flow for IMS#finishViews()
* this CL should not block IME's UI thread because it's only issuing a
new async IPC from IMS start/finish input view call that's running on
the UI thread.
* there should not be performance impact on IMEs if autofill inline
integration is not active
Test: manual verification, atest EditorInfoTest
Test: atest android.autofillservice.cts.inline, with two failing cases:
InlineAugmentedLoginActivityTest#testAugmentedAutoFill_twoDatasetThenFilledSecond
and InlineAugmentedLoginActivityTest#testAugmentedAutoFill_oneDatasetThenFilled
due to the test itself being broken, I'll fix the test in a separate patch
Bug: 149522488
Bug: 149442582
Change-Id: I2faa3577b9f95a122f26a6d7fa7822a769a51e34
* So it can be checked against the autofillId from autofill manager
service
* Currently not checking due to race condition, will find a fix in
follow up CL
Test: manual verification
Bug: 149522488
Change-Id: I49457e33c7a1acb028023cb70f248805a96c5346
* Pipe the augmented autofill's inline actions through to the
InlineSuggestionsResponse
* Do not send the inline action if the number of inline suggestions is
zero
* Refactor autofill inline suggstion session so that all the calls to
the IME has happens within the class
* Send an empty response to IME to hide the inline suggestion UI when
view exits and the previous response wasn't empty
Test: manual verification, atest InlineLoginActivityTest
Bug: 149522488
Bug: 150312201
Change-Id: I7a0dbf44e9fad6e7da857448c0f2b186e1681d17
Pipe the windowToken of the window requesting hideSoftInput just like we
did it for showSoftInput [1].
This calls hideInsets on the correct display when control target is
different from IME target e.g. ActivityView.
Also hideInsets should be called on InsetsControlTarget instead which was
originally attempted in [2]
[1]: Ia49e23dd077d264a58d28a7b8acffde54b7db187
[2]: I7133e151a1037c42b275b97857936437a7a6725f
Bug: 149870112
Bug: 133381284
Test: Manually using steps mentioned in bug.
Change-Id: Ia596a392eb73ae46debd097151c8c9a7edd59833
When IME is no longer shown, notify IMS to hide the window surface.
This reclaims the memory allocated to IME window surface.
Fix: 150163310
Test: atest CtsInputMethodTests
Manually:
1. Open any app that has editor and run
adb shell dumpsys SurfaceFlinger | grep Total
2. Note down total memory
3. Tap on editor and IME shows. Note down memory
using command in #1. It should go up by ~27MB
4. Hide IME and note down memory again.
It should revert to original value in #1.
Change-Id: I6c2a13dd40b22d3fd62b4763ba369992e5ff1138
* The bundle will be generated/consumed by the support library.
* More API documentation and example usage will be added later once we have the support library impl ready.
* The old style resource name approach doesn't work due to the potential mismatch in the support library version across the host IME and the platform renderer service, and the non-static nature of the public attribute int id in the support lib.
* The Bundle added to the onCreateInlineSuggestionsRequest() is intended for the platform/ExtServices to communicate the UI versions it supports.
* The Bundle added to the InlinePresentationSpec is intended for the IME to communicate the custom styles for the chosen UI versions.
Test: manual verification, and also atest CtsInputMethodTestCases:InlineSuggestionInfoTest CtsInputMethodTestCases:InlineSuggestionTest CtsInputMethodTestCases:InlineSuggestionsRequestTest CtsInputMethodTestCases:InlineSuggestionsResponseTest
Bug: 146454892
Change-Id: Id7fea32a7550fb924fec811b376790474a7b92eb
Added a render service in ExtServices to connect to the renderer in
androidx.autofill for inline suggestion slices.
Cleaned up old UI rendering code that lived in system_server.
Bug: 146453086
Test: atest ExtServicesUnitTests
Change-Id: I25a7ea438afe524683671c850625ae80dacccfaa
Replace the existing usages of now-deprecated API
WindowManager.getDefaultDisplay() with WindowMetrics or
Context.getDisplay() in frameworks/base.
Bug: 128338354
Test: Build, auto test
Change-Id: I02d38a022c5e0e6e9d699f03d35b65d6c8126da9
Since we have a better approach of fitting content instead of
setting them on the layout params
(Window.setContentOnApplyWindowInsetsListener), we can now
simplify forcing of filling the screen:
Instead of clearing the fit types when the private flag is set, we
unset the types and straight out reject any params when the client
would like to fit.
Test: DisplayPolicyTests
Bug: 118118435
Change-Id: I845e6b1c81e29ab66a770891d03c62a32418e8cc
When root-level content containers fit insets, they used to just
apply and consume the entire system insets. However, with the new
Inset APIs, and with deprecating ADJUST_RESIZE IME flag, we want
to give apps an easy way to customize this behavior.
For that, we introduce Window.setOnContentApplyWindowInsetsListener
that returns what kind of margins/padding should be applied and
what should be dispatched to the content views. This is essentially
a replacement for SYSTEM_UI_FLAG_LAYOUT_* as well as
SOFT_INPUT_ADJUST_RESIZE: It allows apps to choose which insets
should be handled on the window level vs view level.
For that, we mark the window decor views as
FRAMEWORK_OPTIONAL_FIT_SYSTEM_WINDOWS, in order to distinguish the
case when support library calls makeOptionalFitSystemWindows(). This
is because of two reasons:
- We don't want the listener to be invoked twice.
- We can not do the compat ping-pong between onApplyWindowInsets
and fitSystemWindows. This is because during the ping-pong, the
result of the OnContentApplyWindowInsetsListener would be lost.
However, we still need to do the compat ping-pong for
ActionBarOverlayLayout in the support library (until that gets
migrated to use onApplyWindowInsets), so we have this separate
dispatching path that only gets used for framework optional
fitting views.
Test: WindowTest
Bug: 118118435
Change-Id: I4b514addd9e094163062d651972f85615b4a35db
It takes time from when IME is requested to the time when IME is ready
to be shown. When its ready to be shown, we need to make sure that
window that requested IME is still the IME target in DisplayContent. The
only realistic way of knowing originating window is passing windowToken
from IMM API.
Bug: 111084606
Test: CtsInputMethodTestCases
Change-Id: Ia49e23dd077d264a58d28a7b8acffde54b7db187
Existing annotations in libcore/ and frameworks/ will deleted after the migration. This also means that any java library that compiles @UnsupportedAppUsage requires a direct dependency on "unsupportedappusage" java_library.
Bug: 145132366
Test: m && diff unsupportedappusage_index.csv
Change-Id: I0c336de56bc4a041dc97ff9b7927f62f0b44b457
1. Only print log when debug is on.
2. Callback be notified when input does not start.
3. Avoid using hard code tag in Log class.
4. onCreateInlineSuggestionsRequest() do nothing in NOP.
5. Add missing javadoc.
Bug: 146525448
Test: manual verification
Change-Id: I41e1de92ffcdb8020aef99acbfec274e0294bad3
Introduce new APIs in Window/WindowManager.LayoutParams for developers
to decide which types of insets at which side a window should avoid:
setFitWindowInsetsTypes(@InsetsType int types)
setFitWindowInsetsSides(@InsetsSide int sides)
setFitIgnoreVisibility(boolean ignore)
The existing logic in DisplayPolicy.layoutWindowLw uses combinations of
window types, window flags, and system UI flags to decide what frames a
window should have, which is very complex, difficult to maintain, and
should be replaced with the new APIs.
Bug: 118118435
Test: atest InsetsSourceProviderTest InsetsStateControllerTest
InsetsPolicyTest WindowStateTests CommandQueueTest
RegisterStatusBarResultTest InsetsFlagsTest
LightBarControllerTest RegisterStatusBarResultTest
ViewRootImplTest DisplayPolicyLayoutTests
DisplayPolicyInsetsTests DisplayPolicyTests
TaskSnapshotSurfaceTest
Change-Id: I06ddc9d0d2887ba4ded7bb8adbf9c9c0da4bf7b4
Use visible inset values provided by
IMEs to set gesture exclusion rects
for the EdgeBackGestureHandler to ignore
regions where the keyboard is.
If the IME has not overridden
onComputeInsets(), InputMethodService
uses the location of R.id.inputArea
to approximate the location of where
the IME region starts.
Fixes: 141215181
Test: Tested full screen landscape keyboards
(Messenger, Hangouts), non full screen landscape
keyboards (SMS Messages), searching from the top
of the screen in the Toolbar (Google Play Store)
Change-Id: I359d719493fb92d49cd309c2d00371134cd758fe
Support new IME inset api transitions without using pre-rendering.
This would be the default behavior when ViewRootImpl#sNewInsetsMode > 0
and pre-rendering is not enabled.
Bug: 111084606
Bug: 118599175
Test: Manually verify by just enabling Insets API and keeping
pre-rendering off.
1. Build and flash
2. adb shell setprop persist.wm.new_insets 1
3. adb reboot
4. Make sure tapping on edit text brings keyboard up with new
transition and back closes IME with various apps.
5. Make sure IME behavior is unchanged for apps with
ADJUST_RESIZE like whatsapp.
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Change-Id: If33e9dd45e549e49757237fa66051351b858875d
This feature is configurable and set to false as the default value. It
is overlayed as true for Automotive.
Bug: 138242566
Test: manual
Change-Id: I5dde8adc9d4ce749d895287ccf63017b284caec8
When IME window switches from current display to another display and back quickly,
system will re-bind InputMethodService to re-attach window token for each moving,
the IME service lifecycle will be onBind -> onUnBind -> onDestroy -> onBind.
Ideally, system will deliver window token when onServiceConnected and system should
only receive one onServiceConnected when the last service bound.
But due to user switch display very quick, before unbind service callbacked from client,
the next bind service request comes, caused the connection exists in system side and
client will then callback 2 onServiceConnected to system.
Since CL [1] introduced InputMethodPrivilegedOperationsRegistry to deal with token
registraction singleton-ness, the exception will be thrown for above case due to
the duplicate window token passed to client.
Add InputMethodPrivilegedOperationsRegistry#isRegistered to check if the token
already registered, if so, skip this duplicate initialization request.
[1]: I4a61470f06ffac5f7a512536f8431489db0108f4
Fix: 133624278
Test: manual as below steps:
1) Settings > Developer options, enable "Simulated display" & "force desktop mode"
2) Reboot device.
3) Launch app from simulated display.
4) Launch app from primary display (i.e. contacts), focus EditText to show IME.
5) Tapping primary & simulated display repeatly & quickly.
6) See if any exception log from IME application. (i.e. Gboard)
Change-Id: Ie6bfbae735724fe744590e715124d2737d2b665d
MSIME doesn't work when existing InputConnection is restored. The fix is
to allow MSIME to activate connection when window gains focus and set
inactive when window loses focus. This mimics single session IME.
Bug: 128751901
Test: Manually tested using steps in bug.
Change-Id: I22cef6c1dacb0ca402840da94a2bf02b989da447
For packages:
android.companion
android.filterfw
android.hardware.camera2.utils
android.inputmethodservice
android.net.nsd
android.os
android.preference
android.security.keymaster
android.service.dreams
android.telecom
android.telephony.ims.compat.feature
android.telephony
android.util
android.view.accessibility
android.media.effect
This is an automatically generated CL. See go/UnsupportedAppUsage
for more details.
Exempted-From-Owner-Approval: Mechanical changes to the codebase
which have been approved by Android API council and announced on
android-eng@
Bug: 110868826
Test: m
Change-Id: I9c2f8347952f3cc65759472b0e1a2717b285e44e
With this change, InsetsController.show/hide now links to IME. This also
takes care of animating IME along with other types.
Insets API are reactive i.e. they remain in sync with state of IME.
Test: atest InsetsControllerTest
Test: atest ImeInsetsConsumerTest
Bug: 118118435
Change-Id: Ib3997487bd19351d1d23bc70173fc9bdfd23a704
Add a IME state changes callback that pipes IME state into the Inset
consumer.
Bug: 118599175
Bug: 118118435
Test: atest InsetControllerTest
Test: atest InsetSourceConsumerTest
Test: atest ImeInsetsSourceConsumerTest
Change-Id: Id878226418e19cdf0499a0094f1d5c47fea33125
Pre-render input method views and window when EditText receives focus.
This is a pre-requisite for implementing better IME transitions.
Strategy:
Once EditText receives focus, startInput is called. If optimization is
available, IME views and window (SoftInputWindow) are created and
rendered. Until user taps on EditText or showSoftInput() is called, IME
window remains invisible. This pre-rendered window is kept around until
EditorInfo changes or new connection is started (onStartInput).
IME window's visibility will be set using new Insets controller API
rather than conventional client-side dialog.show().
Behavior:
- This is just IME side preparation CL. No performance improvements yet.
- There should be no user perceptible behavior change.
- As long as IME developers were following official lifecycle, they
shouldn't perceive any behavior change.
Availability:
This optimization, once fully implemented, will be available when:
- Device is not "Low memory"
- AND Master flag DebugFlags.FLAG_PRE_RENDER_IME_VIEWS is set.
- ViewRootImpl.USE_NEW_INSETS_API is enabled
Bug: 118599175
Bug: 111084606
Test: atest CtsInputMethodTestCases
Test: atest CtsInputMethodServiceHostTestCases
Test: atest ActivityManagerMultiDisplayTests
Test: Tested with 4 IMEs and didn't preceive any behavior change.
Scenarios tested:
1. With and without hardware keyboard
2. Screen rotation w/ fullscreen mode.
3. split-screen
Change-Id: I1a6300fe167eb205ee2b4214a6e270a52ebae062
