Commit Graph

320 Commits

Author SHA1 Message Date
Tarandeep Singh
0fe4f784f4 Remove IME surface when window unbinds
Previous attempt to remove IME surface [1] works when IME animates out
within same app window. However, if IME doesn't animate or window hides
without hiding keyboard, surface would still be around in memory.
With this CL, IME surface will be hidden moment it unbinds from the
window.
[1] I6c2a13dd40b22d3fd62b4763ba369992e5ff1138

Bug: 150163310
Test: Manually:
      1. Open any app that has editor and tap on editor to show IME
          adb shell  dumpsys SurfaceFlinger | grep InputMethod
      2. Note down IME Buffer layers that consume memory
         using command in #1
      3. Hide IME and note down memory again.
         It should not have anything other than container layers
      4. Use 3 button navbar and retest by pressing home button
Change-Id: I629fa817eb57e8dd6f7e58c1a358487a16363ddb
2020-03-06 04:35:23 +00:00
Feng Cao
7c85eb79a7 Notify autofill with the IME start/finish input view events
* autofill will cache the inline suggestions response until it receives
  a start input view event from IME
* the data flow from IMS point of view is:
  IMS#startViews and IMS#doStartInput (before calling onStartInputView)
  ->
  [async] InlineSuggestionsRequestCallback#onInputMethodStartInputView()
  --- process boundary ---
  ->
  IMMS.InlineSuggestionsRequestCallbackDecorator
     #onInputMethodStartInputView()
  ->
  InlineSuggestionSession.InlineSuggestionsRequestCallbackImpl
     #onInputMethodStartInputView()
* similar data flow for  IMS#finishViews()
* this CL should not block IME's UI thread because it's only issuing a
  new async IPC from IMS start/finish input view call that's running on
  the UI thread.
* there should not be performance impact on IMEs if autofill inline
  integration is not active

Test: manual verification, atest EditorInfoTest
Test: atest android.autofillservice.cts.inline, with two failing cases:
 InlineAugmentedLoginActivityTest#testAugmentedAutoFill_twoDatasetThenFilledSecond
 and InlineAugmentedLoginActivityTest#testAugmentedAutoFill_oneDatasetThenFilled
 due to the test itself being broken, I'll fix the test in a separate patch
Bug: 149522488
Bug: 149442582
Change-Id: I2faa3577b9f95a122f26a6d7fa7822a769a51e34
2020-03-02 21:18:16 -08:00
Feng Cao
e65a97cef4 Populate the autofillId in the IMS EditorInfo
* So it can be checked against the autofillId from autofill manager
  service
* Currently not checking due to race condition, will find a fix in
  follow up CL

Test: manual verification
Bug: 149522488

Change-Id: I49457e33c7a1acb028023cb70f248805a96c5346
2020-02-28 22:08:32 -08:00
Feng Cao
ec496000ec Several improvements and bug fixes to the inline autofill flow
* Pipe the augmented autofill's inline actions through to the
  InlineSuggestionsResponse
* Do not send the inline action if the number of inline suggestions is
  zero
* Refactor autofill inline suggstion session so that all the calls to
  the IME has happens within the class
* Send an empty response to IME to hide the inline suggestion UI  when
  view exits and the previous response wasn't empty

Test: manual verification, atest InlineLoginActivityTest
Bug: 149522488
Bug: 150312201

Change-Id: I7a0dbf44e9fad6e7da857448c0f2b186e1681d17
2020-02-28 17:25:30 -08:00
Taran Singh
73b1d16d28 Merge "Pipe windowToken for hideSoftInput" into rvc-dev 2020-02-27 07:14:04 +00:00
Tarandeep Singh
4fe5b654a1 Pipe windowToken for hideSoftInput
Pipe the windowToken of the window requesting hideSoftInput just like we
did it for showSoftInput [1].
This calls hideInsets on the correct display when control target is
different from IME target e.g. ActivityView.
Also hideInsets should be called on InsetsControlTarget instead which was
originally attempted in [2]

[1]: Ia49e23dd077d264a58d28a7b8acffde54b7db187
[2]: I7133e151a1037c42b275b97857936437a7a6725f

Bug: 149870112
Bug: 133381284
Test: Manually using steps mentioned in bug.

Change-Id: Ia596a392eb73ae46debd097151c8c9a7edd59833
2020-02-27 04:13:25 +00:00
Tarandeep Singh
94c9a8328e Remove IME surface when hidden
When IME is no longer shown, notify IMS to hide the window surface.
This reclaims the memory allocated to IME window surface.

Fix: 150163310
Test: atest CtsInputMethodTests
      Manually:
      1. Open any app that has editor  and run
         adb shell  dumpsys SurfaceFlinger | grep Total
      2. Note down total memory
      3. Tap on editor and IME shows. Note down memory
         using command in #1. It should go up by ~27MB
      4. Hide IME and note down memory again.
         It should revert to original value in #1.

Change-Id: I6c2a13dd40b22d3fd62b4763ba369992e5ff1138
2020-02-26 18:58:33 +00:00
Feng Cao
36960ee20e Introduces the Bundle to inline suggestions APIs to encode custom UI styling information.
* The bundle will be generated/consumed by the support library.
* More API documentation and example usage will be added later once we have the support library impl ready.
* The old style resource name approach doesn't work due to the potential mismatch in the support library version across the host IME and the platform renderer service, and the non-static nature of the public attribute int id in the support lib.
* The Bundle added to the onCreateInlineSuggestionsRequest() is intended for the platform/ExtServices to communicate the UI versions it supports.
* The Bundle added to the InlinePresentationSpec is intended for the IME to communicate the custom styles for the chosen UI versions.

Test: manual verification, and also atest CtsInputMethodTestCases:InlineSuggestionInfoTest CtsInputMethodTestCases:InlineSuggestionTest CtsInputMethodTestCases:InlineSuggestionsRequestTest CtsInputMethodTestCases:InlineSuggestionsResponseTest
Bug: 146454892

Change-Id: Id7fea32a7550fb924fec811b376790474a7b92eb
2020-02-19 09:42:22 -08:00
Feng Cao
abd6b072ff Refactor the InputMethodService to move inline suggestion stuff to a separate class
Test: manual
Bug: 149442582

Change-Id: I71a3cf2ddd24dcf79709ec89136ad7609089f54c
2020-02-14 23:42:20 -08:00
Adam He
4011625638 Move rendering logic for inline suggestions to ExtServices.
Added a render service in ExtServices to connect to the renderer in
androidx.autofill for inline suggestion slices.

Cleaned up old UI rendering code that lived in system_server.

Bug: 146453086
Test: atest ExtServicesUnitTests
Change-Id: I25a7ea438afe524683671c850625ae80dacccfaa
2020-02-14 11:15:43 -08:00
Wale Ogunwale
ecf66058ea Added OWNERS file for inputmethod
Bug: N/A
Test: N/A
Change-Id: I0dd7bad55dec4f2332d7819d4ccb1e0da7a2d9b7
2020-02-13 13:14:24 -08:00
Jian-Yang Liu
09d558670d Merge "Updated InputMethodService to not inset by navigation bar if requested by automotive." 2020-02-03 18:11:10 +00:00
Andrii Kulian
e57f2dc246 Exempt-From-Owner-Approval: Fix usages of WindowManager.getDefaultDisplay() in f/b
Replace the existing usages of now-deprecated API
WindowManager.getDefaultDisplay() with WindowMetrics or
Context.getDisplay() in frameworks/base.

Bug: 128338354
Test: Build, auto test
Change-Id: I02d38a022c5e0e6e9d699f03d35b65d6c8126da9
2020-01-31 01:15:21 +00:00
JianYang Liu
7eec316f54 Updated InputMethodService to not inset by navigation bar if requested
by automotive.

Bug: 147155538
Test: Manual
Change-Id: I4faf82bdd7536bd2d049ded04034a9635d8ca0d3
2020-01-28 14:42:00 -08:00
Tiger Huang
527244459c Window Manager Flag Migration (12/n)
- Unhide new APIs
- MaxInsets -> InsetsIgnoringVisibility
- Deprecate SystemUI flags
- Deprecate some other stuff

Bug: 118118435
Test: Build, CTS coming soon
Change-Id: I0a0cbdb48258b3779d536668b59e8c88f3c96d18
Exempt-From-Owner-Approval: Trivial rename
2020-01-23 14:11:39 +01:00
Jorim Jaggi
af30559dc3 Merge changes from topic "content_apply"
* changes:
  Force non-floating main app windows to not fit anything
  Introduce Window.setContentOnApplyWindowInsetsListener
2020-01-23 10:52:36 +00:00
Jorim Jaggi
0da8fd165f Force non-floating main app windows to not fit anything
Since we have a better approach of fitting content instead of
setting them on the layout params
(Window.setContentOnApplyWindowInsetsListener), we can now
simplify forcing of filling the screen:

Instead of clearing the fit types when the private flag is set, we
unset the types and straight out reject any params when the client
would like to fit.

Test: DisplayPolicyTests
Bug: 118118435
Change-Id: I845e6b1c81e29ab66a770891d03c62a32418e8cc
2020-01-23 00:27:58 +01:00
Jorim Jaggi
7fb715c425 Introduce Window.setContentOnApplyWindowInsetsListener
When root-level content containers fit insets, they used to just
apply and consume the entire system insets. However, with the new
Inset APIs, and with deprecating ADJUST_RESIZE IME flag, we want
to give apps an easy way to customize this behavior.

For that, we introduce Window.setOnContentApplyWindowInsetsListener
that returns what kind of margins/padding should be applied and
what should be dispatched to the content views. This is essentially
a replacement for SYSTEM_UI_FLAG_LAYOUT_* as well as
SOFT_INPUT_ADJUST_RESIZE: It allows apps to choose which insets
should be handled on the window level vs view level.

For that, we mark the window decor views as
FRAMEWORK_OPTIONAL_FIT_SYSTEM_WINDOWS, in order to distinguish the
case when support library calls makeOptionalFitSystemWindows(). This
is because of two reasons:
- We don't want the listener to be invoked twice.
- We can not do the compat ping-pong between onApplyWindowInsets
and fitSystemWindows. This is because during the ping-pong, the
result of the OnContentApplyWindowInsetsListener would be lost.
However, we still need to do the compat ping-pong for
ActionBarOverlayLayout in the support library (until that gets
migrated to use onApplyWindowInsets), so we have this separate
dispatching path that only gets used for framework optional
fitting views.

Test: WindowTest
Bug: 118118435
Change-Id: I4b514addd9e094163062d651972f85615b4a35db
2020-01-22 18:09:04 +01:00
Tarandeep Singh
bb0e2f753b Pipe windowToken of window requesting IME
It takes time from when IME is requested to the time when IME is ready
to be shown. When its ready to be shown, we need to make sure that
window that requested IME is still the IME target in DisplayContent. The
only realistic way of knowing originating window is passing windowToken
from IMM API.

Bug: 111084606
Test: CtsInputMethodTestCases

Change-Id: Ia49e23dd077d264a58d28a7b8acffde54b7db187
2020-01-21 17:02:22 -08:00
Artur Satayev
269580069b Use new UnsupportedAppUsage annotation.
Existing annotations in libcore/ and frameworks/ will deleted after the migration. This also means that any java library that compiles @UnsupportedAppUsage requires a direct dependency on "unsupportedappusage" java_library.

Bug: 145132366
Test: m && diff unsupportedappusage_index.csv
Change-Id: I0c336de56bc4a041dc97ff9b7927f62f0b44b457
2020-01-06 16:47:35 +00:00
Joanne Chung
d9a916fe4d Finish autofill integration with keyboard cleanup tasks
1. Only print log when debug is on.
2. Callback be notified when input does not start.
3. Avoid using hard code tag in Log class.
4. onCreateInlineSuggestionsRequest() do nothing in NOP.
5. Add missing javadoc.

Bug: 146525448
Test: manual verification
Change-Id: I41e1de92ffcdb8020aef99acbfec274e0294bad3
2020-01-06 16:43:45 +08:00
Adam He
7bc8f60377 Added attributes for IME and AutofillService to indicate they support
inline suggestions.

Fixes: 146452946
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodInfoTest
Change-Id: I709b16d3f12c693bc670600bdcb9125630eb9b8e
2020-01-03 14:14:17 -08:00
Feng Cao
ec3f84a899 Make InputMethodService to never reuse the inline suggestion response callback.
Test: manual
Bug: 146903532

Change-Id: I7bc007fb3009897cba57e63edbc67cef541f0e05
2019-12-27 11:04:02 -08:00
Adam He
b50bc1f669 Merge "API for autofill integration with keyboard." 2019-12-18 00:39:13 +00:00
Adam He
bc67f2e158 API for autofill integration with keyboard.
Bug: 137800469
Test: manual verification
Change-Id: Id222500c373898d576661cacb7a1cb51061041d4
2019-12-16 12:43:01 -08:00
Tiger Huang
4a7835ffb6 Window Manager Flag Migration (7/n)
Introduce new APIs in Window/WindowManager.LayoutParams for developers
to decide which types of insets at which side a window should avoid:

    setFitWindowInsetsTypes(@InsetsType int types)
    setFitWindowInsetsSides(@InsetsSide int sides)
    setFitIgnoreVisibility(boolean ignore)

The existing logic in DisplayPolicy.layoutWindowLw uses combinations of
window types, window flags, and system UI flags to decide what frames a
window should have, which is very complex, difficult to maintain, and
should be replaced with the new APIs.

Bug: 118118435
Test: atest InsetsSourceProviderTest InsetsStateControllerTest
            InsetsPolicyTest WindowStateTests CommandQueueTest
            RegisterStatusBarResultTest InsetsFlagsTest
            LightBarControllerTest RegisterStatusBarResultTest
            ViewRootImplTest DisplayPolicyLayoutTests
            DisplayPolicyInsetsTests DisplayPolicyTests
            TaskSnapshotSurfaceTest
Change-Id: I06ddc9d0d2887ba4ded7bb8adbf9c9c0da4bf7b4
2019-12-16 17:08:21 +08:00
Vinit Nayak
31595bc94e Set gesture exclusion rect for IMEs
Use visible inset values provided by
IMEs to set gesture exclusion rects
for the EdgeBackGestureHandler to ignore
regions where the keyboard is.
If the IME has not overridden
onComputeInsets(), InputMethodService
uses the location of R.id.inputArea
to approximate the location of where
the IME region starts.

Fixes: 141215181
Test: Tested full screen landscape keyboards
(Messenger, Hangouts), non full screen landscape
keyboards (SMS Messages), searching from the top
of the screen in the Toolbar (Google Play Store)

Change-Id: I359d719493fb92d49cd309c2d00371134cd758fe
2019-10-21 11:31:12 -07:00
Taran Singh
d332901cde Merge "IME transitions without pre-rendering." 2019-08-27 20:57:59 +00:00
Tarandeep Singh
92d2dd36cd IME transitions without pre-rendering.
Support new IME inset api transitions without using pre-rendering.
This would be the default behavior when ViewRootImpl#sNewInsetsMode > 0
and pre-rendering is not enabled.

Bug: 111084606
Bug: 118599175
Test: Manually verify by just enabling Insets API and keeping
    pre-rendering off.
     1. Build and flash
     2. adb shell setprop persist.wm.new_insets 1
     3. adb reboot
     4. Make sure tapping on edit text brings keyboard up with new
     transition and back closes IME with various apps.
     5. Make sure IME behavior is unchanged for apps with
        ADJUST_RESIZE like whatsapp.
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases

Change-Id: If33e9dd45e549e49757237fa66051351b858875d
2019-08-26 12:44:10 -07:00
Heemin Seog
26c830dd16 Listen for changes to IME and show/hide nav bar
This feature is configurable and set to false as the default value. It
is overlayed as true for Automotive.

Bug: 138242566
Test: manual
Change-Id: I5dde8adc9d4ce749d895287ccf63017b284caec8
2019-08-16 15:13:16 -07:00
lumark
e0af39403e Skip IME initialization when received the token that already registered
When IME window switches from current display to another display and back quickly,
system will re-bind InputMethodService to re-attach window token for each moving,
the IME service lifecycle will be onBind -> onUnBind -> onDestroy -> onBind.

Ideally, system will deliver window token when onServiceConnected and system should
only receive one onServiceConnected when the last service bound.

But due to user switch display very quick, before unbind service callbacked from client,
the next bind service request comes, caused the connection exists in system side and
client will then callback 2 onServiceConnected to system.

Since CL [1] introduced InputMethodPrivilegedOperationsRegistry to deal with token
registraction singleton-ness, the exception will be thrown for above case due to
the duplicate window token passed to client.

Add InputMethodPrivilegedOperationsRegistry#isRegistered to check if the token
already registered, if so, skip this duplicate initialization request.

[1]: I4a61470f06ffac5f7a512536f8431489db0108f4

Fix: 133624278
Test: manual as below steps:
    1) Settings > Developer options, enable "Simulated display" & "force desktop mode"
    2) Reboot device.
    3) Launch app from simulated display.
    4) Launch app from primary display (i.e. contacts), focus EditText to show IME.
    5) Tapping primary & simulated display repeatly & quickly.
    6) See if any exception log from IME application. (i.e. Gboard)

Change-Id: Ie6bfbae735724fe744590e715124d2737d2b665d
2019-05-30 12:27:17 +08:00
Tarandeep Singh
e1921a7b4d Allow MSIME to set inactive when window loses focus.
MSIME doesn't work when existing InputConnection is restored. The fix is
to allow MSIME to activate connection when window gains focus and set
inactive when window loses focus. This mimics single session IME.

Bug: 128751901
Test: Manually tested using steps in bug.
Change-Id: I22cef6c1dacb0ca402840da94a2bf02b989da447
2019-04-11 15:55:28 -07:00
Andrei Onea
e30e3928f9 Add @UnsupportedAppUsage annotations
For packages:
  android.companion
  android.filterfw
  android.hardware.camera2.utils
  android.inputmethodservice
  android.net.nsd
  android.os
  android.preference
  android.security.keymaster
  android.service.dreams
  android.telecom
  android.telephony.ims.compat.feature
  android.telephony
  android.util
  android.view.accessibility
  android.media.effect

This is an automatically generated CL. See go/UnsupportedAppUsage
for more details.

Exempted-From-Owner-Approval: Mechanical changes to the codebase
which have been approved by Android API council and announced on
android-eng@

Bug: 110868826
Test: m
Change-Id: I9c2f8347952f3cc65759472b0e1a2717b285e44e
2019-03-21 14:04:57 +00:00
Tarandeep Singh
46d59f0ecb Link InsetsController to IME (IME transitons 4/n)
With this change, InsetsController.show/hide now links to IME. This also
takes care of animating IME along with other types.
Insets API are reactive i.e. they remain in sync with state of IME.

Test: atest InsetsControllerTest
Test: atest ImeInsetsConsumerTest
Bug: 118118435

Change-Id: Ib3997487bd19351d1d23bc70173fc9bdfd23a704
2019-02-04 12:41:24 -08:00
Tarandeep Singh
2cbcd7ffbf Pipe IME state into insets (IME transitions 3/n)
Add a IME state changes callback that pipes IME state into the Inset
consumer.

Bug: 118599175
Bug: 118118435
Test: atest InsetControllerTest
Test: atest InsetSourceConsumerTest
Test: atest ImeInsetsSourceConsumerTest

Change-Id: Id878226418e19cdf0499a0094f1d5c47fea33125
2019-01-28 16:58:32 +01:00
Tarandeep Singh
ed3e728055 Merge "Pre-render input method (IME transitions 1/n)" 2019-01-25 02:36:52 +00:00
Tarandeep Singh
eadb1392f8 Pre-render input method (IME transitions 1/n)
Pre-render input method views and window when EditText receives focus.
This is a pre-requisite for implementing better IME transitions.

Strategy:
Once EditText receives focus, startInput is called. If optimization is
available, IME views and window (SoftInputWindow) are created and
rendered. Until user taps on EditText or showSoftInput() is called, IME
window remains invisible. This pre-rendered window is kept around until
EditorInfo changes or new connection is started (onStartInput).
IME window's visibility will be set using new Insets controller API
rather than conventional client-side dialog.show().

Behavior:
- This is just IME side preparation CL. No performance improvements yet.
- There should be no user perceptible behavior change.
- As long as IME developers were following official lifecycle, they
  shouldn't perceive any behavior change.

Availability:
This optimization, once fully implemented, will be available when:
 - Device is not "Low memory"
 - AND Master flag DebugFlags.FLAG_PRE_RENDER_IME_VIEWS is set.
 - ViewRootImpl.USE_NEW_INSETS_API is enabled

Bug: 118599175
Bug: 111084606
Test: atest CtsInputMethodTestCases
Test: atest CtsInputMethodServiceHostTestCases
Test: atest ActivityManagerMultiDisplayTests
Test: Tested with 4 IMEs and didn't preceive any behavior change.
Scenarios tested:
  1. With and without hardware keyboard
  2. Screen rotation w/ fullscreen mode.
  3. split-screen

Change-Id: I1a6300fe167eb205ee2b4214a6e270a52ebae062
2019-01-25 00:47:21 +01:00
Yohei Yukawa
0eb8d16d90 Deprecate InputMethodService#onViewClicked()
This CL marks InputMethodService#onViewClicked() deprecated.  There is
no actual implementation change at this moment, but developers should
avoid relying on this callback in general not only on futuer versions
of Android but also on existing devices.

Since this API was implemented in Android 4.0 ICS [1] it never worked
well for WebView and any other UI libraries that expose themselves as
a giant "Canvas", which can host their own UI hierarchy and sub focus
state.

If IME developers want consistent and reliable user interaction across
devices and applicactions, implementing IMEs' own UI on top of
CursorAnchorInfo API would be a good choice.

If IME developers just want to be notified an existing input
connection is interrupted, they can just monitor
InputMethodService#onFinishInput().

 [1]: Ia71ad325e50a20e9e1001def662ef992bb50a9d0
      863fcd6217

Fix: 123245358
Test: make -j checkbuild
Change-Id: I37a6c6d82d4a424f0df84633b09bca4881efbf02
2019-01-22 21:47:57 -08:00
Yohei Yukawa
9b60ba07e4 Unify IInputMethodPrivilegedOperations as sync IPC
Since its bettining [1] notifyUserAction() has been implemented as an
async IPC method.  This wasn't changed when I recently moved it from
IInputMethodManager to IInputMethodPrivilegedOperations [2].

However, mixing sync and async methods into the same Binder object is
known to be an anti-pattern.  Although sending some signals in an
asynchronous manner would make sense, it should be done by converting
all the methods to async (with some result callbacks for operations
that need to receive responce) like we do so in IInputContext.aidl.

As for IInputMethodPrivilegedOperations, I think converting
notifyUserAction() from async to sync is acceptable because there are
already similar sync methods such as:
 * setImeWindowStatus
 * reportStartInput
 * reportFullscreenMode
 * updateStatusIcon

 [1]: I11ed9a767588f8080753cd9bce011dac7db579ad
      d7443c83ce
 [2]: Icc1f9c7f530f0144ecfd460e86114e109ae0044e
      c07fd4c284

Bug: 114159783
Test: Manually verified as follows
  1. Build and flush aosp_taimen-userdebug into Taimen
  2. make -j SoftKeyboard
  3. adb install -r $OUT/system/app/SoftKeyboard/SoftKeyboard.apk
  4. adb shell ime enable com.example.android.softkeyboard/.SoftKeyboard
  5. Open AOSP Keyboard settings
  6. Enable "English (US)", "French", and "German"
  7. Open SoftKeyboard settings
  8. Enable "English (United States)", "English (GB)"
  9. Open the Dialer app and tap the top edit field.
 10. Make sure that the IME layout rotation order when tapping the
     globe key will be updated only when you tap the keyboard to enter
     some character.
 11. Also confirm it with "adb shell dumpsys input_method" by checking
     "mSwitchingController:" section there.
Change-Id: I9d41f19e30f205acd4d257a105e285bd32288130
2019-01-21 00:06:27 -08:00
Mathew Inwood
31755f94e1 Limit access to suspected false positives.
Members modified herein are suspected to be false positives: i.e. things
that were added to the greylist in P, but subsequent data analysis
suggests that they are not, in fact, used after all.

Add a maxTargetSdk=P to these APIs. This is lower-risk that simply
removing these things from the greylist, as none of out data sources are
perfect nor complete.

For APIs that are not supported yet by annotations, move them to
hiddenapi-greylist-max-p.txt instead which has the same effect.

Exempted-From-Owner-Approval: Automatic changes to the codebase
affecting only @UnsupportedAppUsage annotations, themselves added
without requiring owners approval earlier.

Bug: 115609023
Test: m
Change-Id: I020a9c09672ebcae64c5357abc4993e07e744687
2018-12-28 11:50:04 +00:00
Yohei Yukawa
b7a284e37a Deprecate android.inputmethodservice.{Keyboard,KeyboardView}
This CL deprecates the following two classes.

 * android.inputmethodservice.Keyboard
 * android.inputmethodservice.KeyboardView

Most likely having these classes in the Android Framework does not
make much sense anymore, because basically it is just one
implementation of how keyboard-like UI can be implemented on top of
public Android APIs.  Source code of these classes have been available
as part os AOSP.  Thus it should not be that difficult for app
developers to have their own copy of these classes in their project or
re-implement a similar widget by themselves.

Fix: 116815596
Test: make -j checkbuild
Change-Id: Ie0cb0456c2cc4eacd4b188514508021774f60591
2018-12-13 15:53:16 -08:00
Yohei Yukawa
bae5bea23c Introduce multi-client IME for special form factors
An advanced multi-display support is requested for certain Android
form-factors so that user(s) can type text on each display at the same
time without losing software keyboard focus in other displays. This is
not possible in existing Android IMEs that are built on top of
InputMethodService class, because the assumption that a single IME
client can be focused at the same time was made before Android IME
APIs were introduced in Android 1.5 and many public APIs in
InputMethodService have already relied heavily on that
assumption. Updating InputMethodService class to support multi-client
scenario is, however, quite challenging because:

 1. doing so would introduce an unacceptable amount of complexity into
    InputMethodService, which is already hard to maintain,

 2. IME developers still need to update their implementation to be
    able to support parallel requests from multiple focused IME
    client, which may require non-trivial redesign in their side
    (e.g. input decoder, typing history database, ...), and

 3. actual use cases for multi IME clients are expected to be evolved
    rapidly hence the new protocol is not yet stable and not yet ready
    to be exposed as public APIs.

This is why a new type of IME needs to be designed and developed
specifically for such special multi-display environments, rather than
reusing existing InputMethodService public class.

Note that there must be no behavior change unless multi-client IME is
explicitly enabled with 'adb shell setprop', which requires root
permission.

See multi-client-ime.md for details.

Fix: 114662040
Test: Manually verified as follows:
  1. make -j MultiClientInputMethod
  2. adb install -r $OUT/system/priv-app/MultiClientInputMethod/MultiClientInputMethod.apk
  3. adb root
  4. adb shell setprop persist.debug.multi_client_ime \
       com.example.android.multiclientinputmethod/.MultiClientInputMethod
  5. adb reboot
  6. Try multiple text input scenario
Change-Id: I41dfe854557b178d8af740bc2869c936fc88608b
2018-11-12 15:20:20 -08:00
Yohei Yukawa
530dd76fde Remove IInputMethodSession#finishInput(), which is called nowhere
Currently IInputMethodSession#finishInput() is called nowhere.

Let's remove this IPC method exposed from InputMethodService to IME
client for now until we fully understand what is the right approach on
how and when InputMethodService#finishInput() should be called.

Note that we cannot simply remove InputMethodSession#finishInput()
because it is already published as a public API.  In Bug 9216494
hopefully we can also discuss whether InputMethodSession#finishInput()
should be deprecated or can be reused for some actual use cases.

Bug: 9216494
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Change-Id: I9e378111c9df40cb6821583d1e6ae8f9fe38f2b1
2018-10-22 15:05:47 -07:00
Andrew Solovay
5c05dedda1 cherry-pick from pi-dev docs: Replacing {#link with {@link
Several java files had the typo {#link (for cross-references to other
Javadocs) instead of the proper {@link format. This was confusing the
new doc publish tool (Mivi) since that's the format used for {# Django
comments #}.

Fixed a couple of links that had other errors (which prevented building
once the {# -> {@ was done) and other typos.

Replaced throughout the frameworks/base project; I'll need a separate CL
for the AndroidX fixes.

(Other files were not in the public Javadocs.)

Bug: 111925950
Test: make ds-docs
Change-Id: Ia06e1fffd814671289a1caebd5962aedc18a28d7
Original Change-Id: Ia06e1fffd814671289a1caebd5962aedc18a28d7
Exempt-From-Owner-Approval: Docs-only change
2018-10-04 18:17:05 +00:00
Yohei Yukawa
9d73f2efa0 Deprecate InputMethodService#getInputMethodWindowRecommendedHeight()
InputMethodService#getInputMethodWindowRecommendedHeight() was added
with an assumption that some IMEs may want to call this API in
InputMethodService#onCreate() to adjust its IME window height to be
the same as the previous IME's window height [1], but in reality for
IME developers this API is quite difficult to use because relying on
this API means user-visible behavior is no longer deterministic.
Basically "Recommended" is too vague to rely on.

Let's deprecate this API before we end up having to define what is the
"recommended" height for more complicated scenarios such as
multi-displays and multi-profiles.

With this CL, IMS##getInputMethodWindowRecommendedHeight() always
returns 0.  Basically doing this would not likely to cause
compatibility issues because the possibility of returning 0 has been
clearly mentioned in the API document.  In practice this must have
returned 0 when the previous IME did not show the software keyboard
(e.g. AOSP keyboard with a hardware keyboard).  Therefore IMEs that
have correctly used this API should be able to fall back to a safe
default behavior even if this API returns 0.

 [1]: I0e920ee79c526c3aea6872b063cf294e2ab081c8
      658c7b896a

Fix: 116502957
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Change-Id: Ia2cde031a0e67d45a3631e54226f9b5a0698dd61
2018-09-26 18:19:21 -07:00
Yohei Yukawa
d4d895e2f8 Remove a special optimization for a quite rare scenario
This CL effectively reverts the following 3 CLs.

 * Reduce window resizing during IME transition
   I5723f627ce323b0d12bd7b93f5b35fc4d342b50c
   792faa2c16

 * Clear the inset of previous IME when necessary
   Ib04967f39b2529251e4835c42e9f99dba2cf43f2
   2977eb7b6c

 * Make IMS#clearInsetOfPreviousIme() reliable
   Ib567daa009c1139858dccadcfc6a04465ebecf36
   833bdcedce

The main idea behind the first CL is that the target application can
skil avoid layout resizing if the following two conditions are met.

 A. WindowManagerService (WMS) can remember the last IME's inset until
    the next IME's window is fully shown.
 B. Both the last IME and the new IME have the same inset.

Basically the first CL implements the above A part with an assumption
that some IMEs would do the B part.  However, in reality it is quite
unlikely that two random IMEs have the same inset size.  At the same
time, maintaining this kind of special optimization is getting harder
and harder as more and more use cases and form factors need to be
supported.

Let's remove this optimization given that no one is benefited by it.

Fix: 116492038
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: do not see any noticeable visual difference
Change-Id: I8ffbf9bf7c3a8be54df0ca8eac1a1f041ef7d3c9
2018-09-24 16:01:20 -07:00
Yohei Yukawa
d746a7e893 Let InputMethodPrivilegedOperationsRegistry deal with its singleton-ness
This is a preparation to allow InputMethodManager to have per-display
instances rather than the current per-process singleton instance.

When I introduced InputMethodPrivilegedOperationsRegistry [1], there
was an assumption that InputMethodManager was a per-process global
singleton object.

Now that we are going to break up that global per-process instance
into multiple per-display instances, having multiple
InputMethodPrivilegedOperationsRegistry instances probably does not
make much sense, because it would likely to increases the risk of
compability issues in existing IMEs.  Although IME developers soon
really need to use the right Context to obtain the right instance of
InputMethodManager anyway, unnecessarily introducing compatibility
pitfalls that can be avoided in the Framework side is not my
intention.

With this CL, following 9 methods can continue to work no matter
whether InputMethodManager is a per-process singleton or not.
This is fine because those APIs had been mistakenly exposed in
InputMethodManager and already deprecated in favor of newly added ones
in InputMethodService.

 * InputMethodManager.hideSoftInputFromInputMethod
 * InputMethodManager.hideStatusIcon
 * InputMethodManager.setInputMethod
 * InputMethodManager.setInputMethodAndSubtype
 * InputMethodManager.shouldOfferSwitchingToNextInputMethod
 * InputMethodManager.showSoftInputFromInputMethod
 * InputMethodManager.showStatusIcon
 * InputMethodManager.switchToLastInputMethod
 * InputMethodManager.switchToNextInputMethod

 [1]: If762714b2003fa6477e1318110f63e13968c1d7e
      eec552e9e9

Bug: 115893206
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Change-Id: I4a61470f06ffac5f7a512536f8431489db0108f4
2018-09-18 18:55:02 -07:00
Mathew Inwood
8c854f86a4 Move some members to the "Q blacklist".
Based on some analysis, these fields/methods are likely false positives.
Set maxTargetSdk=P so that any apps using them are required to migrate off
them in future. See the bug for more details.

Exempted-From-Owner-Approval: Automatic changes to the codebase
affecting only @UnsupportedAppUsage annotations, themselves added
without requiring owners approval earlier.

Bug: 115609023
Test: m
Change-Id: I719b5c94e5b1f4fa562dd5d655953422958ad37e
2018-09-14 13:18:34 +01:00
lumark
90120a8b5b Support IME Window to show in secondary display.
1) Moving WMS.setInputMethodWindowLocked to DisplayContent,
   each display can have its own IME window.
2) Add getDisplayIdFromWindow in WindowManagerInternal,
   used for InputMethodManagerService to know which display
   for given IME window token.
3) Support add / remove IME window according displayId.
4) Modify WMS.inputMethodClientHasFocus to traverse all active display
   if inputMethodClient focused.
5) Add displayId parameter for IInputMethod.initializeInternal to
   update context display then client can addView to right display.

Note: 1) There should be zero behavior difference as long as the target
         app is running on the default display.
      2) The current implementation is not final and there are still
         chances that the current IME may not work well or even crash
	 depending on how the IME is implemented.

Bug: 111364446
Test: manual, use ActivityView & launch Messages in VirtualDisplay,
      tap search icon to see if soft input keyboard shown &
      app window size is adjusted by soft input.
Change-Id: I8da315936caebdc8b2c16cff4e24192c06743251
2018-09-13 12:52:40 +08:00
Yohei Yukawa
603f4d00c6 Move InputMethodManagerService to com.android.server.inputmethod
This CL moves InputMethodManagerService (IMMS)
  from com.android.server
    to com.android.server.inputmethod
so that we can mechanically factor out inner classes from IMMS to
separate package private classes.

This is purely a mechanical refactoring of an implementation detail.
There should be no observable behavior difference.

Fix: 114660660
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:com.android.internal.inputmethod
Change-Id: I023d8f2863601ee54f304988117d8ea750770f70
2018-09-11 15:04:58 -07:00