This is another follow up CL to my previous CL [1] that enabled
per-display InputMethodManager (IMM) instance (Bug 115893206).
What we learned in Bug 118341760 is that for some apps
view.getContext() may return a ContextWrapper subclass whose
Context.getDisplayId() and Context.getSystemService() are not
consistent with each other. Although this is considered to be a bug
of such a ContextWrapper subclass, application developers may not be
aware of the issue when such a problematic ContextWrapper came from
libraries that app developers happened to, or had to, depend on.
The key idea of this CL is that
view.getViewRootImpl().getDisplayId()
would be a more robust source of display ID than
view.getContext().getDisplayId()
for most of cases, because to which ViewRootImpl a given View belongs
is already a strong signal about to which display the View belongs.
As far as I've tested locally, this approach seems to be more
promising and is expected to give us better app compatibility in
multi-display scenarios.
[1]: I78ad7cccb9586474c83f7e2f90c0bcabb221c47b
4052a10f29
Bug: 118252837
Fix: 118341760
Test: atest ActivityManagerMultiDisplayTests
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Change-Id: If47e48bc8176657bf4bb882146d2affdfe457c90
1. There used to be double offset from the origin.
This is because we used to (in NYC) make the surface position to display
origin and draw content with a offset in the surface. However we lately
let the surface position be inherited from task surface position, so
there will be an offset from surface position and an offset from drawing
content in that surface.
To fix the bug I removed the offset in drawing content. That offset is
provided by WindowState#getBackdropFrame() so it just solved the issue
by moving frame to the origin.
2. Window quickly jumps when user starts drag resizing a window.
The reason is out of synchronization between surface insets change and
graphical buffer update. When user is drag resizing, we suppress window
shadow to save some graphical resources, which will consequently change
surface insets. Change in surface insets will cause the surface being
repositioned to reflect the new surface insets because window frame
doesn't change. However the content is still drawn at old location with
old surface insets for the first a few frames, so the content jumps to a
wrong location for a split second. This also happens when users stop
drag resizing.
I kept the old surface insets when user is resizing so there won't be
surface reposition at the beginning and end of drag resizing, but still
suppress the shadow by adjusting the elevation of DecorView.
Also fixed a synchronization issue we found in BackdropFrameRenderer,
and cleaned up code in it.
Bug: 113254346
Test: Manual tests show drag resizing for both freeform and split screen
works.
Change-Id: I42349f88f14af35fac7c65e784462b5f2e1a71c7
This is a follow up CL to my previous CL [1] that enabled per-display
InputMethodManager (IMM) instance.
One thing I mistakenly assumed in my previous CL is that there is
always a process boundary between IMM#createRealInstance() and
IMMS#addClient(), which is not true when IMM is being instantiated in
the system server process. This means that IMMS may associate an IME
client to wrong PID/UID/DisplayID when certain conditions are met.
With this CL, IMM#createRealInstance() always clears calling identity
so chat IMMS#addClient() can work correctly even for local Binder call
scenarios.
[1]: I78ad7cccb9586474c83f7e2f90c0bcabb221c47b
4052a10f29
Bug: 115893206
Bug: 117594679
Fix: 118335706
Test: atest ActivityManagerMultiDisplayTests
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Test: Manually tested with manually simulating the problematic scenario
Change-Id: I318d32d8997b0acb4fb193fe46831bdc9a4dd083
This CL aims to factor out InputMethodManager#forContext() and its
related methods. This is still a mechanical refactoring hence there
should be no behavior change.
Bug: 115893206
Bug: 117594679
Bug: 118335706
Test: atest ActivityManagerMultiDisplayTests
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Change-Id: I457745e18bfbd3e2787c5b438eee962a1928326e
In LayoutInflater, moved the member mContext to dark grey list of APIs,
disabling access for apps targeting Q. Developers should use the provided
getter getContext to retrieve the context or the method cloneInContext to
create a LayoutInflater copy with a different context. Both methods are
available since API 1.
Bug: 117519957
Test: android.view.cts.LayoutInflaterTest
Change-Id: Id86969dddb14fafd76fda1b0a97c7f8fc09aca3e
- Includes some fixes to handle null ParcelFileDescriptors.
- Closes fds immediately after the model has been loaded.
Bug: 116020587
Test: atest android.view.textclassifier.TextClassificationManagerTest
Change-Id: Ieb05d081847ac218d2a5b46db95cd512838f67ab
TouchDelegate allows nullable bounds of delegated view. We provide
a default bounds to create TouchDelegateInfo if it's null.
Bug: 117951101
Test: atest atest AccessibilityEndToEndTest
Change-Id: I914a44520edf159bba37af0b0eb00ab97c00b177
This is another step to split InputMethodClient into multiple classes.
With this CL, InputMethodClient is completely removed.
This is a mechanical refactoring. There should be no user-visible
behavior change.
Bug: 118040692
Test: prebuilts/checkstyle/checkstyle.py -f \
frameworks/base/core/java/com/android/internal/inputmethod/UnbindReason.java
Change-Id: I3b96a351413025338776f6c87cbaa8cf28c3a44f
This is another step to split InputMethodClient into multiple classes.
Now that all the integer constants for StartInputReason are defined
inside StartInputReason, "START_INPUT_REASON_" prefix is just
redundant.
This is a mechanical refactoring. There should be no user-visible
behavior change.
Bug: 118040692
Test: prebuilts/checkstyle/checkstyle.py -f \
frameworks/base/core/java/com/android/internal/inputmethod/StartInputReason.java
Change-Id: Ic2476c3d588211e6c61180cde7df4c6b79039ede
This is another step to split InputMethodClient into multiple classes.
With this CL, StartInputReason will be extracted from
InputMethodClient.java into a dedicated file.
This is a mechanical refactoring. There should be no user-visible
behavior change.
Bug: 118040692
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Test: prebuilts/checkstyle/checkstyle.py -f \
frameworks/base/core/java/com/android/internal/inputmethod/StartInputReason.java
Change-Id: I0cc2588c97239a004720f74cbf356bda4c735d53
This is the first step to split InputMethodClient into multiple
classes.
With this CL, utility methods to convert integer constants to String
messages will be moved from InputMethodClient to InputMethodDebug,
which I believe is a bit more descriptive class name than
InputMethodClient.
This is a mechanical refactoring. There should be no user-visible
behavior change.
Bug: 118040692
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Test: prebuilts/checkstyle/checkstyle.py -f \
frameworks/base/core/java/com/android/internal/inputmethod/InputMethodDebug.java
Change-Id: I83f4795e95bc2e8ae325ad6e28d3a42317414e8d
It also introduces a command line tool to rotate external displays for
testing purposes.
Instead of changing signatures of freezeRotation(), thawRotation() and
isRotationFrozen(), introduce 3 new methods to IWindowManager interface.
The old methods are being used pervasively.
Also resolved some style issues in DisplaySettings class.
Bug: 113252523
Bug: 111361251
Test: Rotation locks via settings still work. User rotation mode and
user rotation values for external devices are persisted as expected in
storage. DisplaySettingsTests passes.
Change-Id: I1746bea98a588f0bbd30c9f0617a7a23dc6e4209
Currently InputMethodManagerService#finishInput() does nothing. Until
we fully understand what is the right approach on how and when
InputMethodService#finishInput() (Bug 9216494), let's remove this
unnecessary IPC from the IME client to InputMethodManagerService.
Bug: 9216494
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Change-Id: I614050d20f4a7d9611dc0502e55e6ca3458a836e
The CL moves mChildren and mChildrenCount in ViewGroup to the dark-grey
list of APIs, disabling access to them for apps targeting Q. Developers
should probably use ViewGroup#getChildCount() and ViewGroup#getChildAt()
which exist since public API 1.
Bug: 117521014
Bug: 117521406
Test: atest core/tests/coretests/src/android/view/
Change-Id: I14d3ebd1b16edc92cd7b370404b1f05cd304ab7d
InputMethodManager has been a per-process singleton object. In order
to support behavior changes for multi-display support in Android Q,
however, InputMethodManager now needs to be per-display objects.
With this CL, context.getSystemService(InputMethodManager.class) will
start returning per-display InputMethodManager (IMM) instance.
Why?
There are two major reasons.
1. To support per-display focused window.
2. To support more simplified API for multi-session IME.
Currently per-process InputMethodManager instance directly receives
callback from ViewRootImpl upon windowFocusChanged, then it keeps
track of which Window is focused by storing its root view into
InputMethodManager#mCurRootView.
This design assumes that (within the same process) at most one Window
can have window focus, which is no longer true once we start
supporting per-display focused window (Bug 111361570).
Why we need to do this to support per-display focused window:
For traditional non multi-session IME cases (e.g. apps that use
Virtual Display APIs on phones), internal state of IMM can be easily
messed up once the system starts sending per-display
windowFocusChanged events to the same process, because IMM still
doesn't know that now each display has focused window. It is hard to
precisely predict what kind of issues we would see simply because such
a use case is most likely not expected in the original design.
Why we need to do this for multi-session IME:
For multi-session IME scenarios, in addition to the above concern in
InputMethodManager, the current design allows at most one IME session
per process. This means that if a process X is showing Activities to 3
different displays, only one Activity can interact with the
multi-session IME at the same time. If we do not change the current
design, the only way to work around is to ask app developers to
explicitly use different processes for each Activity, which may
require a lot of work (e.g. SharedPreference is not optimized for
multi-process use cases). This would also make multi-session IME
development complicated because the IME cannot know on which display
the IME is interacting until startInputOrWindowGainedFocus() is
actually called, and needs to do all the preparation and cleanup tasks
whenever startInputOrWindowGainedFocus() is called for a different
display than it's currently interacting with.
Alternative solutions considered:
Another possible approach is to update InputMethodManager singleton to
be able to maintain multiple mCurRootView and mServedView for each
display. This approach was abandoned because those fields and methods
are already marked as @UnsupportedAppUsage. I concluded that touching
@UnsupportedAppUsage things would have bigger compatibility risks than
per-display instance model.
Implementation note:
* Public APIs in IMM that take View instance as the first parameter
will verify whether the given View and IMM are associated with the
same display ID or not. If there is a display ID mismatch, such an
API call will be automatically forwarded to the correct IMM instance
IMM with a clear warning in logcat which tells that app developers
should use the correct IMM instance to avoid unnecessary performance
overhead.
* As a general rule, system server process cannot trust display ID
reported from applications. In order to enable IMMS to verify the
reported display ID, this CL also exposes display ID verification
logic from WMS to other system components via WindowManagerInternal.
* isInputMethodClientFocus() in WindowManagerService (WMS) is updated
to use top-focused-display to determine whether a given IME client
has IME focus or not. This is now necessary because with a recent
change [1] each display can have focused window. The previous logic
to check all the displays that belong to the given pid/uid [2] no
longer makes sense.
* Currently per-display InputMethodManager instances will not be
garbage collected because InputMethodManager#sInstanceMap keeps
holding strong references to them. Freeing those instances is
technically possible, but we need to be careful because multiple
processes (app, system, IME) are involved and at least system
process has a strict verification logic that lets the calling
process crash with SecurityException. We need to carefully
implement such a cleanup logic to avoid random process crash due to
race condition. Bug 116699479 will take care of this task.
Also to make sure that the performance regression (Bug 117434607) we
observed after my initial attempt [3] no longer exists, here are the
benchmark results with and without this CL.
testExpandNotificationsLatency on taimen-userdebug
without this CL:
results=[55, 46, 61, 67, 50, 48, 57, 50, 55, 63]
min:46.0, max:67.0, avg:55.2, median:55.0, std_dev:6.539
with this CL:
results=[45, 55, 58, 57, 47, 60, 59, 60, 56, 53]
min:45.0, max:60.0, avg:55.0, median:56.5, std_dev:4.980
[1]: I776cabaeaf41ff4240f504fb1430d3e40892023d
1e5b10a217
[2]: I8da315936caebdc8b2c16cff4e24192c06743251
90120a8b5b
[3]: I7242e765426353672823fcc8277f20ac361930d7
c53d78e992
Bug: 111364446
Fix: 115893206
Test: atest ActivityManagerMultiDisplayTests
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Test: No perf regression in LatencyTests#testExpandNotificationsLatency()
Change-Id: I78ad7cccb9586474c83f7e2f90c0bcabb221c47b
Before P, if startDragAndDrop is invoked twice, the system goes to wired state
where:
* the system server keeps processing the first drag operation.
* the application loses the drag token but keeps the user local state.
* cancelDrag() no longer cancels the ongoing operation.
* DragEvents are still delivered with the user local state.
At P we unintentionally changed the behavior to:
* the system server keeps processing the first drag operation.
* the application loses the drag token and the user local state.
* cancelDrag() no longer cancels the ongoing operation.
* DragEvents are still delivered without the user local state.
The CL fixed the behavior so that the second startDragAndDrop() calls
does not affect the internal state of drag and drop as it's failed due
to existing ongoing operation.
* the system server keeps processing the first drag operation.
* the application keeps the drag token and the user local state.
* cancelDrag() is still able to cancel the ongoing operation.
* DragEvents are still delivered with the user local state.
Bug: 113310888
Test: Manually invoke startDragAndDrop() and ensures the user local
state delivered with DragEvents is not cleared.
Change-Id: I0a8315a44d655a8a73b7034f340e50e2f50601a8
The perf regression found in my initial attempt [1] to instantiate
InputMethodManager (IMM) for each display revieled that when a Window
gained/lost focus,
getContext().getSystemService(InputMethodManager.class)
gets called for all the View objects that belong to the Window.
This CL introduces a private utility method
View.notifyFocusChangeToInputMethodManager()
to replace existing unnecessary acquisitions of IMM in View.java,
including the most concerning one View.onWindowFocusChanged().
There should be no negative side-effect in doing this optimization.
LatencyTests results:
testExpandNotificationsLatency on taimen-userdebug
without this CL:
results=[43, 46, 58, 47, 52, 59, 55, 59, 58, 46]
min: 43.0, max:59.0, avg:54.7, median:53.5, std_dev:5.967
with this CL:
results=[41, 58, 55, 59, 60, 67, 51, 55, 55, 55]
min: 41.0, max:67.0, avg:55.6, median:55.0, std_dev:6.344
[1]: I7242e765426353672823fcc8277f20ac361930d7
c53d78e992
Bug: 115893206
Test: atest ActivityManagerMultiDisplayTests
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Test: No perf regression observed in Bug 117434607
Change-Id: I5c64b23c3f5cb16f7f3fb9cdc2be063083566050
Removing the child whose visibility changed in
ViewGroup#OnChildVisibilityChanged causes a null pointer exception since
the parent is null immediately after. This CL prevents that from
keeping the parent stored.
Bug: 117520801
Test: remove child in OnChildVisibilityChanged, observe no crash
Change-Id: Ifd20c2fcba9aee476a7714794a90c7ec9a0b3b84
Also add a special API to set them. Internally they are still just
regular private flags
Test: Built
Bug: 116798569
Change-Id: I687b751fa18c7fbcc9bf95aa44d94d8a5614a88f
Previously we have added methods to manipulate color transform for each
surface, this patch exposes this API to Java code land for WindowManager or
display service to set the color transform.
BUG: 111562338
Test: Build, flash and boot.
Change-Id: I0388eed5d72b043820786264f060cde2bd7a6aea
Adding api to get touch delegate behavior for the
represented view of AccessibilityNodeInfo.
Bug: 80061718
Test: atest AccessibilityNodeInfoTest
Test: atest AccessibilityEndToEndTest
Change-Id: I2ae65d7d44fceaf16609e512c3384f766266ecbd
MotionEvent.otain function now can specific displayId.
Test: atest view.MotionEventTest view.KeyEventTes
Bug: 117471623
Change-Id: I5887a4443df6b238a51aba2dbbac994b2327ce68
InputMethodManager has been a per-process singleton object. In order
to support behavior changes for multi-display support in Android Q,
however, InputMethodManager now needs to be per-display objects.
With this CL, context.getSystemService(InputMethodManager.class) will
start returning per-display InputMethodManager (IMM) instance.
Why?
There are two major reasons.
1. To support per-display focused window.
2. To support more simplified API for multi-session IME.
Currently per-process InputMethodManager instance directly receives
callback from ViewRootImpl upon windowFocusChanged, then it keeps
track of which Window is focused by storing its root view into
InputMethodManager#mCurRootView.
This design assumes that (within the same process) at most one Window
can have window focus, which is no longer true once we start
supporting per-display focused window (Bug 111361570).
Why we need to do this to support per-display focused window:
For traditional non multi-session IME cases (e.g. apps that use
Virtual Display APIs on phones), internal state of IMM can be easily
messed up once the system starts sending per-display
windowFocusChanged events to the same process, because IMM still
doesn't know that now each display has focused window. It is hard to
precisely predict what kind of issues we would see simply because such
a use case is most likely not expected in the original design.
Why we need to do this for multi-session IME:
For multi-session IME scenarios, in addition to the above concern in
InputMethodManager, the current design allows at most one IME session
per process. This means that if a process X is showing Activities to 3
different displays, only one Activity can interact with the
multi-session IME at the same time. If we do not change the current
design, the only way to work around is to ask app developers to
explicitly use different processes for each Activity, which may
require a lot of work (e.g. SharedPreference is not optimized for
multi-process use cases). This would also make multi-session IME
development complicated because the IME cannot know on which display
the IME is interacting until startInputOrWindowGainedFocus() is
actually called, and needs to do all the preparation and cleanup tasks
whenever startInputOrWindowGainedFocus() is called for a different
display than it's currently interacting with.
Alternative solutions considered:
Another possible approach is to update InputMethodManager singleton to
be able to maintain multiple mCurRootView and mServedView for each
display. This approach was abandoned because those fields and methods
are already marked as @UnsupportedAppUsage. I concluded that touching
@UnsupportedAppUsage things would have bigger compatibility risks than
per-display instance model.
Implementation note:
* Public APIs in IMM that take View instance as the first parameter
will verify whether the given View and IMM are associated with the
same display ID or not. If there is a display ID mismatch, such an
API call will be automatically forwarded to the correct IMM instance
IMM with a clear warning in logcat which tells that app developers
should use the correct IMM instance to avoid unnecessary performance
overhead.
* As a general rule, system server process cannot trust display ID
reported from applications. In order to enable IMMS to verify the
reported display ID, this CL also exposes display ID verification
logic from WMS to other system components via WindowManagerInternal.
* isInputMethodClientFocus() in WindowManagerService (WMS) is updated
to use top-focused-display to determine whether a given IME client
has IME focus or not. This is now necessary because with a recent
change [1] each display can have focused window. The previous logic
to check all the displays that belong to the given pid/uid [2] no
longer makes sense.
* Currently per-display InputMethodManager instances will not be
garbage collected because InputMethodManager#sInstanceMap keeps
holding strong references to them. Freeing those instances is
technically possible, but we need to be careful because multiple
processes (app, system, IME) are involved and at least system
process has a strict verification logic that lets the calling
process crash with SecurityException. We need to carefully
implement such a cleanup logic to avoid random process crash due to
race condition. Bug 116699479 will take care of this task.
[1]: I776cabaeaf41ff4240f504fb1430d3e40892023d
1e5b10a217
[2]: I8da315936caebdc8b2c16cff4e24192c06743251
90120a8b5b
Bug: 111364446
Fix: 115893206
Test: atest ActivityManagerMultiDisplayTests
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Change-Id: I7242e765426353672823fcc8277f20ac361930d7
