Currently, listModelFiles and findBestModelFile methods only support annotator model.
But we want to extend them to support other models as well, like langID and actions.
Thus, introducing ModelFileManager, which provides listModelFiles and
findBestModelFile. ModelFileManager takes a Supplier<List<ModelFile>> to list model files.
For different types of model, we just need to provide a different supplier to the ModelFileManager.
There should be no behavior change.
Test: atest frameworks/base/core/tests/coretests/src/android/view/textclassifier/ModelFileManagerTest.java
Test: atest frameworks/base/core/tests/coretests/src/android/view/textclassifier/TextClassificationManagerTest.java
Change-Id: I4fc3fd1c9246383ee5d906792bb14b96dbf0a79f
Include below refectoring items to support per display AppTransition:
WMS / AM refectoring parts:
- Move AppTransition related stuff from WMS into DisplayContent.
- Move WMS.prepareAppTransition into DisplayWindowController.
- Move WMS.executeAppTransition to DisplayWindowController.
- Move ATM.isNextTransitionForward to DisplayWindowController.
- Move WMS.getPendingAppTransition to DisplayWindowController.
- Move WMS.overrideAppTransition like APIs to DisplayWindowController.
- Move ActivityRecord.applyOptionsLocked to AppContainerController.
- Support tracing all display's AppTransition status for
DisplayContent.pendingLayoutChanges & window hierachy update.
- Modify logics for AppTransition related caller parts.
- Move WindowSurfacePlacer.handleAppTransitionReadyLocked related
stuffs into added class AppTransitionController.
WM unit test parts:
- Add test case for verifying app transition state per display:
- AppTransitionTests.testAppTransitionStateForMultiDisplay
- AppTransitionTests.testCleanAppTransitionWhenTaskStackReparent
- Rename WindowSurfacePlacerTest to AppTransitionControllerTest since
the test is related handle AppTransition flow.
Bug: 111362605
Test: go/wm-smoke
Test: atest ActivityManagerTransitionSelectionTests
Test: atest ActivityManagerMultiDisplayTests
Test: atest FrameworksServicesTests for DisplayContent / AppTransition
related tests.
Change-Id: Ic1793aa794eb161bec31fda57847a6ba2ff4f84f
This is a mechanical refactoring like we did to split
InputMethodClient for Bug 118040692.
So-called "controlFlags" in InputMethodManager (IMM) was originally
introduced for IMM#startInput()/windowGainedFocus() [1] to carry
additional client information then reused when we unify startInput()
and windowGainedFocus() into startInputOrWindowGainedFocus() [2].
This CL mechanically moves the location where those flags are defined,
from InputMethodManager.java to a newly created file
StartInputFlags.java.
This is still just a mechanical refacotring / renaming. There should
be no user-visible behavior change.
[1]: Icb58bef75ef4bf9979f3e2ba88cea20db2e2c3fb
7663d80f6b
[2]: I56934f18e30d90fcdf77bcbb0c35a92a5feb1b82
05c25f8a3a
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:com.android.internal.inputmethod.InputMethodDebugTest
Change-Id: If5a4810dece852edcff0d1119463711249bb7ef2
Also create a new MANAGE_ACCESSIBILITY permission to
perform the shortcut.
Bug: 116118615
Test: make, activate accessibility shortcut
Change-Id: Ic65a0cdf7393429e14cb98f4fb0734d20069b05a
This is another follow up CL to my previous CL [1] that enabled
per-display InputMethodManager (IMM) instance (Bug 115893206).
What we learned in Bug 118341760 is that for some apps
view.getContext() may return a ContextWrapper subclass whose
Context.getDisplayId() and Context.getSystemService() are not
consistent with each other. Although this is considered to be a bug
of such a ContextWrapper subclass, application developers may not be
aware of the issue when such a problematic ContextWrapper came from
libraries that app developers happened to, or had to, depend on.
The key idea of this CL is that
view.getViewRootImpl().getDisplayId()
would be a more robust source of display ID than
view.getContext().getDisplayId()
for most of cases, because to which ViewRootImpl a given View belongs
is already a strong signal about to which display the View belongs.
As far as I've tested locally, this approach seems to be more
promising and is expected to give us better app compatibility in
multi-display scenarios.
[1]: I78ad7cccb9586474c83f7e2f90c0bcabb221c47b
4052a10f29
Bug: 118252837
Fix: 118341760
Test: atest ActivityManagerMultiDisplayTests
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Change-Id: If47e48bc8176657bf4bb882146d2affdfe457c90
1. There used to be double offset from the origin.
This is because we used to (in NYC) make the surface position to display
origin and draw content with a offset in the surface. However we lately
let the surface position be inherited from task surface position, so
there will be an offset from surface position and an offset from drawing
content in that surface.
To fix the bug I removed the offset in drawing content. That offset is
provided by WindowState#getBackdropFrame() so it just solved the issue
by moving frame to the origin.
2. Window quickly jumps when user starts drag resizing a window.
The reason is out of synchronization between surface insets change and
graphical buffer update. When user is drag resizing, we suppress window
shadow to save some graphical resources, which will consequently change
surface insets. Change in surface insets will cause the surface being
repositioned to reflect the new surface insets because window frame
doesn't change. However the content is still drawn at old location with
old surface insets for the first a few frames, so the content jumps to a
wrong location for a split second. This also happens when users stop
drag resizing.
I kept the old surface insets when user is resizing so there won't be
surface reposition at the beginning and end of drag resizing, but still
suppress the shadow by adjusting the elevation of DecorView.
Also fixed a synchronization issue we found in BackdropFrameRenderer,
and cleaned up code in it.
Bug: 113254346
Test: Manual tests show drag resizing for both freeform and split screen
works.
Change-Id: I42349f88f14af35fac7c65e784462b5f2e1a71c7
This is a follow up CL to my previous CL [1] that enabled per-display
InputMethodManager (IMM) instance.
One thing I mistakenly assumed in my previous CL is that there is
always a process boundary between IMM#createRealInstance() and
IMMS#addClient(), which is not true when IMM is being instantiated in
the system server process. This means that IMMS may associate an IME
client to wrong PID/UID/DisplayID when certain conditions are met.
With this CL, IMM#createRealInstance() always clears calling identity
so chat IMMS#addClient() can work correctly even for local Binder call
scenarios.
[1]: I78ad7cccb9586474c83f7e2f90c0bcabb221c47b
4052a10f29
Bug: 115893206
Bug: 117594679
Fix: 118335706
Test: atest ActivityManagerMultiDisplayTests
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Test: Manually tested with manually simulating the problematic scenario
Change-Id: I318d32d8997b0acb4fb193fe46831bdc9a4dd083
This CL aims to factor out InputMethodManager#forContext() and its
related methods. This is still a mechanical refactoring hence there
should be no behavior change.
Bug: 115893206
Bug: 117594679
Bug: 118335706
Test: atest ActivityManagerMultiDisplayTests
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Change-Id: I457745e18bfbd3e2787c5b438eee962a1928326e
In LayoutInflater, moved the member mContext to dark grey list of APIs,
disabling access for apps targeting Q. Developers should use the provided
getter getContext to retrieve the context or the method cloneInContext to
create a LayoutInflater copy with a different context. Both methods are
available since API 1.
Bug: 117519957
Test: android.view.cts.LayoutInflaterTest
Change-Id: Id86969dddb14fafd76fda1b0a97c7f8fc09aca3e
- Includes some fixes to handle null ParcelFileDescriptors.
- Closes fds immediately after the model has been loaded.
Bug: 116020587
Test: atest android.view.textclassifier.TextClassificationManagerTest
Change-Id: Ieb05d081847ac218d2a5b46db95cd512838f67ab
TouchDelegate allows nullable bounds of delegated view. We provide
a default bounds to create TouchDelegateInfo if it's null.
Bug: 117951101
Test: atest atest AccessibilityEndToEndTest
Change-Id: I914a44520edf159bba37af0b0eb00ab97c00b177
This is another step to split InputMethodClient into multiple classes.
With this CL, InputMethodClient is completely removed.
This is a mechanical refactoring. There should be no user-visible
behavior change.
Bug: 118040692
Test: prebuilts/checkstyle/checkstyle.py -f \
frameworks/base/core/java/com/android/internal/inputmethod/UnbindReason.java
Change-Id: I3b96a351413025338776f6c87cbaa8cf28c3a44f
This is another step to split InputMethodClient into multiple classes.
Now that all the integer constants for StartInputReason are defined
inside StartInputReason, "START_INPUT_REASON_" prefix is just
redundant.
This is a mechanical refactoring. There should be no user-visible
behavior change.
Bug: 118040692
Test: prebuilts/checkstyle/checkstyle.py -f \
frameworks/base/core/java/com/android/internal/inputmethod/StartInputReason.java
Change-Id: Ic2476c3d588211e6c61180cde7df4c6b79039ede
This is another step to split InputMethodClient into multiple classes.
With this CL, StartInputReason will be extracted from
InputMethodClient.java into a dedicated file.
This is a mechanical refactoring. There should be no user-visible
behavior change.
Bug: 118040692
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Test: prebuilts/checkstyle/checkstyle.py -f \
frameworks/base/core/java/com/android/internal/inputmethod/StartInputReason.java
Change-Id: I0cc2588c97239a004720f74cbf356bda4c735d53
This is the first step to split InputMethodClient into multiple
classes.
With this CL, utility methods to convert integer constants to String
messages will be moved from InputMethodClient to InputMethodDebug,
which I believe is a bit more descriptive class name than
InputMethodClient.
This is a mechanical refactoring. There should be no user-visible
behavior change.
Bug: 118040692
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Test: prebuilts/checkstyle/checkstyle.py -f \
frameworks/base/core/java/com/android/internal/inputmethod/InputMethodDebug.java
Change-Id: I83f4795e95bc2e8ae325ad6e28d3a42317414e8d
It also introduces a command line tool to rotate external displays for
testing purposes.
Instead of changing signatures of freezeRotation(), thawRotation() and
isRotationFrozen(), introduce 3 new methods to IWindowManager interface.
The old methods are being used pervasively.
Also resolved some style issues in DisplaySettings class.
Bug: 113252523
Bug: 111361251
Test: Rotation locks via settings still work. User rotation mode and
user rotation values for external devices are persisted as expected in
storage. DisplaySettingsTests passes.
Change-Id: I1746bea98a588f0bbd30c9f0617a7a23dc6e4209
Currently InputMethodManagerService#finishInput() does nothing. Until
we fully understand what is the right approach on how and when
InputMethodService#finishInput() (Bug 9216494), let's remove this
unnecessary IPC from the IME client to InputMethodManagerService.
Bug: 9216494
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Change-Id: I614050d20f4a7d9611dc0502e55e6ca3458a836e
The CL moves mChildren and mChildrenCount in ViewGroup to the dark-grey
list of APIs, disabling access to them for apps targeting Q. Developers
should probably use ViewGroup#getChildCount() and ViewGroup#getChildAt()
which exist since public API 1.
Bug: 117521014
Bug: 117521406
Test: atest core/tests/coretests/src/android/view/
Change-Id: I14d3ebd1b16edc92cd7b370404b1f05cd304ab7d
InputMethodManager has been a per-process singleton object. In order
to support behavior changes for multi-display support in Android Q,
however, InputMethodManager now needs to be per-display objects.
With this CL, context.getSystemService(InputMethodManager.class) will
start returning per-display InputMethodManager (IMM) instance.
Why?
There are two major reasons.
1. To support per-display focused window.
2. To support more simplified API for multi-session IME.
Currently per-process InputMethodManager instance directly receives
callback from ViewRootImpl upon windowFocusChanged, then it keeps
track of which Window is focused by storing its root view into
InputMethodManager#mCurRootView.
This design assumes that (within the same process) at most one Window
can have window focus, which is no longer true once we start
supporting per-display focused window (Bug 111361570).
Why we need to do this to support per-display focused window:
For traditional non multi-session IME cases (e.g. apps that use
Virtual Display APIs on phones), internal state of IMM can be easily
messed up once the system starts sending per-display
windowFocusChanged events to the same process, because IMM still
doesn't know that now each display has focused window. It is hard to
precisely predict what kind of issues we would see simply because such
a use case is most likely not expected in the original design.
Why we need to do this for multi-session IME:
For multi-session IME scenarios, in addition to the above concern in
InputMethodManager, the current design allows at most one IME session
per process. This means that if a process X is showing Activities to 3
different displays, only one Activity can interact with the
multi-session IME at the same time. If we do not change the current
design, the only way to work around is to ask app developers to
explicitly use different processes for each Activity, which may
require a lot of work (e.g. SharedPreference is not optimized for
multi-process use cases). This would also make multi-session IME
development complicated because the IME cannot know on which display
the IME is interacting until startInputOrWindowGainedFocus() is
actually called, and needs to do all the preparation and cleanup tasks
whenever startInputOrWindowGainedFocus() is called for a different
display than it's currently interacting with.
Alternative solutions considered:
Another possible approach is to update InputMethodManager singleton to
be able to maintain multiple mCurRootView and mServedView for each
display. This approach was abandoned because those fields and methods
are already marked as @UnsupportedAppUsage. I concluded that touching
@UnsupportedAppUsage things would have bigger compatibility risks than
per-display instance model.
Implementation note:
* Public APIs in IMM that take View instance as the first parameter
will verify whether the given View and IMM are associated with the
same display ID or not. If there is a display ID mismatch, such an
API call will be automatically forwarded to the correct IMM instance
IMM with a clear warning in logcat which tells that app developers
should use the correct IMM instance to avoid unnecessary performance
overhead.
* As a general rule, system server process cannot trust display ID
reported from applications. In order to enable IMMS to verify the
reported display ID, this CL also exposes display ID verification
logic from WMS to other system components via WindowManagerInternal.
* isInputMethodClientFocus() in WindowManagerService (WMS) is updated
to use top-focused-display to determine whether a given IME client
has IME focus or not. This is now necessary because with a recent
change [1] each display can have focused window. The previous logic
to check all the displays that belong to the given pid/uid [2] no
longer makes sense.
* Currently per-display InputMethodManager instances will not be
garbage collected because InputMethodManager#sInstanceMap keeps
holding strong references to them. Freeing those instances is
technically possible, but we need to be careful because multiple
processes (app, system, IME) are involved and at least system
process has a strict verification logic that lets the calling
process crash with SecurityException. We need to carefully
implement such a cleanup logic to avoid random process crash due to
race condition. Bug 116699479 will take care of this task.
Also to make sure that the performance regression (Bug 117434607) we
observed after my initial attempt [3] no longer exists, here are the
benchmark results with and without this CL.
testExpandNotificationsLatency on taimen-userdebug
without this CL:
results=[55, 46, 61, 67, 50, 48, 57, 50, 55, 63]
min:46.0, max:67.0, avg:55.2, median:55.0, std_dev:6.539
with this CL:
results=[45, 55, 58, 57, 47, 60, 59, 60, 56, 53]
min:45.0, max:60.0, avg:55.0, median:56.5, std_dev:4.980
[1]: I776cabaeaf41ff4240f504fb1430d3e40892023d
1e5b10a217
[2]: I8da315936caebdc8b2c16cff4e24192c06743251
90120a8b5b
[3]: I7242e765426353672823fcc8277f20ac361930d7
c53d78e992
Bug: 111364446
Fix: 115893206
Test: atest ActivityManagerMultiDisplayTests
Test: atest CtsInputMethodTestCases CtsInputMethodServiceHostTestCases
Test: atest FrameworksCoreTests:android.view.inputmethod.InputMethodManagerTest
Test: No perf regression in LatencyTests#testExpandNotificationsLatency()
Change-Id: I78ad7cccb9586474c83f7e2f90c0bcabb221c47b
Before P, if startDragAndDrop is invoked twice, the system goes to wired state
where:
* the system server keeps processing the first drag operation.
* the application loses the drag token but keeps the user local state.
* cancelDrag() no longer cancels the ongoing operation.
* DragEvents are still delivered with the user local state.
At P we unintentionally changed the behavior to:
* the system server keeps processing the first drag operation.
* the application loses the drag token and the user local state.
* cancelDrag() no longer cancels the ongoing operation.
* DragEvents are still delivered without the user local state.
The CL fixed the behavior so that the second startDragAndDrop() calls
does not affect the internal state of drag and drop as it's failed due
to existing ongoing operation.
* the system server keeps processing the first drag operation.
* the application keeps the drag token and the user local state.
* cancelDrag() is still able to cancel the ongoing operation.
* DragEvents are still delivered with the user local state.
Bug: 113310888
Test: Manually invoke startDragAndDrop() and ensures the user local
state delivered with DragEvents is not cleared.
Change-Id: I0a8315a44d655a8a73b7034f340e50e2f50601a8
