* changes:
Revert "Uses SF callback windows for the A11yWindowInfo [1/n]"
Revert "Uses SF callback windows for the A11yWindowInfo [2/n]"
Revert "Use shellRoot to identify PIP menu window"
Revert "Apply display transform to window bounds and node bounds"
Revert submission 16249364-Surface Flinger
Reason for revert: Large increase (~200%) in system_server memory usage (b/214255324)
Reverted Changes:
I14c5e9889:Apply display transform to window bounds and node ...
I0ac0d0487:Use shellRoot to identify PIP menu window
I83e898d91:Uses SF callback windows for the A11yWindowInfo [2...
I94d7ed4fb:Uses SF callback windows for the A11yWindowInfo [1...
Change-Id: Icb3b07750ebaa0c26d38165dd1a01b2af4b8b5a6
* changes:
Apply display transform to window bounds and node bounds
Use shellRoot to identify PIP menu window
Uses SF callback windows for the A11yWindowInfo [2/n]
Uses SF callback windows for the A11yWindowInfo [1/n]
1. Implements AccessibilityWindowsPopulator class including the
new accessibilityWindow class generated by the inputWindowHandle
from the surface flinger.
2. Replaces the windowState by the accessibilityWindow used in the
AccessibiltyController class for computing the changed windows
reported to the A11y framework.
3. The embedded hiarerchy windows would be reported to A11y framework
due to they are reported from the surface fligner. This will make
the traversal function is broken when the Auotfill features is using
talkback. We find out the embedded hiarerchy windows by its window
token and don't add them into the A11y windows list.
4. Removes the modal windows calculation of the unAccountedSpace
including the task fragment due to we intersec the region between
the frame and touch region for all windows.
5. When reported windows counts from the surface flinger changed,
calling the accessibilty controller class to compute the changed
windows and send the results to the A11y framework immediately.
This can avoid the CTS failure with no corresponding window found
when the activity is launched due to the timing issue between the
callbacks for computing changed windows from the WM and the
callback for changed windows from the surface flinger.
Bug: 191736824
Test: a11y CTS & unit tests
Test: Manual testing including the A11y services
Change-Id: I94d7ed4fbc5abab2252ddccae55678014ac9e600
There are 3 milestones in this feature.
1. Refactor the callbacks for Accessibility in WindowManagerInternal.
2. Implement this feature in such new architecture.
3. Implement the setting choice in preference page.
This CL is for the 3rd milestone.
We add a ACCESSIBILITY_MAGNIFICATION_FOLLOW_TYPING_ENABLED key to store
a user preference for following typing focus. The default value of
preference is true.
If the value is false, we wouldn't do anything in
onRectangleOnScreenRequested and just return.
If the value is true, we would dispatch the event to an activated
magnification controller.
Bug: 194668976
Test: atest AccessibilityManagerServiceTest
atest AccessibilityUserStateTest
atest FullScreenMagnificationControllerTest
atest MagnificationControllerTest
atest WindowMagnificationManagerTest
atest SettingsBackupTest
Change-Id: I7203698d2edcc474ab1638ac81e72e054a61d6fd
There are 3 milestones in this feature.
1. Refactor the callbacks for Accessibility in WindowManagerInternal.
2. Implement this feature in such new architecture.
3. Implement the setting choice in preference page.
This CL is for the 2nd milestone.
We move the window magnification to the typing focus' center position
based on the condition of whether a user takes control or not. We only
make a movement when the control is not taken by a user or we don't
preform the movement for the window magnification.
There are 2 methods for a user to take the control.
1. A user use 1 finger to drag the window magnification.
2. A user use 2 finger to drag the window magnification.
There is 1 method for a user to release the control.
1. When IME is shown, the control would be released.
So, we can decide whether we should make a movement to typing focus
given the condition of who take the control.
Bug: 194668976
Test: atest MagnificationControllerTest
atest WindowMagnificationTest
atest WindowMagnificationControllerTest
atest WindowMagnificationManagerTest
Change-Id: I145f893d412b74c20afe1685449370d1dba99961
The flaky is caused by the deadlock.
WindowMagnificationManager has its own lock and the
WindowMagnificationConnectionWrapper which uses
AccessibilityTraceManager. When the public method of
WindowMagnificationManager is invoked, it hold the lock of
WindowMagnificationManager and the lock of
AccessibilityManagerService. However, when the service connection
is unbound, it will hold the lock of AccessibilityManagerService
first then the lock of WindowMagnificationManager.
To simplify the archiecuture, we inject the lock of
AccessibilityManagerService to WindowMagnificationManager.
Bug: 213120663
Test: atest com.android.server.accessibility
atest CtsAccessibilityServiceTestCases
Change-Id: I818c57a5140269a5800eb1a52e25aa2ce0919245
See go/b200769372 - the legacy public methods
The legacy magnification controller methods of a11y service,
getMagnificationRegion(),
reset(),
will keep the behavior before that control
only full-screen magnification.
To make the a11y service able to control the actiavted
magnifier on the display, the service should use the
new public APIs,
1. getCurrentMagnificationRegion(),
2. resetCurrentMagnification(),
instead of the legacy APIs.
Bug: 210069654
Test: atest AccessibilityMagnificationTest,
atest MagnificationProcessorTest,
atest AbstractAccessibilityServiceConnectionTest,
Change-Id: I76bfee43cbb76a9aeec33574a8db18430903e08c
See Design doc: go/b200769372
To make the behavior consistent on the new platform,
The legcay callback,
onMagnificationChanged(MagnificationController controller,
Region region, float scale, float centerX, float centerY)
keep notifying only full-screen magnification change as
before.
To support listening to the magnification changes of all
magnification modes, the service should overide
The new callback proposed in T,
onMagnificationChanged(MagnificationController controller,
Region region, MagnificationConfig config).
TODO: Notify magnifcation change when window magnifier turns off
Bug: 203013925
Test: atest AccessibilityMagnificationTest,
atest MagnificationControllerTest,
atest WindowMagnificationManagerTest,
atest WindowMagnificationControllerTest,
atest FullScreenMagnificationControllerTest,
Change-Id: Ia73195a71f55bb26ef3f2eafce0dc73d8a65583b
After the service connection is ubineded or the binder died,
a11y framework should reset all the magnifiers that had been
controlled by the service connection.
Now the framework should also support reseting the
window magnfiers that had been controlled by the service.
Bug: 203532041
Test: atest AccessibilityMagnificationTest,
atest AccessibilityServiceConnectionTest,
atest MagnificationProcessorTest,
atest MagnificationControllerTest,
atest WindowMagnificationManagerTest,
Change-Id: I605a2cb17e3fb6677b9da0715ca7d7a84af70295
Does not yet include the prefetching API, which will be introduced later.
- Have a CacheController as the source of cache manipulation
- Notify system to update user relevant types
Bug: 192489177
Test: Build, added cts tests, atest CacheControllerTest
Change-Id: I150a01192dcf764ffcfe9848b499034a48aded08
There are 2 milestones in this feature.
1. Refactor the callbacks for Accessibility in WindowManagerInternal.
2. Implement this feature in such new architecture.
This CL is for the 1st milestone.
The goal of refactor:
When the window manager invokes callbacks which are related to
contextual changes affecting the screen magnification feature, it will
be handle by the MagnificationController first now. Then, it will
dispatch the events to the activated magnification controller such as
FullScreenMagnificationController or WindowMagnificationManager.
We use delegate design pattern to make MagnificationController as proxy.
It will assign a specific delegate when the activated controller is changed.
Then, it will dispatch the events from window manager to activated
magnification controller. Therefore, the activated magnification
controller will handle this event.
For now, we introduce WindowManagerEventDispatcher in
AccessibilityControllerInternalImpl to pass the callback events to
MagnificationController. However, AccessibilityControllerInternalImpl
would be refactored since it should have no nested class relation with
AccessibilityController.
Bug: 194668976
Test: atest FullScreenMagnificationControllerTest
atest WindowMagnificationManagerTest
atest MagnificationControllerTest
atest CtsAccessibilityServiceTestCases
Change-Id: Ic4d87c48a15b9179afeda48132635adc8e86107c
Make accessibility services able to enable/disable a specified
IME in the same package for the current user.
Bug: 195476910
Fix: 191386474
Test: atest AccessibilitySoftKeyboardTest. Also tested with
modified talkback (use the swith input method API for enable
ime), and self created empty a11y service with an empty ime.
Mananged device and profile which set limit on user 0 are also
tested.
Change-Id: I4187468076705ac597d680f2f5dc32d7b166da1f
See go/b200769372 - the legacy public methods
The legacy magnification controller methods,
setScale(), setCenter(),
getScale(), getCenterX(), getCenterY(),
getMagnificationRegion(),
reset(),
will keep the behavior before that control
only full-screen magnification.
To make the service able to control the actiavted magnifier
on the display, the service should use the new public APIs,
1. setMagnificationConfig(),
2. getMagnificationConfig(),
3. getCurrentMagnificationRegion()
instead of the legacy APIs.
This change
1. Updates the doc of the legcacy APIs.
2. The legacy APIs controls only fullscreen magnification
TODO: Add new public methods, getCurrentMagnificationRegion(),
resetCurrentMagnification() for the current controlling magnifier
Bug: 210069654
Test: atest MagnificationProcessorTest,
atest AbstractAccessibilityServiceConnectionTest,
atest AccessibilityMagnificationTest,
Change-Id: Ia157971d2c9716df7aa14024525ee0adfdc0d46f
The split-screen behavior is changed, so systemui doesn't
support this action with a single activity. We don't handle
this action by default from sc-v2. If SystemUI has any behavior
changed, they could register this action to support it.
Bug: 207056257
Test: presubmit pass
Change-Id: If569eff1ac52601e6b88324d7b47630a53e198f2
This window type is for distinguishing the windows used for
magnification. Accessibility apps could move the window magnifier
to follow accessibility focus, but they need to exclude the case that
the focus is on the window magnfier with this information
Bug: 188136191
Test: manual test , will add cts when the public api is ready
Change-Id: Ic2598d25ead5b1d74ebb2c0ea181bfe139675750
See go/b200769372
AccessibilityService adds new magnificationController APIs,
setMagnificationConfig and getMagnificationConfig, to control
magnification on demand.
The service can set the magnification mode, scale, center postion
of the magnifier on the display to control magnification.
And the service controls the least recently activated magnifier
on the display if the config mode is not specified.
Bug: 199732498
Test: atest MagnificationProcessorTest,
atest AbstractAccessibilityServiceConnectionTest,
atest AccessibilityMagnificationTest,
Change-Id: I51be2bbf96411c6c54c9da30873556d09533bb65
1. Add a range to the parameter in setScale(), setCenterX(), setCenterY()
2. The IntDef annotation name of MagnificationMode should not be
capitalized. The constants names should also start with the same prefix
set in the IntDef annotation.
3. Adds @hide for the IntDef annotation.
4. The related constants uses consistent prefix
Bug: 208422801
Test: none
Change-Id: Ibd997c3a13385d8884e208564092d829833ae637
1. Detect triple-tap and hold gesture.
2. Magnify the content at a user's tap location.
3. Offset the frame to the top-left position about a half of frame size
4. The definition and the implementation of
WindowMagnificationController's getCenterX and getCenterY are different
due to these patches. We need to correct them. (ref: b/207812394)
Bug: 202465806
Test: atest IWindowMagnificationConnectionTest
atest MagnificationControllerTest
atest WindowMagnificationAnimationControllerTest (@Ignore)
atest WindowMagnificationConnectionWrapperTest
atest WindowMagnificationControllerTest
atest WindowMagnificationGestureHandlerTest
atest WindowMagnificationManagerTest
Change-Id: I59bf8a3e35e42c963de1698bec915a4a590e5c5f
To make only one activated magnification mode
show on the display.
When fullscreen magnifier is turned to be activated on
the display. MagnificationController will disable the
window magnifier if it is enabled.
Bug: 206820500
Test: atest MagnificationControllerTest
Change-Id: I465e7293b4262d3cf8caf9873ea772aba50f1d6d
Context: Move reverted AuioDescription implementation CL ag/15617076 forward, (Step 2 out of 2)
Test: local test on ADT3, read status via the API in TvSettings.
Test: atest AccessibilityManagerTest
Bug: 160823970
Change-Id: If55a94299e21c47a247f8ffa3d0eb3aab2622611
In go/b200769372 , Magnification mode transition
When the controlling magnifier switches the magnification mode,
MagnificationController should transitions to the config of the
targeting mode.
Bug: 204897462
Test: atest MagnificationProcessorTest,
atest MagnificationControllerTest
Change-Id: Ife7369e39b9b256ee600e471b66e9869cd04024f
When talkback is detecting gestures, we save hover events in case we want to start touch exploring. Thses need to be cleared every new interaction.
Bug: 159651900
Test: atest touchInteractionControllerTest
Change-Id: Ia340d96205064bc953df2fbe80f87bc94885af15
- Maintain a static collection of caches (sCaches). Lock is
sConnectionCache
- Initialize a cache when a connection is added. (client.addConnection)
- Remove the cache when a connection is removed. (client.removeConnection)
- System server should not have its cache initialized, can remove
parameter in AccessibiltyInteractionClient constructor
I tried to maintain a local reference (mAccessibilityCache) to the
service cache for each client to avoid accessing the static
collection for each request. But since the process is not immediately
killed when the service is turned off, ensuring resources are cleared
correctly is big time investment that can be done later if desired.
Bug: 192489177
Test: Build and use talkback, do the slew of tests.
atest CtsAccessibilityServiceTestCases CtsAccessibilityTestCases
CtsUiAutomationTestCases
FrameworksServicesTests:com.android.server.accessibility
FrameworksCoreTests:com.android.internal.accessibility
FrameworksCoreTests:android.view.accessibiliity
Change-Id: Ic30f6915c7b3c186a512ed8b410bf6e842eff5fb
MagnificationConfig stores the magnification configuration of the
controlling magnifier, such as magnification mode, scale and center
postion.
And MagnificationProcessor uses the config to controll the magnifier for
AccessibilityService.
See
API review doc: go/b200769372
Design doc: go/a11yservice_control_magnification_in_t
The chapter at Proposal A and MagnificationConfig
CTS: ag/15824302
Bug: 199732498
Test: atest AbstractAccessibilityServiceConnectionTest,
atest MagnificationProcessorTest,
atest WindowMagnificationManagerTest,
atest MagnificationConfigTest
Change-Id: I20323865d2efe1b40626f0c86767848733856482
This ensures that new display's existence is sent to accessibility
services. Otherwise, there is a race where service gets a list of
windows without a new display until any window event is dispatched.
Bug: 198560965
Test: AccessibilityOverlayTest
Change-Id: I0097bc53de15ddb04df3b3ff7641b329964872de
After ag/5612023, the click point calculation should depend on
the global screen coordinate instead of the coordinate before
magnification.
Bug: 193200262
Test: manual test on launcher
Change-Id: If51885f43e78c1554fdcd0642fb301a466d3bdba
To extend the magnification API to window magnification mode,
we support enabling the window magnification without animation.
Besides, we also disable the window magnification if the given scale
is less than or equal to 1.0f.
For the screen off scenario, we change to disable the window
magnification without animation.
Bug: 204844820
Test: atest com.android.systemui.accessibility
atest com.android.server.accessibility.magnification
Change-Id: I8a35492776f08ecee353f035a94a16fec8e61218
This reverts commit 4ec7b5c3fb.
Bug: 196389741
Test: Ensure PIP windows are trusted overlays
Change-Id: I49477ffd489ecbc969ab9b9310c2344ab03280e8
(cherry picked from commit 0cf0d420e1)
Currently we only have a globale magnification mode which
is stored in settings. It ends up the mode is shared between
multi-displays. To provide better experience, we store the mode
for each none-default display until the device reboot,
and the default mode is full-screen.
Test: atest com.android.server.accessibility
atest com.android.systemui.accessibility
Bug: 194667380
Change-Id: I099e1ee945938aac07cc079f68bccbc745d40b08
