We add a mIsImeWindowVisibleArray in WindowMagnificationManager to
indicate whether there is a ime in the displayId Display or not.
The element in mIsImeWindowVisibleArray would keep the ime's visibility
attribute which can be changed by onImeWindowVisibilityChanged callback.
Then, we add this element which should be true to the decision of
triggering tracking typing focus functionality.
Bug: 215813890
Test: Manually, I use log print to trace whether the mIsImeWindowVisibleArray
is set during AccessibilityController#setMagnificationCallbacks.
atest WindowMagnificationManagerTest
atest FullScreenMagnificationControllerTest
atest MagnificationControllerTest
Change-Id: I14f172255e79d04428db86769ed49d679f2790f5
After a user perform a11y action, namely moving up/down/left/right, we
have to disable tracking focus functionality.
We refactor #onDrag to a more general term #onMove to cover any move
action which has to disable tracking focus functionality.
Bug: 218935435
Test: atest WindowMagnificationTest
atest WindowMagnificationControllerTest
atest WindowMagnificationManagerTest
Change-Id: I578b301645a08c71e874fe2f4fcf2557202abefd
There are 3 conditions that we have to reset the tracking typing focus
state to on.
1. IME is shown again. (It already exists.)
2. A user re-enable a window magnifier.
3. A user switch to a window magnifier from a full-screen magnifier.
In this CL, we add the 2nd and 3rd condition to turn on this
functionality in a window magnifier.
Bug: 216394179
Test: atest WindowMagnificationManagerTest
Change-Id: I78ca62b00559c242d5562f4247e3854e8386a512
For following typing focus feature, we add a
moveWindowMagnifierToPosition method for a window magnifier to move it
to the center of onRectangleOnScreenRequested.
Apart from calling this method to fulfill movement functionality, we can
separate the movement from calling enableWindowMagnification which is a
temporary replacement for movement functionality. Therefore, we can
distinguish whether the it is enabled by the first time or the further
movement behavior. It would help us differentiate the condition whether
we should reset the following typing focus to on/off state. (b/216394179)
Bug: 219654280
Test: atest IWindowMagnificationConnectionTest
atest WindowMagnificationTest
atest WindowMagnificationAnimationControllerTest
atest WindowMagnificationControllerTest
atest WindowMagnificationConnectionWrapperTest
atest WindowMagnificationManagerTest
Change-Id: Iaac3ca6868d948571c1aee54a5f122c9697059d8
It fixes the low quality CTS test, AccessibilityMagnificationTest.
And this fix also makes enableWindowMagnification() a robust API
that can be called frequently when the connection is toggled.
CL comments and discussion are at ag/16785609
Root cause:
1. when the test cases run iteratively, the test a11y service would be
enabled and disabled repeatedly so the windowMagnification
connection would also be connected and disconnected repeatedly.
And when the test a11y service is enabled in a short time, the
connection may not be ready to be established.
Solution:
1. So we add a wait lock and notify to unclock after the connection
is established. And prevent the duplicated connection request.
Bug: 2147470987
Test: atest AccessibilityMagnificationTest --iterations,
atest com.android.server.accessibility
Change-Id: I6ec22e66796aaf565c10b0fedcd4de359ce5fc7b
1. Implements AccessibilityWindowsPopulator class including the
new accessibilityWindow class generated by the inputWindowHandle
from the surface flinger.
2. Replaces the windowState by the accessibilityWindow used in the
AccessibiltyController class for computing the changed windows
reported to the A11y framework.
3. The embedded hiarerchy windows would be reported to A11y framework
due to they are reported from the surface fligner. This will make
the traversal function is broken when the Auotfill features is using
talkback. We find out the embedded hiarerchy windows by its window
token and don't add them into the A11y windows list.
4. Removes the modal windows calculation of the unAccountedSpace
including the task fragment due to we intersec the region between
the frame and touch region for all windows.
5. When reported windows counts from the surface flinger changed,
calling the accessibilty controller class to compute the changed
windows and send the results to the A11y framework immediately.
This can avoid the CTS failure with no corresponding window found
when the activity is launched due to the timing issue between the
callbacks for computing changed windows from the WM and the
callback for changed windows from the surface flinger.
Bug: 191736824
Test: a11y CTS & unit tests
Test: Manual testing including the A11y services
Change-Id: Ie9150ddc8e53ca88ad8cd7ba9376837e31436518
Uses
MagnificationInfoChangedCallback.onFullScreenMagnificationChanged
callback method to notify magnification hange instead of
accessing the public method of AccessibilityManagerService directly.
It reduces the interaction between AccessibilityManagerService and
FullScreenMagnificationControlller.
So that MagnificationController would take responsible for handling
notifyMagnificationChange().
Bug: 218540156
Test: atest FullScreenMagnificationControllerTest,
atest FullScreenMagnificationGestureHandlerTest,
atest MagnificationControllerTest,
Change-Id: Ib9f31681e442f1700a76250a5206199bad458c84
(cherry picked from commit 3c91f067b1)
Merged-In: Ib9f31681e442f1700a76250a5206199bad458c84
We used to use MagnificationScaleProvider to retrive the
scale when transitioning the mode, which is incorrect
because the provider only persists the scale changed by users.
If the scale is changed by public api, then the scale will
become inconsistent.
To fix it we choose the scale based on the target mode.
We also fix that mode transition without animation should be
considered as a transition case.
We also add more log for flaky CTS debugging.
Bug: 218792845
Test: atest MagnificationProcessorTest
atest com.android.server.accessibility.magnification
Change-Id: I47f49345ce46f726d6e06e203110ef793f35c76e
This way, we can pass back a11y sessions from direct startInput()
return value. And a11yManagerService will only request sessions
from a11y services which the client doesn't have sessions yet
(will not request existing a11y sessions again).
Bug: 187453053
Test: tested manually with talkback. Tested client switching within
"recent apps".
Change-Id: I8efdc9886ce33185a2195b741668c12e319ea660
An accessibility service needs to set FLAG_INPUT_METHOD_EDITOR
accessibilityFlags to use ime apis. Capability is not added
per Phil's suggestion.
Bug: 187453053
Test: CTS test added.
Change-Id: I78703c48e343ba09c2a0bf61c3447eea7337cb49
InputMethodService is the primary and a11y is the secondary.
InputMethodService is not
affected by a11y status. When the session from input method is
established, app can start input (pass input context to input method).
When an a11y session comes back, it will be passed to the app. When
InputMethodManagerService binds to/start input with
InputMethodService, it does the same to a11y services which
requested IME functionalities.
It is possible that input method can edit text before ally sessios are
established. So the EditorInfo passed to a11y could be stale.
So when an a11y session is passed to client, client will send
a notification (input method doesn't have this extra notification) for
the current selection. I think since the time for a11y session establish
shouldn't be long, and we get the current state later, it should be fine
for a11y services.
When input method is disconnected from app (client) (even for input
method switching), we cleared a11y and sessions too. When
input method request sessions, we must rerequest sessions for a11y. This
is mainly because when we unbindCurrentClientLocked(SWITCH_IME), we set
active to false for the current client. Suppose we don't want to
change the current structure of input method, an inactive client
probably should clear accessibility sessions too.
When we switch to a client which already has a session with input
method, there might be some a11y sessions with this client, and
some a11y services might be disabled or enabled while the client was
switched out. We pass unchanged a11y
sessions to client, and request sessions for newly enabled a11y services.
When an a11y service is disabled, it removes its session
from all clients in InputMethodManagerService.
Test: type word through modified "switchToInputMethod". Tested session
notification through logs. Tested client switching, input method
switching, a11y service enabled/disable, multiple a11y services,
a11y service enabled before device reboot.
Also tested work profile.
Bug: 187453053
Change-Id: Ia651a811093a939d00c081be1961e24ed3ad0356
Changes the name of the APIs to reflect the methods actually do.
Bug: 216474106
Test: atest CaptioningManagerTest
Change-Id: I152fa9b791be0f35bdc78478e0eb0d524c120e51
To make the magnification debugging easier, we add the dump information
for magnification tracking typing focus feature.
For full screen magnification, it includes
MagnificationFollowTypingEnabled.
For window magnification, it includes both
MagnificationFollowTypingEnabled and TrackingTypingFocusEnabled.
MagnificationFollowTypingEnabled is controlled by system preference
setting.
TrackingTypingFocusEnabled is controlled by a user behavior.
Bug: 216208936
Test: manual - Turn on the magnifier
$adb shell settings put secure
accessibility_magnification_follow_typing_enabled 0/1
$adb shell dumpsys accessibility
Change-Id: Id584ece3b2e3d9b849ff1275b55f4fd0ba09a6b8
To make the magnification debugging easier, we add the dump information for magnification.
AccessibilityManagerService dumps magnification information
of the controlling magnifiers on the displays
When
1. $adb shell dumpsys accessibility
2. CTS a11y test cause failed,
Bug: 214835232
Test: manual - Turn on the magnifier and $adb shell dumpsys accessibility
Change-Id: I39c393e6c41928f303c2aee8c0d7f0f6478162f9
1. Proposes the public APIs to get and the system APIs to set the
status of the captions and UI show/hide, and notify the status change.
2. Creates a new class, CaptioningManagerImpl, to get/set the value
of the secure setting keys through the A11yManagerService.
3. Adds a new permission, to protect the setter methods which can't
be used by 3rd party apps.
Bug: 136282740
Test: a11y CTS & unit tests
Change-Id: I591b98578dbb94ca8b5be8a553b81d23a143ee16
Ignore-AOSP-First: new permissions (not yet in AOSP)
* changes:
Revert "Uses SF callback windows for the A11yWindowInfo [1/n]"
Revert "Uses SF callback windows for the A11yWindowInfo [2/n]"
Revert "Use shellRoot to identify PIP menu window"
Revert "Apply display transform to window bounds and node bounds"
Revert submission 16249364-Surface Flinger
Reason for revert: Large increase (~200%) in system_server memory usage (b/214255324)
Reverted Changes:
I14c5e9889:Apply display transform to window bounds and node ...
I0ac0d0487:Use shellRoot to identify PIP menu window
I83e898d91:Uses SF callback windows for the A11yWindowInfo [2...
I94d7ed4fb:Uses SF callback windows for the A11yWindowInfo [1...
Change-Id: Icb3b07750ebaa0c26d38165dd1a01b2af4b8b5a6
* changes:
Apply display transform to window bounds and node bounds
Use shellRoot to identify PIP menu window
Uses SF callback windows for the A11yWindowInfo [2/n]
Uses SF callback windows for the A11yWindowInfo [1/n]
1. Implements AccessibilityWindowsPopulator class including the
new accessibilityWindow class generated by the inputWindowHandle
from the surface flinger.
2. Replaces the windowState by the accessibilityWindow used in the
AccessibiltyController class for computing the changed windows
reported to the A11y framework.
3. The embedded hiarerchy windows would be reported to A11y framework
due to they are reported from the surface fligner. This will make
the traversal function is broken when the Auotfill features is using
talkback. We find out the embedded hiarerchy windows by its window
token and don't add them into the A11y windows list.
4. Removes the modal windows calculation of the unAccountedSpace
including the task fragment due to we intersec the region between
the frame and touch region for all windows.
5. When reported windows counts from the surface flinger changed,
calling the accessibilty controller class to compute the changed
windows and send the results to the A11y framework immediately.
This can avoid the CTS failure with no corresponding window found
when the activity is launched due to the timing issue between the
callbacks for computing changed windows from the WM and the
callback for changed windows from the surface flinger.
Bug: 191736824
Test: a11y CTS & unit tests
Test: Manual testing including the A11y services
Change-Id: I94d7ed4fbc5abab2252ddccae55678014ac9e600
There are 3 milestones in this feature.
1. Refactor the callbacks for Accessibility in WindowManagerInternal.
2. Implement this feature in such new architecture.
3. Implement the setting choice in preference page.
This CL is for the 3rd milestone.
We add a ACCESSIBILITY_MAGNIFICATION_FOLLOW_TYPING_ENABLED key to store
a user preference for following typing focus. The default value of
preference is true.
If the value is false, we wouldn't do anything in
onRectangleOnScreenRequested and just return.
If the value is true, we would dispatch the event to an activated
magnification controller.
Bug: 194668976
Test: atest AccessibilityManagerServiceTest
atest AccessibilityUserStateTest
atest FullScreenMagnificationControllerTest
atest MagnificationControllerTest
atest WindowMagnificationManagerTest
atest SettingsBackupTest
Change-Id: I7203698d2edcc474ab1638ac81e72e054a61d6fd
There are 3 milestones in this feature.
1. Refactor the callbacks for Accessibility in WindowManagerInternal.
2. Implement this feature in such new architecture.
3. Implement the setting choice in preference page.
This CL is for the 2nd milestone.
We move the window magnification to the typing focus' center position
based on the condition of whether a user takes control or not. We only
make a movement when the control is not taken by a user or we don't
preform the movement for the window magnification.
There are 2 methods for a user to take the control.
1. A user use 1 finger to drag the window magnification.
2. A user use 2 finger to drag the window magnification.
There is 1 method for a user to release the control.
1. When IME is shown, the control would be released.
So, we can decide whether we should make a movement to typing focus
given the condition of who take the control.
Bug: 194668976
Test: atest MagnificationControllerTest
atest WindowMagnificationTest
atest WindowMagnificationControllerTest
atest WindowMagnificationManagerTest
Change-Id: I145f893d412b74c20afe1685449370d1dba99961
The flaky is caused by the deadlock.
WindowMagnificationManager has its own lock and the
WindowMagnificationConnectionWrapper which uses
AccessibilityTraceManager. When the public method of
WindowMagnificationManager is invoked, it hold the lock of
WindowMagnificationManager and the lock of
AccessibilityManagerService. However, when the service connection
is unbound, it will hold the lock of AccessibilityManagerService
first then the lock of WindowMagnificationManager.
To simplify the archiecuture, we inject the lock of
AccessibilityManagerService to WindowMagnificationManager.
Bug: 213120663
Test: atest com.android.server.accessibility
atest CtsAccessibilityServiceTestCases
Change-Id: I818c57a5140269a5800eb1a52e25aa2ce0919245
See go/b200769372 - the legacy public methods
The legacy magnification controller methods of a11y service,
getMagnificationRegion(),
reset(),
will keep the behavior before that control
only full-screen magnification.
To make the a11y service able to control the actiavted
magnifier on the display, the service should use the
new public APIs,
1. getCurrentMagnificationRegion(),
2. resetCurrentMagnification(),
instead of the legacy APIs.
Bug: 210069654
Test: atest AccessibilityMagnificationTest,
atest MagnificationProcessorTest,
atest AbstractAccessibilityServiceConnectionTest,
Change-Id: I76bfee43cbb76a9aeec33574a8db18430903e08c
See Design doc: go/b200769372
To make the behavior consistent on the new platform,
The legcay callback,
onMagnificationChanged(MagnificationController controller,
Region region, float scale, float centerX, float centerY)
keep notifying only full-screen magnification change as
before.
To support listening to the magnification changes of all
magnification modes, the service should overide
The new callback proposed in T,
onMagnificationChanged(MagnificationController controller,
Region region, MagnificationConfig config).
TODO: Notify magnifcation change when window magnifier turns off
Bug: 203013925
Test: atest AccessibilityMagnificationTest,
atest MagnificationControllerTest,
atest WindowMagnificationManagerTest,
atest WindowMagnificationControllerTest,
atest FullScreenMagnificationControllerTest,
Change-Id: Ia73195a71f55bb26ef3f2eafce0dc73d8a65583b
After the service connection is ubineded or the binder died,
a11y framework should reset all the magnifiers that had been
controlled by the service connection.
Now the framework should also support reseting the
window magnfiers that had been controlled by the service.
Bug: 203532041
Test: atest AccessibilityMagnificationTest,
atest AccessibilityServiceConnectionTest,
atest MagnificationProcessorTest,
atest MagnificationControllerTest,
atest WindowMagnificationManagerTest,
Change-Id: I605a2cb17e3fb6677b9da0715ca7d7a84af70295
Does not yet include the prefetching API, which will be introduced later.
- Have a CacheController as the source of cache manipulation
- Notify system to update user relevant types
Bug: 192489177
Test: Build, added cts tests, atest CacheControllerTest
Change-Id: I150a01192dcf764ffcfe9848b499034a48aded08
There are 2 milestones in this feature.
1. Refactor the callbacks for Accessibility in WindowManagerInternal.
2. Implement this feature in such new architecture.
This CL is for the 1st milestone.
The goal of refactor:
When the window manager invokes callbacks which are related to
contextual changes affecting the screen magnification feature, it will
be handle by the MagnificationController first now. Then, it will
dispatch the events to the activated magnification controller such as
FullScreenMagnificationController or WindowMagnificationManager.
We use delegate design pattern to make MagnificationController as proxy.
It will assign a specific delegate when the activated controller is changed.
Then, it will dispatch the events from window manager to activated
magnification controller. Therefore, the activated magnification
controller will handle this event.
For now, we introduce WindowManagerEventDispatcher in
AccessibilityControllerInternalImpl to pass the callback events to
MagnificationController. However, AccessibilityControllerInternalImpl
would be refactored since it should have no nested class relation with
AccessibilityController.
Bug: 194668976
Test: atest FullScreenMagnificationControllerTest
atest WindowMagnificationManagerTest
atest MagnificationControllerTest
atest CtsAccessibilityServiceTestCases
Change-Id: Ic4d87c48a15b9179afeda48132635adc8e86107c
Make accessibility services able to enable/disable a specified
IME in the same package for the current user.
Bug: 195476910
Fix: 191386474
Test: atest AccessibilitySoftKeyboardTest. Also tested with
modified talkback (use the swith input method API for enable
ime), and self created empty a11y service with an empty ime.
Mananged device and profile which set limit on user 0 are also
tested.
Change-Id: I4187468076705ac597d680f2f5dc32d7b166da1f
See go/b200769372 - the legacy public methods
The legacy magnification controller methods,
setScale(), setCenter(),
getScale(), getCenterX(), getCenterY(),
getMagnificationRegion(),
reset(),
will keep the behavior before that control
only full-screen magnification.
To make the service able to control the actiavted magnifier
on the display, the service should use the new public APIs,
1. setMagnificationConfig(),
2. getMagnificationConfig(),
3. getCurrentMagnificationRegion()
instead of the legacy APIs.
This change
1. Updates the doc of the legcacy APIs.
2. The legacy APIs controls only fullscreen magnification
TODO: Add new public methods, getCurrentMagnificationRegion(),
resetCurrentMagnification() for the current controlling magnifier
Bug: 210069654
Test: atest MagnificationProcessorTest,
atest AbstractAccessibilityServiceConnectionTest,
atest AccessibilityMagnificationTest,
Change-Id: Ia157971d2c9716df7aa14024525ee0adfdc0d46f
The split-screen behavior is changed, so systemui doesn't
support this action with a single activity. We don't handle
this action by default from sc-v2. If SystemUI has any behavior
changed, they could register this action to support it.
Bug: 207056257
Test: presubmit pass
Change-Id: If569eff1ac52601e6b88324d7b47630a53e198f2
This window type is for distinguishing the windows used for
magnification. Accessibility apps could move the window magnifier
to follow accessibility focus, but they need to exclude the case that
the focus is on the window magnfier with this information
Bug: 188136191
Test: manual test , will add cts when the public api is ready
Change-Id: Ic2598d25ead5b1d74ebb2c0ea181bfe139675750
See go/b200769372
AccessibilityService adds new magnificationController APIs,
setMagnificationConfig and getMagnificationConfig, to control
magnification on demand.
The service can set the magnification mode, scale, center postion
of the magnifier on the display to control magnification.
And the service controls the least recently activated magnifier
on the display if the config mode is not specified.
Bug: 199732498
Test: atest MagnificationProcessorTest,
atest AbstractAccessibilityServiceConnectionTest,
atest AccessibilityMagnificationTest,
Change-Id: I51be2bbf96411c6c54c9da30873556d09533bb65
1. Add a range to the parameter in setScale(), setCenterX(), setCenterY()
2. The IntDef annotation name of MagnificationMode should not be
capitalized. The constants names should also start with the same prefix
set in the IntDef annotation.
3. Adds @hide for the IntDef annotation.
4. The related constants uses consistent prefix
Bug: 208422801
Test: none
Change-Id: Ibd997c3a13385d8884e208564092d829833ae637
