This fixes an issue where the device was crashing and rebooting when
triple tapping to activate magnification at the same time as covering
the proxmity sensor while on a voice call.
Bug: b/236875528
Test: manual:
1. Perform a voice call
2. While on voice call, execute the "Triple tap on screen" gesture,
holding down your finger on the last tap and then cover the proximity
sensor.
3. The device should not crash
Change-Id: I9415c5b378b32342c016b17dfcef12cde45ce237
Since the WindowInfo is null if the FLAG_RETRIEVE_INTERACTIVE_WINDOWS is
unset, we would fail to calculate the node bound correctly.
The reason behind this failure is that the accessibility framework only
tracks windows when the FLAG_RETRIEVE_INTERACTIVE_WINDOWS is set.
As being a short term solution, we could retrieve the transformation matrix
and magnification spec from WindowState, making us have this CL.
However, as a long term solution, we should keep tracking WindowInfo in
DisplayWindowsObserver whether the flag is set or unset and mitigate the
additional system overhead.
Bug: 229379635
Test: atest android.accessibilityservice.cts
atest com.android.server.accessibility
We use a test apk to test the node bound value when the flag is
unset equals to the value from WindowInfo when the flag is set.
Change-Id: I02e7e74d027dc65605f5e8fcc10d3c808e316476
windows
If accessibility framework doesn't track windows, the active window
is updated when receving window_state_changed event. ViewRootImpl
sends this event when performing traversal first time, however the
focused window token is not updated yet from InputManagerService.
To fix the regression issue, we fallback to the legacy approach for
short term solution.
Test: atest android.accessibilityservice.cts
Bug: 228442331
Change-Id: I032a73b9f4cf408a5dd6cae8a87bd26131ebdbfa
ViewRootImpl sends window_state_changed event to notify
AccessibilityService that itself is visible on the screen. However,
the corresponding window is not available from accessibility framework
perspective because the window information is from SurfaceFlinger process
now, so the window is added only when it is really visible on the screen.
To ensure accessibilityService could get the node or the window when
receiving this event, we postpone this event until the corresponding
window is added. We also set a timeout to send those pending events.
Bug: 228442331
Bug: 226371995
Test: atest CtsAccessibilityServiceTestCases
manual test: write a sample AccessibilityService to test the apis
when receving window_state_changed events
Change-Id: I92cf2ddb9de90c050cf145f746a53f31d3a5df83
This is a follow up CL to our previuos CL [1], which enabled
AccessibilityService to use a subset of InputConnection APIs.
In that CL we have reused existing AIDL interfaces that were designed
and maintained for IMEs for simplicity, where a non trivial amount of
unnecessary IPC endpoints were included.
From the security and maintainability viewpoints, however, exposing
unnecessary IPC endpoints is discouraged in general. To address such
concerns this CL introduces a set of dedicated IPC definitions for
A11yIME so that we do not need to reuse IPCs for IMEs.
This CL also stops passing InputBinding object to A11yIME process as
it contains IInputContext Binder Proxy, which can still be used to
directly invoke fallback InputConnection. This is doable now because
A11yIME no longer relies on fallback InputConnection [2].
This CL is should not have any observable changes in the semantics.
End-to-end CTS tests guarantee that everything is still working as
intended now and in the future.
[1]: Ia651a811093a939d00c081be1961e24ed3ad0356
fb17e5ae7a
[2]: I2af3cd50444d8ddf25aa0f6479238156914e6fff
dc635efb68
Fix: 215633021
Fix: 215636776
Test: atest CtsInputMethodTestCases:AccessibilityInputMethodTest
Test: atest CtsAccessibilityServiceTestCases:AccessibilityInputConnectionTest
Test: atest CtsAccessibilityServiceTestCases:AccessibilityImeTest
Change-Id: I5ff2e804cbcf90828370a0612ff54111130bdff4
The attachInfo from ViewRootImpl is not reliable
for DisplayArea manipulation or windowless window.
To fix this problem, we use the transform matrix of
InputWindowHandle, which could transform the bounds
from window coorindate to screen coordinate. We also
transfrom the bounds to logical display coordinates
with the associated display matrix.
Besides, we also record the magnification spec of the window,
which could get the bounds before magnification. We use
this value to decide the property 'visibleToUser'.
Bug: 200797785
Test: atest android.accessibilityservice.cts WindowInfoTest
atest com.android.server.accessibility
Change-Id: I0917b04fe8b027fb2bd932a6f0604ba1449ebc66
When the magnification settings mode or config mode transitions,
MagnificationController would
1. Disables the current magnification mode. Then
2. Starts to enable the target magnification mode.
To prevent the magnification change callback send twice during the
mode transition, MagnificationController should only notify the
change of the transitioning target mode and ignore the notifcation
change for the disabling magnification mode.
And when WindowMagnification resets or disables by
WindowMagnificationController#deleteWindowMagnification(),
it should notify source bounds empty when magnification is deleted.
And when FullscreenMagnification resets and the animator is animating,
FullScreenMagnificationController should notify magnification
change.
Bug: 209701404
Bug: 210636326
Test: atest AccessibilityMagnificationTest,
atest WindowMagnificationControllerTest,
atest com.android.server.accessibility.magnification
Change-Id: Ief718614d80177a4a076eddc708a7ecfc4dfb73f
We need to be able to initiate touch exploration even if there are no pointers on the screen in order to reliably cover the case where a user just taps the screen.
The user expects to hear if they have tapped on anything.
Bug: 223673301
Test: manual. Tap on the screen and confirm that talkback reads the item in that position.
Change-Id: Id96fa8bd7f000ea153f45024e06915a43ca80826
non-A11yTool notification is going to be integranted
in PermissionController. We will stop secheduling intent
and sending notification if the safety center is enabled, and
reschedule the intents for none-Accessibility servicewhen the
flag is flipped to true.
Bug: 221369374
Test: atest AccessibilitySecurityPolicyTest
atest PolicyWarningUIControllerTest
manual test
Change-Id: I5ab46da053e5ded5422ac932ca3de969c89b8c21
In previous, we only use a variable to record the state of magnification
mode. Therefore, it would be confused when there are multiple displays.
Now, we use sparse array to keep the state of current magnification mode
and last magnification activated modes with corresponding displayId,
making us have ability to distinguish different modes in different
displayId.
Bug: 182227245
Test: atest MagnificationControllerTest
atest MagnificationProcessorTest
atest FullScreenMagnificationGestureHandlerTest
atest MagnificationConfigTest
Change-Id: I239faab22bb6abe3e6cadde9b09eba496c4688c0
When some applications create an external service and register this service
to be an A11y service, then this service can kill the system and restart
it continuously.
This is a security vulnerability and we add not allow an external service
to register as an A11y service to fix it.
Bug: 216115176
Test: a11y CTS & unit tests
Change-Id: If75b9c300df73e98be92d6c14d69301cc377b205
Sending without copying causes some motion events to get duplicated in the service's motion event stream.
Fix: 223674686
Bug: 223673301
Test: Manual. Perform three-finger swipes and insure they work as expected.
Change-Id: Ie94ef06605902f2b3be5fb426511d5c80cf1ac09
Only lock when obtaining the input filter to avoid deadlock when bringing up talkback settings.
Fix:223673816
Test: Manual. Bring up talkback settings and insure that there is no ten second wait.
Change-Id: Ic26a3a08a7c67b096a7995332e3ed5096fde4847
When the platform does not support window magnification feature,
AccessibilityManagerService should fallback magnification mode
settings if the magnification mode of userState is window mode
that is invalid.
Bug: 213414385
Test: manual test
Change-Id: I810131f7b1925b6b010c21d822b521d29ee3a646
In the previous version, GesturesObserver#clear would be called by
MagnificationGesturesObserver, making the matchers' state be clear by
outside class. It would make the matchers' state not consistency in
GesturesObserver#onMotionEvent. Especially, when the matcher is
completed and becomes STATE_GESTURE_COMPLETED, it would be clear as
STATE_CLEAR by MagnificationGesturesObserver#clear which would trigger
GesturesObserver#clear. It would make the GesturesObserver keep going to
check other matchers since matcher.getState() !=
GestureMatcher.STATE_GESTURE_COMPLETED and wouldn't early return.
Therefore, we make GesturesObserver#clear as a private method and make
it to manager its matchers' state.
Bug: 206061247
Test: atest GesturesObserverTest
atest MagnificationGesturesObserverTest
atest MagnificationGestureHandlerTest
atest WindowMagnificationGestureHandlerTest
Change-Id: I19294e231f969d7dd54b50dc9a2deb36d06b532a
We print the readable string instead of integer value
to improve debugging efficiency.
Bug: 214785717
Test: build pass
Change-Id: I0f8ac7cd545674455019433d76db374cd6696387
When IMF calls into A11yManagerService, A11yMannagerService
will post the call to main handler so the function call
returns without holding the a11y mLock.
Fix: 218182733
Test: atest AccessibilityImeTest
Change-Id: I3f7fcb1906b4084b2458ce2b65bb6fec8e9c6ae1
In previous, requestRectangleOnScreen would only apply the shortest
movement on the magnifier to the rectangle. However, it will make typing
words out of the screen. Therefore, we add an additional distance for
moving the magnifier to help a user know the surrounding context.
Bug: 148882866
Test: atest FullScreenMagnificationControllerTest
Change-Id: Iee7a4f13395ed8acf3ffa7df759542fcb08190f8
When the flag is set dynmatically, we need to create the InputMethod()
instance and request to bind ime. If the flag is unset, we clear the
InputMethod() instance and unbind ime.
Fix: 218193835
Test: Tested with talkback. When the flag is not set, we can't use
ime. When the flag is set, we can use ime again (dynamically set
flags).
Change-Id: Ied48d452108a593e8e4ee5c67c3208acbc87335e
Change-Id: I2a85c64b089983807cb7627de9b8512e8f716426
(cherry picked from commit 14e78228a1)
We were queueing up ACTION_HOVER_EXIT when initiating touch exploration which doesn't make sense. ACTION_HOVER_EXIT only makes sense on ACTION_UP when touch exploring.
Fix: 219814318
Fix: 220387502
Test: Verify that entering a pin works as expected.
Test: Verify that touch exploration does not get stuck often, only occasionally per b/215654623
Change-Id: Iab29f7736ab4ff3ea24c26109073481a876ffb58
This reverts commit f1f2c84756.
The stragery is chaged, we don't need to check the installer
nor if it is sytem app. We also delete the tests that are
obsolete.
Bug: 221318082
Test: AccessibilitySecurityPolicyTest, PolicyWarningUIControllerTest
Change-Id: Ide89213dbf90cdb57001e5b05a2eeefc35f6adee
We add a mIsImeWindowVisibleArray in WindowMagnificationManager to
indicate whether there is a ime in the displayId Display or not.
The element in mIsImeWindowVisibleArray would keep the ime's visibility
attribute which can be changed by onImeWindowVisibilityChanged callback.
Then, we add this element which should be true to the decision of
triggering tracking typing focus functionality.
Bug: 215813890
Test: Manually, I use log print to trace whether the mIsImeWindowVisibleArray
is set during AccessibilityController#setMagnificationCallbacks.
atest WindowMagnificationManagerTest
atest FullScreenMagnificationControllerTest
atest MagnificationControllerTest
Change-Id: I14f172255e79d04428db86769ed49d679f2790f5
After a user perform a11y action, namely moving up/down/left/right, we
have to disable tracking focus functionality.
We refactor #onDrag to a more general term #onMove to cover any move
action which has to disable tracking focus functionality.
Bug: 218935435
Test: atest WindowMagnificationTest
atest WindowMagnificationControllerTest
atest WindowMagnificationManagerTest
Change-Id: I578b301645a08c71e874fe2f4fcf2557202abefd
There are 3 conditions that we have to reset the tracking typing focus
state to on.
1. IME is shown again. (It already exists.)
2. A user re-enable a window magnifier.
3. A user switch to a window magnifier from a full-screen magnifier.
In this CL, we add the 2nd and 3rd condition to turn on this
functionality in a window magnifier.
Bug: 216394179
Test: atest WindowMagnificationManagerTest
Change-Id: I78ca62b00559c242d5562f4247e3854e8386a512
For following typing focus feature, we add a
moveWindowMagnifierToPosition method for a window magnifier to move it
to the center of onRectangleOnScreenRequested.
Apart from calling this method to fulfill movement functionality, we can
separate the movement from calling enableWindowMagnification which is a
temporary replacement for movement functionality. Therefore, we can
distinguish whether the it is enabled by the first time or the further
movement behavior. It would help us differentiate the condition whether
we should reset the following typing focus to on/off state. (b/216394179)
Bug: 219654280
Test: atest IWindowMagnificationConnectionTest
atest WindowMagnificationTest
atest WindowMagnificationAnimationControllerTest
atest WindowMagnificationControllerTest
atest WindowMagnificationConnectionWrapperTest
atest WindowMagnificationManagerTest
Change-Id: Iaac3ca6868d948571c1aee54a5f122c9697059d8
It fixes the low quality CTS test, AccessibilityMagnificationTest.
And this fix also makes enableWindowMagnification() a robust API
that can be called frequently when the connection is toggled.
CL comments and discussion are at ag/16785609
Root cause:
1. when the test cases run iteratively, the test a11y service would be
enabled and disabled repeatedly so the windowMagnification
connection would also be connected and disconnected repeatedly.
And when the test a11y service is enabled in a short time, the
connection may not be ready to be established.
Solution:
1. So we add a wait lock and notify to unclock after the connection
is established. And prevent the duplicated connection request.
Bug: 2147470987
Test: atest AccessibilityMagnificationTest --iterations,
atest com.android.server.accessibility
Change-Id: I6ec22e66796aaf565c10b0fedcd4de359ce5fc7b
1. Implements AccessibilityWindowsPopulator class including the
new accessibilityWindow class generated by the inputWindowHandle
from the surface flinger.
2. Replaces the windowState by the accessibilityWindow used in the
AccessibiltyController class for computing the changed windows
reported to the A11y framework.
3. The embedded hiarerchy windows would be reported to A11y framework
due to they are reported from the surface fligner. This will make
the traversal function is broken when the Auotfill features is using
talkback. We find out the embedded hiarerchy windows by its window
token and don't add them into the A11y windows list.
4. Removes the modal windows calculation of the unAccountedSpace
including the task fragment due to we intersec the region between
the frame and touch region for all windows.
5. When reported windows counts from the surface flinger changed,
calling the accessibilty controller class to compute the changed
windows and send the results to the A11y framework immediately.
This can avoid the CTS failure with no corresponding window found
when the activity is launched due to the timing issue between the
callbacks for computing changed windows from the WM and the
callback for changed windows from the surface flinger.
Bug: 191736824
Test: a11y CTS & unit tests
Test: Manual testing including the A11y services
Change-Id: Ie9150ddc8e53ca88ad8cd7ba9376837e31436518
Uses
MagnificationInfoChangedCallback.onFullScreenMagnificationChanged
callback method to notify magnification hange instead of
accessing the public method of AccessibilityManagerService directly.
It reduces the interaction between AccessibilityManagerService and
FullScreenMagnificationControlller.
So that MagnificationController would take responsible for handling
notifyMagnificationChange().
Bug: 218540156
Test: atest FullScreenMagnificationControllerTest,
atest FullScreenMagnificationGestureHandlerTest,
atest MagnificationControllerTest,
Change-Id: Ib9f31681e442f1700a76250a5206199bad458c84
(cherry picked from commit 3c91f067b1)
Merged-In: Ib9f31681e442f1700a76250a5206199bad458c84
We used to use MagnificationScaleProvider to retrive the
scale when transitioning the mode, which is incorrect
because the provider only persists the scale changed by users.
If the scale is changed by public api, then the scale will
become inconsistent.
To fix it we choose the scale based on the target mode.
We also fix that mode transition without animation should be
considered as a transition case.
We also add more log for flaky CTS debugging.
Bug: 218792845
Test: atest MagnificationProcessorTest
atest com.android.server.accessibility.magnification
Change-Id: I47f49345ce46f726d6e06e203110ef793f35c76e
This way, we can pass back a11y sessions from direct startInput()
return value. And a11yManagerService will only request sessions
from a11y services which the client doesn't have sessions yet
(will not request existing a11y sessions again).
Bug: 187453053
Test: tested manually with talkback. Tested client switching within
"recent apps".
Change-Id: I8efdc9886ce33185a2195b741668c12e319ea660
An accessibility service needs to set FLAG_INPUT_METHOD_EDITOR
accessibilityFlags to use ime apis. Capability is not added
per Phil's suggestion.
Bug: 187453053
Test: CTS test added.
Change-Id: I78703c48e343ba09c2a0bf61c3447eea7337cb49
InputMethodService is the primary and a11y is the secondary.
InputMethodService is not
affected by a11y status. When the session from input method is
established, app can start input (pass input context to input method).
When an a11y session comes back, it will be passed to the app. When
InputMethodManagerService binds to/start input with
InputMethodService, it does the same to a11y services which
requested IME functionalities.
It is possible that input method can edit text before ally sessios are
established. So the EditorInfo passed to a11y could be stale.
So when an a11y session is passed to client, client will send
a notification (input method doesn't have this extra notification) for
the current selection. I think since the time for a11y session establish
shouldn't be long, and we get the current state later, it should be fine
for a11y services.
When input method is disconnected from app (client) (even for input
method switching), we cleared a11y and sessions too. When
input method request sessions, we must rerequest sessions for a11y. This
is mainly because when we unbindCurrentClientLocked(SWITCH_IME), we set
active to false for the current client. Suppose we don't want to
change the current structure of input method, an inactive client
probably should clear accessibility sessions too.
When we switch to a client which already has a session with input
method, there might be some a11y sessions with this client, and
some a11y services might be disabled or enabled while the client was
switched out. We pass unchanged a11y
sessions to client, and request sessions for newly enabled a11y services.
When an a11y service is disabled, it removes its session
from all clients in InputMethodManagerService.
Test: type word through modified "switchToInputMethod". Tested session
notification through logs. Tested client switching, input method
switching, a11y service enabled/disable, multiple a11y services,
a11y service enabled before device reboot.
Also tested work profile.
Bug: 187453053
Change-Id: Ia651a811093a939d00c081be1961e24ed3ad0356
