This API allows accessibility services to intercept motions events, to
expand device control capabilities to more types of input devices
such as watch rotary wheels and gamepad joysticks.
Unlike KeyEvents which are optionally consumed, for MotionEvents the
service must declare the event sources it wishes to consume and then
events from these sources are *always* consumed. The source list can be
changed by the service at runtime. By always consuming the requested
events we avoid the potential "wait and see" latency concerns that
onKeyEvent deals with.
Bug: 247550565
Test: atest AccessibilityEndtoEndTest
Test: use in an a11yservice to observe joystick events
Change-Id: Iea54b00b21f5348f50d94d6e2213df24c0c8a36a
This is used by end-to-end tests exercising the input filter for generic
motion events.
Bug: 247550565
Test: atest AccessibilityEndToEndTest, using the dependent change topic
Change-Id: I2695ecb3919d8336dc3f6df1a525a53d84c03fec
These are:
- onProxyConnected (equivalent to onServiceConnected)
- onAccessibilityEvent
- onInterrupt
- getWindows
- set/getInstalledAndEnabledServices
Connect the proxy client (IAccessibilityServiceClient) to a
proxy service connection
(ProxyAbstractAccessibilityServiceConnection) in ProxyManager,
so the A11yDisplayProxy can access the UI.
AccessibilityServices run on their own thread. To avoid using the
app's main thread, the app that registers the A11yDisplayProxy
can call it on a different thread. Add a note in the proxy class.
Future TODOs (b/254545943):
The a11y state returned to A11yManager includes state
of all proxies and AccessibilityServices.
Separation will require AccessibiilityManager to go from being a
process singleton to a per-display instance.
Input and a11y focus are singletons and may
shift between a proxy display and a display on the device.
Separation will require future changes.
ProxyManager behaves more like UiAutomationManager,
and centralizes proxy management. Separation into different user
states will require future refactoring.
Bug: 241429275
Test: atest AccessibilityDisplayProxyTest,
AccessibilityManagerServiceTest, manual test app that
registers/unregisters A11yDisplayProxy
Change-Id: I830c6613c2c2f8abcc2343c5110f4e7b7494fb0e
This enables accessibility services to take a screenshot of
a window without any overlay windows drawn on top, which is
beneficial for ML-based understanding of the visual content
in the window.
Bug: 249830826
Test: atest AccessibilityTakeScreenshotTest
Test: use in an a11y service, observe screenshots
contain only content from the requested windows
and no overlays
Change-Id: Ic886aeace9cb2984a472273858281b4f2c3a0e46
An A11yDisplayProxy will be used by Exo to listen to UI changes and
perform actions on the UI.
For a Display that is shown on a ChromeBook/client device,
a privileged app like Exo, or an app with SystemAPI privileges and
the MANAGE_ACCESSIBILITY permission, should call
A11yManager.registerA11yProxy and A11yManager.unRegisterA11yProxy to
enable/disable a11y.
The app will only have visibility in the specified proxy display.
Currently only few A11yProxy methods are exposed. When the proxy is
plugged into the system via ProxyAccessibilityServiceConnection
other methods will be exposed. (See link in comments)
Bug: 241429275
Test: atest A11yProxyTest(cts), atest A11yManagerTest(unit)
Change-Id: I019732e50e6beba0af7f5898a07b44d25e03416b
Otherwise, as soon as we rebuild the input filter e.g. enable magnification, service gesture detection gets turned off.
Bug: 243487199
Test: atest TouchInteractionControllerTest
Change-Id: Ifdfaffd2ff2addc7a4dd7d69cf6128947efdacea
This could cause race conditions if a service connection is exited after
starting binding but before finishing binding.
If the service is mid-initialization when it gets disabled then there is
already logic to prevent full initialization and to exit the connection.
Bug: 252884894
Test: atest AccessibilityPrivacySourceTest --iterations 30
Change-Id: Ib5aa761bb4c9189df53d04d7dc5329aece650141
A proxy will give a privileged app access to a11y data typically available to AccessibilityServices. (Refer to the design doc linked in the comments for more information.)
A proxy will easily be able to set modify a list of service infos
depending on what a11y services/features are running on the client
device.
Introduce the system connection for a proxy:
- The service connection needs to maintain a single info that
represents this proxy-supplied list, so merge list values.
- Populate properties for each listed info that are default populated
for services installed on the device. This will prevent exceptions if
an app accesses these properties.
- Make setAccessibilityTool a SystemAPI, so a proxy can set this if it
needs to, since it cannot set this before runtime.
Test: builds, atest ProxyAccessibilityServiceConnectionTest
Bug: 241429275
Change-Id: Id99e2fa08be567738751d7a9a84f0014279924a1
Previous logic would exit the loop after removing the first service
matching the uninstalled package.
Bug: 243378132
Test: atest AccessibilityEndToEndTest
Test: m sts;
sts-tradefed run sts-dynamic-develop -m \
CtsAccessibilityServiceTestCases
Change-Id: I4ba30345d8600674ee8a9ea3ff411aecbf3655a3
- Implement register/unregisterProxy minimally. These are mostly
validity checks. Future CLs will expose the SystemAPI methods on
A11yManager so an app can call through to these.
- Introduce a ProxyManager skeleton. This will be populated and tested
in future CLs.
Bug: 241429275
Test: builds, atest AccessibilityManagerServiceTest
Change-Id: Idebf46590b5a129381a34496de67ed18d43369ed
This will simplify the management of active a11y services when dealing
with proxies.
Test: builds
Bug: 241429275
Change-Id: I6785b9bea218946e95941453d637198bb0230a78
This is a follow up CL to our previous CL [1], which added a
method invocation of
AccessibilityManager#isTouchExplorationEnabled()
from InputMethodManagerService. The issue is that it is not clear
about which user we are querying. In order to avoid future
confusions, it would make much more sense if we have a more dedicated
internal method in AccessibilityManagerInternal as we planned.
Bug: 207007377
Fix: 219056452
Test: presubmit
Change-Id: I99a1b6706c51709e82ca31449753b35919e7247c
This was useful when the A11yUserState was temporarily modified
while the keyguard was on. However, ever since the a11y gesture
from commit 106fe73205
the A11yUserState is no longer temporarily modified this way.
Bug: 247045101
Test: a11y presubmit tests
Change-Id: I6fde21fe84cea3cae5f0dfa86fcd811a22542cc5
When window is removed, there can be a race between a11y and wm,
and sometimes A11yManagerService#sendA11yEvent cannot get a correct
display id.
There's also a case where a11y events for windowless windows don't have
a display id.
This change fixes the issue by populating display id in
A11yWindowManager#DisplayWindowsObserver for windows change events.
Bug: 240888087
Test: AccessibilityWindowManagerTest
Test: CtsAccessibilityServiceTestCases
Change-Id: I441d98e4431913ae24ab354423828d2a031b27b4
