A proxy will give a privileged app access to a11y data typically available to AccessibilityServices. (Refer to the design doc linked in the comments for more information.)
A proxy will easily be able to set modify a list of service infos
depending on what a11y services/features are running on the client
device.
Introduce the system connection for a proxy:
- The service connection needs to maintain a single info that
represents this proxy-supplied list, so merge list values.
- Populate properties for each listed info that are default populated
for services installed on the device. This will prevent exceptions if
an app accesses these properties.
- Make setAccessibilityTool a SystemAPI, so a proxy can set this if it
needs to, since it cannot set this before runtime.
Test: builds, atest ProxyAccessibilityServiceConnectionTest
Bug: 241429275
Change-Id: Id99e2fa08be567738751d7a9a84f0014279924a1
This will simplify the management of active a11y services when dealing
with proxies.
Test: builds
Bug: 241429275
Change-Id: I6785b9bea218946e95941453d637198bb0230a78
AccessibilityServiceInfo#setAccessibilityTool() is already a hidden
method but reflection techniques may be able to bypass this. So, we
shouldn't allow this property to be modified at runtime by an
a11yservice after the service info is parsed from the XML.
This field was originally added to configurable properties to allow
modification during CTS tests.
This change replaces dynamic configuration with a flag in UiAutomation
that is used when connecting a new UiAutomation instance. CTS tests
which wish to change isAccessibilityTool mid-test can reconnect their
UiAutomation with a new flag value. The flag is @TestApi since it is
unlikely non-CTS developers will want it for their own tests.
Fix: 248335757
Test: atest AccessibilityEndToEndTest
Change-Id: I6ca3d9e6fb94207897cdfc94925bbd4d54348a29
This is a follow up CL to my previous CL [1], which aimed to clean up
A11yIME without changing any semantics. One thing I overlooked is that
android.accessibilityservice.InputMethod#mStartedInputConnection
can be set to null at any time (on the UI thread) and
InputMethod.SessionImpl#invalidateInput()
needs to gracefully do early-exit as needed.
[1]: I5ff2e804cbcf90828370a0612ff54111130bdff4
c60176c1f3
Fix: 245973012
Test: atest CtsInputMethodTestCases:AccessibilityInputMethodTest
Test: atest CtsAccessibilityServiceTestCases:AccessibilityInputConnectionTest
Test: atest CtsAccessibilityServiceTestCases:AccessibilityImeTest
Change-Id: I3bacff7772d20660bfdbeff724a0d19b5f0e6fe4
Revert submission 19246592-accessibility-tool-only
Reason for revert: Jank regression b/242267091
Reverted Changes:
I0685e6dd4:CTS tests for the accessibilityDataPrivate propert...
I1c02af88c:Limit off-label accessibility access using a new p...
Change-Id: I701b811fd4fb2634872a759134356736a7e8b7dd
Callback will continue to happen on the main thread, which is
wrapped in an executor. Remove the AccessibilityEvent recycling as well.
EXO support will leverage this.
Bug: 240979189
Test: Manual with TalkBack, atest CtsAccessibilityServiceTestCases
atest CtsAccessibilityTestCases
atest CtsUiAutomationTestCases
atest FrameworksServicesTests:com.android.server.accessibility
atest FrameworksCoreTests:com.android.internal.accessibility
atest FrameworksCoreTests:android.view.accessibility
Change-Id: I12c7677d85ad81a3a5def52039642cb167c17ca8
This is a follow up CL to our previuos CL [1], which enabled
AccessibilityService to use a subset of InputConnection APIs.
In that CL we have reused existing AIDL interfaces that were designed
and maintained for IMEs for simplicity, where a non trivial amount of
unnecessary IPC endpoints were included.
From the security and maintainability viewpoints, however, exposing
unnecessary IPC endpoints is discouraged in general. To address such
concerns this CL introduces a set of dedicated IPC definitions for
A11yIME so that we do not need to reuse IPCs for IMEs.
This CL also stops passing InputBinding object to A11yIME process as
it contains IInputContext Binder Proxy, which can still be used to
directly invoke fallback InputConnection. This is doable now because
A11yIME no longer relies on fallback InputConnection [2].
This CL is should not have any observable changes in the semantics.
End-to-end CTS tests guarantee that everything is still working as
intended now and in the future.
[1]: Ia651a811093a939d00c081be1961e24ed3ad0356
fb17e5ae7a
[2]: I2af3cd50444d8ddf25aa0f6479238156914e6fff
dc635efb68
Fix: 215633021
Fix: 215636776
Test: atest CtsInputMethodTestCases:AccessibilityInputMethodTest
Test: atest CtsAccessibilityServiceTestCases:AccessibilityInputConnectionTest
Test: atest CtsAccessibilityServiceTestCases:AccessibilityImeTest
Change-Id: I5ff2e804cbcf90828370a0612ff54111130bdff4
This is a follow up CL to our previous CL [1], which copied many
things from
android.inputmethodservice.InputMethodService
to
android.accessibilityservice.InputMethod.
As a result currently those two classes share the same systrace method
names as follows.
* IMS.startInput
* IMS.restartInput
With this CL, those systrace method names will be renamed to
* AccessibilityService.startInput
* AccessibilityService.restartInput
so that readers can understand which implementation is used without
having to check its process name.
There should be no user-observable behavior change in this CL.
[1]: Ia651a811093a939d00c081be1961e24ed3ad0356
fb17e5ae7a
Fix: 229118866
Test: Manually verified as follows
1. Build aosp_coral-userdebug and flash it.
2. make -j EditTextVariations
3. adb install -r $ANDROID_TARGET_OUT_TESTCASES/EditTextVariations/arm64/EditTextVariations.apk
4. make -j SampleInputMethodAccessibilityService
5. adb install -r $OUT/system/app/SampleInputMethodAccessibilityService/SampleInputMethodAccessibilityService.apk
6. adb shell settings put secure enabled_accessibility_services com.example.android.sampleinputmethodaccessibilityservice/com.example.android.sampleinputmethodaccessibilityservice.SampleInputMethodAccessibilityService
7. external/perfetto/tools/record_android_trace wm
8. adb shell am start -n com.android.inputmethod.tools.edittextvariations/.EditTextVariations
9. Tap an input field '<input type="text" />' in the WevView section.
10. Tap an input field '<input type="email" />' in the WevView section.
11. Hit Ctrl-C.
12. Verify trace event names from the sample A11yIME.
Change-Id: If89cfaec108a15aa168cfa32c689ca832b410324
Unlike InputMethodService, A11y IME should never receive fallback
InputConnection, which is subject to be deprecated or demoted in the
future.
Fix: 228848796
Test: atest CtsInputMethodTestCases:AccessibilityInputMethodTest
Change-Id: I2af3cd50444d8ddf25aa0f6479238156914e6fff
With my previous CL [1],
android.accessibilityservice.InputMethod#invalidateInputInternal
should now be executed during CTS.
Fix: 217788708
Test: atest CtsInputMethodTestCases:AccessibilityInputMethodTest
Change-Id: I74b06d4211d771f769f3ced6b8dedd37007257a0
We need to be able to initiate touch exploration even if there are no pointers on the screen in order to reliably cover the case where a user just taps the screen.
The user expects to hear if they have tapped on anything.
Bug: 223673301
Test: manual. Tap on the screen and confirm that talkback reads the item in that position.
Change-Id: Id96fa8bd7f000ea153f45024e06915a43ca80826
If this feature is not supported,
1. Settings should hide window magnification settings.
2. Magnification mode settings should fallback to.
full-screen settings.
Bug: 225086379
Test: none
Change-Id: I3248034d25eb88136bda1874092319d4702be455
Per API review feedback.
Test: it builds fine.
Fix: 219740392
Change-Id: Ibad4ead04f4420d8cd80969a9266a419c6ea93e8
(cherry picked from commit 8c4ce40ec9)
When the flag is set dynmatically, we need to create the InputMethod()
instance and request to bind ime. If the flag is unset, we clear the
InputMethod() instance and unbind ime.
Fix: 218193835
Test: Tested with talkback. When the flag is not set, we can't use
ime. When the flag is set, we can use ime again (dynamically set
flags).
Change-Id: Ied48d452108a593e8e4ee5c67c3208acbc87335e
Change-Id: I2a85c64b089983807cb7627de9b8512e8f716426
(cherry picked from commit 14e78228a1)
- Explicitly say that this is only the class name,to be looked up in the a11y services package, and not a full component name
- Fix typo of Javadoc refers to class names (plurar)
- Rename to getTileServiceName()
Bug: 219873375
Test: atest AccessibilityShortcutInfoTest
Change-Id: Ic7231f6e0b34edabea8b4086102e1d7f6ebe0eae
Make the prefetching strategies public so a service can choose
which strategy works best in a particular spot. This should reduce
unnecessary/redundant prefetching.
For example, the FW currently only allows hybrid descendant
prefetching, but a service may want to do depth-first or
breadth-first traversal of the view hierarchy.
Currently, if there is another user interactive request, we
immediately return prefetched nodes. Also allow services to prevent
this interruption and force prefetching to a max of 50 nodes.
Services could potentially request a certain number of nodes, but
since asynchronous prefetching immediately returns the requested
node, the service can force prefetching of 50 nodes if desired,
and only exposing strategies touches less code, I prefer limiting this.
Also use a LinkedHashMap so ordering is kept when prefetching
descendants.
Test: Manual, talkback builds
atest AccessibilityCacheTest,
AccessibilityInteractionControllerNodeRequestsTest
Bug: 192489177
Change-Id: I3d8358411ece5d2e1380282824cd3cf1835658ac
An accessibility service needs to set FLAG_INPUT_METHOD_EDITOR
accessibilityFlags to use ime apis. Capability is not added
per Phil's suggestion.
Bug: 187453053
Test: CTS test added.
Change-Id: I78703c48e343ba09c2a0bf61c3447eea7337cb49
InputMethodService is the primary and a11y is the secondary.
InputMethodService is not
affected by a11y status. When the session from input method is
established, app can start input (pass input context to input method).
When an a11y session comes back, it will be passed to the app. When
InputMethodManagerService binds to/start input with
InputMethodService, it does the same to a11y services which
requested IME functionalities.
It is possible that input method can edit text before ally sessios are
established. So the EditorInfo passed to a11y could be stale.
So when an a11y session is passed to client, client will send
a notification (input method doesn't have this extra notification) for
the current selection. I think since the time for a11y session establish
shouldn't be long, and we get the current state later, it should be fine
for a11y services.
When input method is disconnected from app (client) (even for input
method switching), we cleared a11y and sessions too. When
input method request sessions, we must rerequest sessions for a11y. This
is mainly because when we unbindCurrentClientLocked(SWITCH_IME), we set
active to false for the current client. Suppose we don't want to
change the current structure of input method, an inactive client
probably should clear accessibility sessions too.
When we switch to a client which already has a session with input
method, there might be some a11y sessions with this client, and
some a11y services might be disabled or enabled while the client was
switched out. We pass unchanged a11y
sessions to client, and request sessions for newly enabled a11y services.
When an a11y service is disabled, it removes its session
from all clients in InputMethodManagerService.
Test: type word through modified "switchToInputMethod". Tested session
notification through logs. Tested client switching, input method
switching, a11y service enabled/disable, multiple a11y services,
a11y service enabled before device reboot.
Also tested work profile.
Bug: 187453053
Change-Id: Ia651a811093a939d00c081be1961e24ed3ad0356
This is necessary to make the drag -> delegate execution path work.
Bug: 159651900
Test: atest TouchInteractionControllerTest
Change-Id: Idfb5d5ea8290353a7f4dac9742826a38fdd17b04
When transitioning between states, TouchInteractionController will queue motion events and not deliver them until the state transition has been confirmed.
This avoids the case where a service will request a state transition twice because it processes a motion event after sending the transition request but before the transition completes.
Bug: 159651900
Test: atest TouchInteractionControllerTest
Change-Id: I2437b635a8053de28212d93e67c7144134c1dc08
