Make the prefetching strategies public so a service can choose
which strategy works best in a particular spot. This should reduce
unnecessary/redundant prefetching.
For example, the FW currently only allows hybrid descendant
prefetching, but a service may want to do depth-first or
breadth-first traversal of the view hierarchy.
Currently, if there is another user interactive request, we
immediately return prefetched nodes. Also allow services to prevent
this interruption and force prefetching to a max of 50 nodes.
Services could potentially request a certain number of nodes, but
since asynchronous prefetching immediately returns the requested
node, the service can force prefetching of 50 nodes if desired,
and only exposing strategies touches less code, I prefer limiting this.
Also use a LinkedHashMap so ordering is kept when prefetching
descendants.
Test: Manual, talkback builds
atest AccessibilityCacheTest,
AccessibilityInteractionControllerNodeRequestsTest
Bug: 192489177
Change-Id: I3d8358411ece5d2e1380282824cd3cf1835658ac
An accessibility service needs to set FLAG_INPUT_METHOD_EDITOR
accessibilityFlags to use ime apis. Capability is not added
per Phil's suggestion.
Bug: 187453053
Test: CTS test added.
Change-Id: I78703c48e343ba09c2a0bf61c3447eea7337cb49
InputMethodService is the primary and a11y is the secondary.
InputMethodService is not
affected by a11y status. When the session from input method is
established, app can start input (pass input context to input method).
When an a11y session comes back, it will be passed to the app. When
InputMethodManagerService binds to/start input with
InputMethodService, it does the same to a11y services which
requested IME functionalities.
It is possible that input method can edit text before ally sessios are
established. So the EditorInfo passed to a11y could be stale.
So when an a11y session is passed to client, client will send
a notification (input method doesn't have this extra notification) for
the current selection. I think since the time for a11y session establish
shouldn't be long, and we get the current state later, it should be fine
for a11y services.
When input method is disconnected from app (client) (even for input
method switching), we cleared a11y and sessions too. When
input method request sessions, we must rerequest sessions for a11y. This
is mainly because when we unbindCurrentClientLocked(SWITCH_IME), we set
active to false for the current client. Suppose we don't want to
change the current structure of input method, an inactive client
probably should clear accessibility sessions too.
When we switch to a client which already has a session with input
method, there might be some a11y sessions with this client, and
some a11y services might be disabled or enabled while the client was
switched out. We pass unchanged a11y
sessions to client, and request sessions for newly enabled a11y services.
When an a11y service is disabled, it removes its session
from all clients in InputMethodManagerService.
Test: type word through modified "switchToInputMethod". Tested session
notification through logs. Tested client switching, input method
switching, a11y service enabled/disable, multiple a11y services,
a11y service enabled before device reboot.
Also tested work profile.
Bug: 187453053
Change-Id: Ia651a811093a939d00c081be1961e24ed3ad0356
This is necessary to make the drag -> delegate execution path work.
Bug: 159651900
Test: atest TouchInteractionControllerTest
Change-Id: Idfb5d5ea8290353a7f4dac9742826a38fdd17b04
When transitioning between states, TouchInteractionController will queue motion events and not deliver them until the state transition has been confirmed.
This avoids the case where a service will request a state transition twice because it processes a motion event after sending the transition request but before the transition completes.
Bug: 159651900
Test: atest TouchInteractionControllerTest
Change-Id: I2437b635a8053de28212d93e67c7144134c1dc08
This reverts commit 331be9a643.
Reintroducing ag/16366278 since it seems unrelated to b/214053959 (more details on b/214053959#comment55).
Original commit message:
Migrate unsafe parcel APIs in framework-minus-apex
Migrate the following unsafe parcel APIs in framework-minus-apex:
* Parcel.readSerializable()
* Parcel.readArrayList()
* Parcel.readList()
* Parcel.readParcelable()
* Parcel.readParcelableList()
* Parcel.readSparseArray()
This CL was generated by applying lint fixes that infer the expected
type from the caller code and provide that as the type parameter
(ag/16365240).
A few observations:
* In some classes we couldn't migrate because the class also belonged to
another build module whose min SDK wasn't current (as is the case for
framework-minus-apex), hence I suppressed the lint check
(since I'll eventually submit the lint check to the tree).
* In some cases, I needed to do the cast in
https://stackoverflow.com/a/1080525/5765705 to make the compiler happy
since there isn't another way of providing a class of type
Class<MyClassWithGenerics<T>>.
* In the readSerializable() case, the new API also requires the class
loader, that was inferred to by InferredClass.class.getClassLoader().
* Note that automatic formatting and import rely on running hooked up
to the IDE, which wasn't the case here.
Bug: 195622897
Change-Id: I272432e6e082a973f7a50492ec35d79c2b577c93
Test: TH passes
Manually migrate the few cases of:
* readArray()
* readParcelableArray()
To the new parcel APIs that take the expected type as the last
parameter. This enhances security because it prevents unexpected types
*before* running unparcelling code. More details at go/safer-parcel.
Owners, please check that the type of the objects expected is always a
subtype of the type provided as the 3rd parameter. This is usually easy
to verify due to casts that happen shortly after.
These changes often allowed further transformations but I decided to
avoid them to keep this change small and targeted.
This was manual since it's tricky to get lint to infer the type in
those cases and it was only a few.
Bug: 195622897
Test: TH passes
Change-Id: I262ed7cd6d3bc15b32e9296e88a8a67fdb59e880
See go/b200769372 - the legacy public methods
The legacy magnification controller methods of a11y service,
getMagnificationRegion(),
reset(),
will keep the behavior before that control
only full-screen magnification.
To make the a11y service able to control the actiavted
magnifier on the display, the service should use the
new public APIs,
1. getCurrentMagnificationRegion(),
2. resetCurrentMagnification(),
instead of the legacy APIs.
Bug: 210069654
Test: atest AccessibilityMagnificationTest,
atest MagnificationProcessorTest,
atest AbstractAccessibilityServiceConnectionTest,
Change-Id: I76bfee43cbb76a9aeec33574a8db18430903e08c
See Design doc: go/b200769372
To make the behavior consistent on the new platform,
The legcay callback,
onMagnificationChanged(MagnificationController controller,
Region region, float scale, float centerX, float centerY)
keep notifying only full-screen magnification change as
before.
To support listening to the magnification changes of all
magnification modes, the service should overide
The new callback proposed in T,
onMagnificationChanged(MagnificationController controller,
Region region, MagnificationConfig config).
TODO: Notify magnifcation change when window magnifier turns off
Bug: 203013925
Test: atest AccessibilityMagnificationTest,
atest MagnificationControllerTest,
atest WindowMagnificationManagerTest,
atest WindowMagnificationControllerTest,
atest FullScreenMagnificationControllerTest,
Change-Id: Ia73195a71f55bb26ef3f2eafce0dc73d8a65583b
Migrate the following unsafe parcel APIs in framework-minus-apex:
* Parcel.readSerializable()
* Parcel.readArrayList()
* Parcel.readList()
* Parcel.readParcelable()
* Parcel.readParcelableList()
* Parcel.readSparseArray()
This CL was generated by applying lint fixes that infer the expected
type from the caller code and provide that as the type parameter
(ag/16365240).
A few observations:
* In some classes we couldn't migrate because the class also belonged to
another build module whose min SDK wasn't current (as is the case for
framework-minus-apex), hence I suppressed the lint check
(since I'll eventually submit the lint check to the tree).
* In some cases, I needed to do the cast in
https://stackoverflow.com/a/1080525/5765705 to make the compiler happy
since there isn't another way of providing a class of type
Class<MyClassWithGenerics<T>>.
* In the readSerializable() case, the new API also requires the class
loader, that was inferred to by InferredClass.class.getClassLoader().
* Note that automatic formatting and import rely on running hooked up
to the IDE, which wasn't the case here.
Bug: 195622897
Test: TH passes
Change-Id: I11a27b9bdab7959ee86e90aa1e1cbebd7aaf883c
Does not yet include the prefetching API, which will be introduced later.
- Have a CacheController as the source of cache manipulation
- Notify system to update user relevant types
Bug: 192489177
Test: Build, added cts tests, atest CacheControllerTest
Change-Id: I150a01192dcf764ffcfe9848b499034a48aded08
Make accessibility services able to enable/disable a specified
IME in the same package for the current user.
Bug: 195476910
Fix: 191386474
Test: atest AccessibilitySoftKeyboardTest. Also tested with
modified talkback (use the swith input method API for enable
ime), and self created empty a11y service with an empty ime.
Mananged device and profile which set limit on user 0 are also
tested.
Change-Id: I4187468076705ac597d680f2f5dc32d7b166da1f
See go/b200769372 - the legacy public methods
The legacy magnification controller methods,
setScale(), setCenter(),
getScale(), getCenterX(), getCenterY(),
getMagnificationRegion(),
reset(),
will keep the behavior before that control
only full-screen magnification.
To make the service able to control the actiavted magnifier
on the display, the service should use the new public APIs,
1. setMagnificationConfig(),
2. getMagnificationConfig(),
3. getCurrentMagnificationRegion()
instead of the legacy APIs.
This change
1. Updates the doc of the legcacy APIs.
2. The legacy APIs controls only fullscreen magnification
TODO: Add new public methods, getCurrentMagnificationRegion(),
resetCurrentMagnification() for the current controlling magnifier
Bug: 210069654
Test: atest MagnificationProcessorTest,
atest AbstractAccessibilityServiceConnectionTest,
atest AccessibilityMagnificationTest,
Change-Id: Ia157971d2c9716df7aa14024525ee0adfdc0d46f
The split-screen behavior is changed, so systemui doesn't
support this action with a single activity. We don't handle
this action by default from sc-v2. If SystemUI has any behavior
changed, they could register this action to support it.
Bug: 207056257
Test: presubmit pass
Change-Id: If569eff1ac52601e6b88324d7b47630a53e198f2
See go/b200769372
AccessibilityService adds new magnificationController APIs,
setMagnificationConfig and getMagnificationConfig, to control
magnification on demand.
The service can set the magnification mode, scale, center postion
of the magnifier on the display to control magnification.
And the service controls the least recently activated magnifier
on the display if the config mode is not specified.
Bug: 199732498
Test: atest MagnificationProcessorTest,
atest AbstractAccessibilityServiceConnectionTest,
atest AccessibilityMagnificationTest,
Change-Id: I51be2bbf96411c6c54c9da30873556d09533bb65
1. Add a range to the parameter in setScale(), setCenterX(), setCenterY()
2. The IntDef annotation name of MagnificationMode should not be
capitalized. The constants names should also start with the same prefix
set in the IntDef annotation.
3. Adds @hide for the IntDef annotation.
4. The related constants uses consistent prefix
Bug: 208422801
Test: none
Change-Id: Ibd997c3a13385d8884e208564092d829833ae637
- Maintain a static collection of caches (sCaches). Lock is
sConnectionCache
- Initialize a cache when a connection is added. (client.addConnection)
- Remove the cache when a connection is removed. (client.removeConnection)
- System server should not have its cache initialized, can remove
parameter in AccessibiltyInteractionClient constructor
I tried to maintain a local reference (mAccessibilityCache) to the
service cache for each client to avoid accessing the static
collection for each request. But since the process is not immediately
killed when the service is turned off, ensuring resources are cleared
correctly is big time investment that can be done later if desired.
Bug: 192489177
Test: Build and use talkback, do the slew of tests.
atest CtsAccessibilityServiceTestCases CtsAccessibilityTestCases
CtsUiAutomationTestCases
FrameworksServicesTests:com.android.server.accessibility
FrameworksCoreTests:com.android.internal.accessibility
FrameworksCoreTests:android.view.accessibiliity
Change-Id: Ic30f6915c7b3c186a512ed8b410bf6e842eff5fb
MagnificationConfig stores the magnification configuration of the
controlling magnifier, such as magnification mode, scale and center
postion.
And MagnificationProcessor uses the config to controll the magnifier for
AccessibilityService.
See
API review doc: go/b200769372
Design doc: go/a11yservice_control_magnification_in_t
The chapter at Proposal A and MagnificationConfig
CTS: ag/15824302
Bug: 199732498
Test: atest AbstractAccessibilityServiceConnectionTest,
atest MagnificationProcessorTest,
atest WindowMagnificationManagerTest,
atest MagnificationConfigTest
Change-Id: I20323865d2efe1b40626f0c86767848733856482
From Android R, using the WindowManager from non-ui context
violates the vm policy. It may cause incorrect window bounds
with the given floating window. AccessibilityService provides
Accesibility overlay usage that needs to set window token to
the windowManager. However, Setting it when the service is bounded
violates the policy.
To fixt it, we set the window token when developers get WindowManager
first time.
Bug: 175785781
Test: atest AccessibilityOverlayTest, use the test apk to check if
the warning message is shown in the log
manual test to see if SelectToSpeak works well
Change-Id: Ia873488626aa4da111499282d1971f836cf111cd
Change-Id: Ia87df60c393bb0e40d85ced4184c78a09d7b427b
Allow AccessibilityService instances to request screenshots more
frequently (up to three screenshots per second). This used to be capped
at one screenshot per second in Android R.
Test: cts
Bug: 188077373
Change-Id: I880e2d922db3ce2bd075efcb02466ea1f4914792
(cherry picked from commit 113436f49f)
