Log the UI latency with UIActionLatencyReported atom. We plan to
calulate the latency from when the SoundTrigger HAL emits an event to
when the VoiceInteraction system UI view is shown.
Test: device_config put latency_tracker enabled true; verify traces
and latency is logged in WW.
Bug: 247879896
Change-Id: I3edb91691e81f8f23660c0823a6388338fe926ad
Introducing a flag to gate the Attention Service usage by Hotword
Detection Service. The Attention Service will be disabled by default.
Bug: 247890608
Test: atest CtsVoiceInteractionTestCases
Change-Id: Ic8a6a11b2a0843a96f135b360f10f7fef5fce274
Revert submission 19600381-revert-17936037-hotword_proximity-ZZIDPDULZP
Test: atest CtsVoiceInteractionTestCases, atest CtsVoiceInteractionTestCases:android.voiceinteraction.cts.AlwaysOnHotwordDetectorTest#testAlwaysOnHotwordDetector_startRecognitionWithData -- --abi x86_64
Reason for revert: Including the fix for broken tests
Reverted Changes:
Ibfc659015:Revert "Refactor ProximityUpdateCallbackInternal t...
I1a03c21d5:Revert "Verify that proximity key is added to the ...
I6b79d9400:Revert "Add proximity state to the HotwordDetected...
Change-Id: Iee6ab086ccc3f27b95ee601d6be11231f300170c
Merged-In: Iee6ab086ccc3f27b95ee601d6be11231f300170c
Reverting the previous revert commit which was merged to resolve the
broken tests. Adding a fix to the broken tests by surrounding
mAttentionManagerInternal with null checks.
dbe5ad1a72
Test: atest CtsVoiceInteractionTestCases, atest CtsVoiceInteractionTestCases:android.voiceinteraction.cts.AlwaysOnHotwordDetectorTest#testAlwaysOnHotwordDetector_startRecognitionWithData -- --abi x86_64
Change-Id: Ife5fb9220518a32a89494217f56932f290cce5f3
Merged-In: Ife5fb9220518a32a89494217f56932f290cce5f3
RoleService should not be calling onRoleHoldersChanged() on
pre-created users, but that behavior cannot be changed because its
implemented by a system app, and pre-created users APIs are hidden.
To test it:
$ adb shell pm create-user --pre-create-only --guest
Success: created user id 12
$ adb shell pm create-user --guest ElGuesto
Success: created user id 12
$ adb shell cmd user list --all -v
4 users:
0: id=0, name=Driver, type=system.HEADLESS, flags=ADMIN|INITIALIZED|PRIMARY|SYSTEM (running)
1: id=10, name=Driver, type=full.SECONDARY, flags=ADMIN|FULL|INITIALIZED (running) (current)
2: id=11, name=null, type=full.SECONDARY, flags=FULL|INITIALIZED (pre-created)
3: id=12, name=ElGuesto, type=full.GUEST, flags=EPHEMERAL|FULL|GUEST|INITIALIZED (converted)
$ adb shell am switch-user 12
$ adb shell dumpsys voiceinteraction |egrep '(mBound|implementation)'
mBound=true mService=android.service.voice.IVoiceInteractionService$Stub$Proxy@77da1bc
$ adb shell cmd voiceinteraction show
Test: see bove
Bug: 216141085
Bug: 226201975
Merged-In: I12d9bb32e144ecf91ee4b452affea0dcea546127
Change-Id: I12d9bb32e144ecf91ee4b452affea0dcea546127
(cherry picked from commit 39acca1d72)
We add more metrics for software detection and
external source detection.
Android Metrics Design Review : eldar/276723226
Verify the metrics logs manually as below:
Step 1: m statsd_testdrive
Step 2: statsd_testdrive 434
Step 3: run the test
Step 4: We can see the log as below after 1 minutes from Step 2.
hotword_detector_events {
event: EXTERNAL_SOURCE_DETECTED
}
hotword_detector_events {
event: ON_DISCONNECTED
}
hotword_detector_events {
event: START_EXTERNAL_SOURCE_DETECTION
}
Bug: 207717787
Bug: 233704035
Test: build pass
Test: manual. The logs are expected after following test steps
Change-Id: Ib268902d63c580cec9cbd65a3816fd5ba003eaff
(cherry picked from commit 37f8b3ec2b)
this also fixes an NPE b/234828308
Bug: 226167520
Test: atest android.voiceinteraction.cts
Change-Id: Ibe20fb4da91fa80f8539bb19c90515167bf6e85b
Merged-In: Ibe20fb4da91fa80f8539bb19c90515167bf6e85b
By the original design, we register the activity event in the
ActivityManagerService. After getting the activity event, we will
query current visible activities and report them to Assistant.
Assistant will start to get direct actions of those visible
activities. But sometimes they can not get the direct actions
due to the stage of activity lifecycle before onStart().
We will postpone the request direct actions a few times
before onStart().
Bug: 224789902
Test: atest CtsVoiceInteractionTestCases
Change-Id: If7cd52c5ba749523e575c0be5595e8f20704c4f2
There are three event in which a client app is started:
1. When Assistant starts Voice Activity (startVoiceActivity)
2. When Assistant requests for a Direct Actions (requestDirectActions)
3. When the app starts invoice interation session
(startLocalVoiceInteraction)
Change-Id: I261ccc1a7136a601b5c8c8b766dc03a024cfe93d
Test: atest android.voiceinteraction.cts
By the behavior, the interactor of assistant needs to call the
startRecognition function again after hotword event is triggered.
Otherwise it will not receive the result of detection.
Currently when security exception occurs during onDetected, we
won't inform the interactor of assistant about the hotword event.
It will cause the interactor still to wait the hotword event and
doesn't call startRecognition function again.
After syncing with assistant, we should use onError() callback
to inform interactor.
The behavior of onError() in the interactor:
For both of DSP and Software, assistant will call startRecognition
function.
The behavior of onRejected() in the interactor:
Only for DSP, assistant will call startRecognition function.
For Software, assistant doesn't expect to get onRejected()
Bug: 230738063
Test: atest HotwordDetectionServiceBasicTest
Change-Id: I3083fd5a4a78ad9d2f35acd7fa1a4f2036cf64de
Merged-In: Ideb210d4364d4c48c747d66268d77a2f5b017186
Android 13 introduces a feature that allows users to kill foreground
services that will break the home screen mic function. The root cause
is because we don't handle the binding die case. Ideally we should
migrate the force stop, package update cases to onBindingDied but we
would like to minimize the risk before release. From the fgs task
manager description, we can check new REASON_USER_REQUESTED reason on
ApplicationExitInfo, we only handle this case to rebind the service
to minimize the scope.
Bug: 225166047
Test: atest CtsVoiceInteractionTestCases
Test: manual. The function still works after adb shell cmd activity
stop-app com.google.android.googlequicksearchbox
Change-Id: Ia8395fc6f7b6df36b158f0578d57d948748a43b3
The requirement to use a sandboxed HotwordDetectionService to avoid
incurring the mic indicator will be added back in a future release.
With this change, SoundTrigger events and hotword-source audio do not
incur the mic indicator for any process.
This change adds back the permissions logic from
I3275647d0f9a6e3ce8b97a556f56723b49170c8e, adjusted to account for the
changes from I4fc3b3e8defed59a900fd156273e9e695a322b0c (preflight
permission checks were made to accept soft denials, so simply reusing
enforcePermissionForPreflight does not work anymore - it would pop up a
dialog if the mic sensor is muted).
We preserve the behavior of showing the indicator when the detected
event is delivered from the Trusted Hotword process (no false-positives
here, so we can accurately show it).
Fix: 228092460
Test: manual - dsp and non-dsp; true and false positives; mute mic
sensor - events aren't delivered and no dialog is shown, hotword still
works after unmuting; mute mic and restart, unmute after starting reco;
Trusted and non-Trusted hotwording
Test: atest CameraMicIndicatorsPermissionTest
Test: atest HotwordDetectionServiceBasicTest
Test: atest android.cts.statsdatom.appops.AppOpsTests#testAppOps
Change-Id: I68f2f37b5ce835e7fd8649b382eaee9fc299ec79
(cherry picked from commit 3f961f24f4)
- The listener currently tracks session shown/hidden, but doesn't track
the visibility of the session window which can change on the client
side without the server knowing. In some cases, it's useful for SysUI
to be able to track the visibility of the session window to show/hide
bars appropriately.
Bug: 222308557
Test: Manual, launch the assistant and verify the calls are made
Change-Id: I9b4bb27c9497b2a828616caf72eaf345b072c294
We use "DETECT_EXCEPTION" in three different places, it is difficult
to distinguish what is realy happening. We use different metrics values
to replace them. And we also add a new metrics value to log onRejected
when the trusted process is restarted.
Android Metrics Design Review : eldar/276723226
Bug: 207717787
Test: build pass
Change-Id: Ib631bac3e5d6663c7eb64f54516b86b7fd9b8144
Because the client callbacks from SoundTriggerMiddleware service are
async, if a client quickly stops and starts a model, then receives a
detection event for that model, it is impossible for it to tell
whether the event corresponding to the previous or current session,
and thus is unable to reason about the resulting state of the model.
To mitigate that, we will always wait for an abort event when a model
is stopped before any other operation is attempted. This way the
model state stays synchronized.
Bug: 191935600
Fixes: 226926627
Test: Manual verification of soundtrigger use-cases. Specifically,
quick toggling of Assitant and Now Playing via Settings.
Change-Id: Ia1b91e78aa6981a7b3612ae9213af1b37a333d00
This reverts commit 6e2eb81ac4.
The original commit fixed b/223922855, which resulted from fixing
b/191935600, but created another.
Blocking the stop() call until the event confirming the stop arrives is
a bad idea, since:
- We have to give up the lock when waiting for the event, and thus lose
the atomicity of some of the operations. b/226926627 is an example of
this.
- It turns out some of the forward calls happen from the main thread's
looper, which is also used for callbacks, and we risk deadlocks.
A follow-up commit will address the fix in a better way.
Bug: 191935600
Bug: 226926627
Bug: 223922855
Test: See next commit in the chain.
Change-Id: I2b9987ae93803d3b4f72a1df56c88e284bed43c5
If the app starts a new Activity in the same task then the
finishTranslation() is called, the finish translation doesn't work
because our logic is to check if task top Activity is our expected
translation Activity. In this case, the top Activity is the new
Activity, the original Activity is paused in the same task. To fix
the problem, we get all non-finishing Activities in the task and
check if the target Activity in this task.
Bug: 202570916
Test: atest UiTranslationManagerTest
Test: atest CtsVoiceInteractionTestCases
Test: atest CtsAssistTestCases
Change-Id: I2471e04b894890777e13dd45c290f059dbe5f487
We use a flag to control the restart period time, the value is static
and set in class level that means if we update the device config, we
should reboot the device to get the updated value.
This change reads the value in the constructor, so the value will be
updated when the voice interaction app updates no need to reboot.
Ideally it's good to listen the value change dynamically but it
increases the complexity here, we provide a workable and safe change
now.
Bug: 224618257
Test: update the package and the value is expected by command:
adb shell dumpsys voiceinteraction | grep RESTART_PERIOD_SECONDS
Change-Id: Id7d5f2d4f17de1f36baf6509598a82d6d73c5b53
On native audioserver recovery the assistant UIDs will reset to an empty
list, followed by adding the current role or setting holder for
assistant. Also added a settings lock around the update assistant
method. Added remove assistant UID for reset hotword detection
connection.
Bug: 222584963
Bug: 222777782
Test: atest VoiceInteractionTest
Test: atest AudioManagerTest
Test: atest HotwordDetectionServiceBasicTest
Test: kill audio server and verify assistant UIDs is not increasing
unbounded.
Test: adb shell cmd voiceinteraction restart-detection, verify assistant
UIDs list is not increasing unbounded.
Change-Id: I475bfee1f0bcd56a750e4fef8aa8a2b93fe090fc
The modelData callback may be nullified before actually being invoked,
e.g. when calling startRecognition() on an already running model, with
a different callback.
Fixes: 225768984
Test: Manual reproduction on the issue and then applying the fix and
verifying that it works.
Change-Id: I089061123c8cf12eef19fdf16bb3a2c106ea2438
Because the client callbacks from SoundTriggerMiddleware service are
async, if a client quickly stops and starts a model, then receives a
detection event for that model, it is impossible for it to tell
whether the event corresponding to the previous or current session,
and thus is unable to reason about the resulting state of the model.
To mitigate that, we will always wait for an abort event when a model
is stopped before any other operation is attempted. This way the
model state stays synchronized.
Fixes: 191935600
Test: Manual verification of soundtrigger use-cases. Specifically,
quick toggling of Assitant and Now Playing via Settings.
Change-Id: I010558e8fa1891922f20dcdb7b0ca04c9500c2e1
Bug: 224618257
Test: adb shell cmd device_config get/put voice_interaction \
restart_period_in_seconds works fine
Test: the value is expected by command: adb shell dumpsys \
voiceinteraction | grep RESTART_PERIOD_SECONDS
Change-Id: Icd8d19c2a482d0d6b9bf0432e2176ca10f3bd2eb
Mitigates quality risk that may arise due to mismanaged state around the
restarts (either on the platform or application side).
Restarts will be flag-controlled and ramped back up later.
Bug: 224618257
Test: manual
Change-Id: I3b17b47fab4120b42d75443b213be3c1ef5491aa
Bug: 207717787
Test: build & boot pass.
Test: manual. Both trust and no-trust still work.
Test: manual. In nornal case, log with expected values, see bug for
details. Log expected value for some error cases (not all are tested)
by local changes.
Change-Id: Iaa778616eca4cfad83a94297d9cd6116bb9577e7
This change only provide a utilility class to help hotword dection
metrics logging. The log writing will on the follow up changes.
Bug: 207717787
Test: build pass can boot success
Change-Id: If436aae1b4ebdb06c7bd5a1c0b8d6df4e94488cd
Adds a new API to VoiceInteractor to allow apps to query
the package name of the Voice Interactor associated with
this the current voice interaction
BUG: 192061561
CTS-Coverage-Bug: 218687104
Test: Manual validation from personal and work profiles
Change-Id: If381c38498535bf94d13c0b5ad5f6c4fa5495117
