There are three event in which a client app is started:
1. When Assistant starts Voice Activity (startVoiceActivity)
2. When Assistant requests for a Direct Actions (requestDirectActions)
3. When the app starts invoice interation session
(startLocalVoiceInteraction)
Change-Id: I261ccc1a7136a601b5c8c8b766dc03a024cfe93d
Test: atest android.voiceinteraction.cts
By the behavior, the interactor of assistant needs to call the
startRecognition function again after hotword event is triggered.
Otherwise it will not receive the result of detection.
Currently when security exception occurs during onDetected, we
won't inform the interactor of assistant about the hotword event.
It will cause the interactor still to wait the hotword event and
doesn't call startRecognition function again.
After syncing with assistant, we should use onError() callback
to inform interactor.
The behavior of onError() in the interactor:
For both of DSP and Software, assistant will call startRecognition
function.
The behavior of onRejected() in the interactor:
Only for DSP, assistant will call startRecognition function.
For Software, assistant doesn't expect to get onRejected()
Bug: 230738063
Test: atest HotwordDetectionServiceBasicTest
Change-Id: I3083fd5a4a78ad9d2f35acd7fa1a4f2036cf64de
Merged-In: Ideb210d4364d4c48c747d66268d77a2f5b017186
Android 13 introduces a feature that allows users to kill foreground
services that will break the home screen mic function. The root cause
is because we don't handle the binding die case. Ideally we should
migrate the force stop, package update cases to onBindingDied but we
would like to minimize the risk before release. From the fgs task
manager description, we can check new REASON_USER_REQUESTED reason on
ApplicationExitInfo, we only handle this case to rebind the service
to minimize the scope.
Bug: 225166047
Test: atest CtsVoiceInteractionTestCases
Test: manual. The function still works after adb shell cmd activity
stop-app com.google.android.googlequicksearchbox
Change-Id: Ia8395fc6f7b6df36b158f0578d57d948748a43b3
The requirement to use a sandboxed HotwordDetectionService to avoid
incurring the mic indicator will be added back in a future release.
With this change, SoundTrigger events and hotword-source audio do not
incur the mic indicator for any process.
This change adds back the permissions logic from
I3275647d0f9a6e3ce8b97a556f56723b49170c8e, adjusted to account for the
changes from I4fc3b3e8defed59a900fd156273e9e695a322b0c (preflight
permission checks were made to accept soft denials, so simply reusing
enforcePermissionForPreflight does not work anymore - it would pop up a
dialog if the mic sensor is muted).
We preserve the behavior of showing the indicator when the detected
event is delivered from the Trusted Hotword process (no false-positives
here, so we can accurately show it).
Fix: 228092460
Test: manual - dsp and non-dsp; true and false positives; mute mic
sensor - events aren't delivered and no dialog is shown, hotword still
works after unmuting; mute mic and restart, unmute after starting reco;
Trusted and non-Trusted hotwording
Test: atest CameraMicIndicatorsPermissionTest
Test: atest HotwordDetectionServiceBasicTest
Test: atest android.cts.statsdatom.appops.AppOpsTests#testAppOps
Change-Id: I68f2f37b5ce835e7fd8649b382eaee9fc299ec79
(cherry picked from commit 3f961f24f4)
- The listener currently tracks session shown/hidden, but doesn't track
the visibility of the session window which can change on the client
side without the server knowing. In some cases, it's useful for SysUI
to be able to track the visibility of the session window to show/hide
bars appropriately.
Bug: 222308557
Test: Manual, launch the assistant and verify the calls are made
Change-Id: I9b4bb27c9497b2a828616caf72eaf345b072c294
We use "DETECT_EXCEPTION" in three different places, it is difficult
to distinguish what is realy happening. We use different metrics values
to replace them. And we also add a new metrics value to log onRejected
when the trusted process is restarted.
Android Metrics Design Review : eldar/276723226
Bug: 207717787
Test: build pass
Change-Id: Ib631bac3e5d6663c7eb64f54516b86b7fd9b8144
Because the client callbacks from SoundTriggerMiddleware service are
async, if a client quickly stops and starts a model, then receives a
detection event for that model, it is impossible for it to tell
whether the event corresponding to the previous or current session,
and thus is unable to reason about the resulting state of the model.
To mitigate that, we will always wait for an abort event when a model
is stopped before any other operation is attempted. This way the
model state stays synchronized.
Bug: 191935600
Fixes: 226926627
Test: Manual verification of soundtrigger use-cases. Specifically,
quick toggling of Assitant and Now Playing via Settings.
Change-Id: Ia1b91e78aa6981a7b3612ae9213af1b37a333d00
This reverts commit 6e2eb81ac4.
The original commit fixed b/223922855, which resulted from fixing
b/191935600, but created another.
Blocking the stop() call until the event confirming the stop arrives is
a bad idea, since:
- We have to give up the lock when waiting for the event, and thus lose
the atomicity of some of the operations. b/226926627 is an example of
this.
- It turns out some of the forward calls happen from the main thread's
looper, which is also used for callbacks, and we risk deadlocks.
A follow-up commit will address the fix in a better way.
Bug: 191935600
Bug: 226926627
Bug: 223922855
Test: See next commit in the chain.
Change-Id: I2b9987ae93803d3b4f72a1df56c88e284bed43c5
If the app starts a new Activity in the same task then the
finishTranslation() is called, the finish translation doesn't work
because our logic is to check if task top Activity is our expected
translation Activity. In this case, the top Activity is the new
Activity, the original Activity is paused in the same task. To fix
the problem, we get all non-finishing Activities in the task and
check if the target Activity in this task.
Bug: 202570916
Test: atest UiTranslationManagerTest
Test: atest CtsVoiceInteractionTestCases
Test: atest CtsAssistTestCases
Change-Id: I2471e04b894890777e13dd45c290f059dbe5f487
We use a flag to control the restart period time, the value is static
and set in class level that means if we update the device config, we
should reboot the device to get the updated value.
This change reads the value in the constructor, so the value will be
updated when the voice interaction app updates no need to reboot.
Ideally it's good to listen the value change dynamically but it
increases the complexity here, we provide a workable and safe change
now.
Bug: 224618257
Test: update the package and the value is expected by command:
adb shell dumpsys voiceinteraction | grep RESTART_PERIOD_SECONDS
Change-Id: Id7d5f2d4f17de1f36baf6509598a82d6d73c5b53
On native audioserver recovery the assistant UIDs will reset to an empty
list, followed by adding the current role or setting holder for
assistant. Also added a settings lock around the update assistant
method. Added remove assistant UID for reset hotword detection
connection.
Bug: 222584963
Bug: 222777782
Test: atest VoiceInteractionTest
Test: atest AudioManagerTest
Test: atest HotwordDetectionServiceBasicTest
Test: kill audio server and verify assistant UIDs is not increasing
unbounded.
Test: adb shell cmd voiceinteraction restart-detection, verify assistant
UIDs list is not increasing unbounded.
Change-Id: I475bfee1f0bcd56a750e4fef8aa8a2b93fe090fc
The modelData callback may be nullified before actually being invoked,
e.g. when calling startRecognition() on an already running model, with
a different callback.
Fixes: 225768984
Test: Manual reproduction on the issue and then applying the fix and
verifying that it works.
Change-Id: I089061123c8cf12eef19fdf16bb3a2c106ea2438
Because the client callbacks from SoundTriggerMiddleware service are
async, if a client quickly stops and starts a model, then receives a
detection event for that model, it is impossible for it to tell
whether the event corresponding to the previous or current session,
and thus is unable to reason about the resulting state of the model.
To mitigate that, we will always wait for an abort event when a model
is stopped before any other operation is attempted. This way the
model state stays synchronized.
Fixes: 191935600
Test: Manual verification of soundtrigger use-cases. Specifically,
quick toggling of Assitant and Now Playing via Settings.
Change-Id: I010558e8fa1891922f20dcdb7b0ca04c9500c2e1
Bug: 224618257
Test: adb shell cmd device_config get/put voice_interaction \
restart_period_in_seconds works fine
Test: the value is expected by command: adb shell dumpsys \
voiceinteraction | grep RESTART_PERIOD_SECONDS
Change-Id: Icd8d19c2a482d0d6b9bf0432e2176ca10f3bd2eb
Mitigates quality risk that may arise due to mismanaged state around the
restarts (either on the platform or application side).
Restarts will be flag-controlled and ramped back up later.
Bug: 224618257
Test: manual
Change-Id: I3b17b47fab4120b42d75443b213be3c1ef5491aa
Bug: 207717787
Test: build & boot pass.
Test: manual. Both trust and no-trust still work.
Test: manual. In nornal case, log with expected values, see bug for
details. Log expected value for some error cases (not all are tested)
by local changes.
Change-Id: Iaa778616eca4cfad83a94297d9cd6116bb9577e7
This change only provide a utilility class to help hotword dection
metrics logging. The log writing will on the follow up changes.
Bug: 207717787
Test: build pass can boot success
Change-Id: If436aae1b4ebdb06c7bd5a1c0b8d6df4e94488cd
Adds a new API to VoiceInteractor to allow apps to query
the package name of the Voice Interactor associated with
this the current voice interaction
BUG: 192061561
CTS-Coverage-Bug: 218687104
Test: Manual validation from personal and work profiles
Change-Id: If381c38498535bf94d13c0b5ad5f6c4fa5495117
Added API to set the UIDs of the services that can be used as assistants.
Also modified code that uses the previous APIs for setting the unique
assistant UID and its corresponding hotword service. The new API can set
multiple UIDs all which will be treated the same for the purposes of
proviving assistant services.
Bug: 189312611
Test: m -j, and run assistant
Change-Id: Ieb28b7141206b58ae1d5d68580c08f4c733560b6
If a voice interaction session is started, then it should be allowed to
use the network & wakelocks unrestricted while it is ongoing, since it
is direct user interaction.
Bug: 190822356
Test: atest LowPowerStandbyControllerTest PowerManagerServiceTest
Test: With reference VIS: adb shell cmd voiceinteraction show
To verify allowlist/wakelock state: adb shell dumpsys power
Change-Id: I63cd0cf2e97b574ba8fa6c6caa8f046cce2bb821
Currently we are adding the metrics to find why there is a significant
drop in assistant successful queries for using trusted hotword. In the
metrics, we need to know which detector is used. In this patch, we
first provide the way to distinguish which detector triggered the
HotwordDetectionService.
Bug: 215633663
Test: atest CtsVoiceInteractionTestCases
Test: atest CtsVoiceInteractionTestCases --instant
Change-Id: I2fa904c5a52d714c9ecb545084ffb60e81ed1d8a
The VoiceInteractionManagerServiceImpl receiver that listens for
CLOSE_SYSTEM_DIALOGS broadcasts was initially marked as not exported
since this broadcast should be dropped by the platform when sent from
apps. However since this broadcast is still valid to be used in tests,
receivers registering for this broadcast should be marked as exported.
This commit updates the receiver to exported to allow any tests that
send this broadcast to complete successfully.
Bug: 161145287
Test: Build
Change-Id: I2238b2e2d3c3b6ecec155b628ac59e8db6e863b6
VoiceInteractionService is bound with the BIND_FOREGROUND_SERVICE flag.
Do the same for the HotwordDetectionService now.
Test: manual
Fix: 206812592
Change-Id: Iddd992fe475976ac68b91580c67525f56d534e32
This change allows the HAL driver, at its discretion, indicate that
recognition is still active after a success event.
This is achieved by an additional flag added to the event.
The behavior to support this case has already been in place, for the
sake of supporting a FORCED event. This change just generalizes this
behavior to be able to cover SUCCESS as well.
For b/w compat, when the status is FORCED, we override the new flag
with 'true', indicating that recognition is still active.
We do not allow the flag to be set for status codes other than FORCED
or SUCCESS.
Test: atest FrameworksServicesTests:{SoundTriggerMiddlewareImplTest,SoundHw2CompatTest}
Test: Manual verification of sound trigger operation by invoking the
assistant and now playing multiple times.
Bug: 186031938
Change-Id: Ie4edf82607c72ccb0b8d90a828b04c93153ec8f3
To enable more voice-oriented in-app user journeys, Google Assistant
needs to access more information regarding the visible activities
via the Voice Interaction Session. Thus, we provide the APIs to receive
the changed visible activities in Voice Interaction Session.
Bug: 178244510
Test: atest CtsVoiceInteractionTestCases
Test: atest CtsVoiceInteractionTestCases --instant
Change-Id: I019d09fee8105ae1eadbc76803f46fd8b1948f6b
