Commit Graph

96 Commits

Author SHA1 Message Date
Songchun Fan
1b76ccfa7c [incremental] expose duration since oldest pending read
As requested by go/incremental-disablement-metrics, we will expose the
duration since oldest pending read as part of the crash/ANR metrics.
This is the first step that exposes the value to Incremental Service.

BUG: 180951530
Test: unit test
Change-Id: Ic67460072556ef01780a1794b40924ca2092060d
2021-03-09 23:06:57 +00:00
Yurii Zubrytskyi
3fde572afc Fix the progress getting for mapped files
Mapped files don't support querying their loading progress,
so we should simply skip them - they are already a part of
some other file, and will get accounted for loading when
that file's progress get queried

+ a bunch of small improvements

Bug: 180535478
Test: atest service.incremental_test, adb install --incremental
  with mapped native libs

Change-Id: Ifc8a402144f2f3669a0419124fb0f35d7002190a
(cherry picked from commit 7731ebd1d8)
2021-02-22 05:15:50 +00:00
Yurii Zubrytskyi
a5946f7056 [incremental] Use file range mapping for .so if available
Bug: 180535478
Test: IncrementalService unit tests
Change-Id: I663dcdce337c289cacc5dc7224dedf5a55605c86
2021-02-22 04:05:11 +00:00
Alex Buynytskyy
060c9d6869 Potential deadlock mitigation.
+better error logging

DataLoaders might report user statuses from lifecycle callbacks.
Immediate processing of such might introduce infinite loops/deadlocks e.g.
DataLoader_OnStop -> reportStatus(UNRECOVERABLE) -> fsmStep -> DataLoader_OnStop

Bug: 160634487
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: Ic68657d7a8cd6c6855b6f5295276a42b3cb09117
2021-02-20 06:21:19 +00:00
Alex Buynytskyy
b19ee3ebd8 DataLoader lifecycle.
- restarting DataLoaders for not fully downloaded mounts,
- exponential backoff retry on DataLoader restart.

Bug: 173223115
Bug: 160634487
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: I5a93fd515ef0fec09452ce7bb4cd79fbd283d45f
2021-02-11 08:47:27 +00:00
Alex Buynytskyy
2b2f5f7c5f Per UID DataLoader API.
Bug: 160634487
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest

Change-Id: Id423d838ac7950696b5f75bec2d1610fbc929210
2021-02-02 17:25:04 +00:00
Alex Buynytskyy
07694ed49b Inherited installation support for Incremental.
Bug: 162345970
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: I360f44bc52e05553eacc448faa26f603d9eaae59
Merged-In: I360f44bc52e05553eacc448faa26f603d9eaae59
2021-01-29 11:16:02 -08:00
Yurii Zubrytskyi
5f69292dcf Incfs: Add the new .blocks_written control file
Bug: 170231230
Test: incremental and PackageManager unit tests
Change-Id: Ia26306a4b1ca70ce70d55ce807fea62540f89334
2021-01-14 22:01:47 -08:00
Alex Buynytskyy
42d4ba418e Require INTERACT_ACROSS_USERS to enable readlogs.
As requested by PWG.

Bug: 177330562
Fixes: 177330562
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest
Change-Id: I1b03bc1c5b4c3cffaa0f97d864cc088782290bc2
2021-01-12 12:35:53 -08:00
Alex Buynytskyy
aa8e95ea33 Per package read timeouts.
Bug: 162345970
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest

Change-Id: I2599db1ed8827fff16387c11254a5d607f27ea46
2021-01-12 05:19:38 +00:00
TreeHugger Robot
af764c805b Merge "incfs: Create incomplete folder" 2020-11-21 01:09:21 +00:00
Paul Lawrence
87a92e1bc5 incfs: Create incomplete folder
incfs 2 also creates a .incomplete folder, so we need to set correct
permissions on that folder too

Bug: 169084168
Test: With updated kernel, incfs_test works and adb install
	--incremental works
Change-Id: Id7b9d017973777e37628664530dfed18c55c269f
2020-11-20 13:18:01 -08:00
Songchun Fan
6944f1e484 [incremental] add last pending reads info in dumpsys
Example section of dumpsys output:
        lastPendingReads:
          fileId: 03300000000000000000000000000000
          metadataHex: 0330
          blockIndex: 857738
          bootClockTsUs: 166877488

Test: manual
BUG: 162600251
Change-Id: I37b04751c2a11972450ce7e9082116f8047e87df
2020-11-09 22:07:54 +00:00
Songchun Fan
2570ec01c6 [incremental/pm] set health listener on commit and on reboot
This changes allow Incremental Service to directly report health status
to package manager service.

A health listener is created during package installation session to
monitor incremental storage health. After commit, a new listener is
created and will overwrite the old one.

The new listener will listen to incremental storage health and report
the status to package manager service, which will then send the status
to IncrementalStates, where the startability state and unstartable
reason might change, based on the health status code.

During reboot, for each incremental package, if it is not fully loaded,
the package manager service will register a health status listener to
continue monitor the health status of this package.

Test: unit test
Test: manual
BUG: 170435166
Change-Id: I220f230c523cfaf2c96019f9478554665e6af486
2020-10-20 01:13:07 +00:00
Songchun Fan
3309398cbf [incremental/pm] app states and transitions
Based on go/incremental-states-design with basic
setter/getters.

Defines IncrementalStates class which handles state transitions.

New (internal) Intent actions: PACKAGE_FULLY_LOADED, PACKAGE_STARTABLE,
PACKAGE_UNSTARTABLE.

BUG: 168043976
Test: unit tests
Change-Id: I7b0ec2dd9f028ee620a9307a1e71ddf12ea5a9af
2020-10-01 16:36:25 -07:00
Songchun Fan
2ff2a488e6 [incremental] fix x86 native lib extraction
Test: atest CtsIncrementalInstallHostTestCases
BUG: 168138334
Change-Id: I7b76afbd88cf3f38c1527c9c917ce179e0bde589
2020-09-29 18:53:18 +00:00
Alex Buynytskyy
b65a77f7b5 Using wake to forcefully wake up the looper instead of 1s timeout.
Bug: 168156055
Test: atest ApkVerityTest:com.android.apkverity.ApkVerityTest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest ChecksumsTest
Change-Id: If65f65f9214b34bf057ed5f1877348d835251d14
2020-09-22 11:45:21 -07:00
Alex Buynytskyy
b39d13ed7f Installer digests for Incremental installations.
Bug: 160605420
Test: atest ChecksumsTest
Change-Id: I9d46c218cccf87781e9b33711c4d02d94bf824f5
2020-09-21 12:40:56 -07:00
Songchun Fan
a70985945b [incremental/pm] register progress listener
Incremental Serivce periodically polls loading progress and sends to
Package Manager Service. Package Manager provides APIs for other
interested parties to listen to the loading progress.

BUG: 165841827
Test: unit test
Change-Id: I44b9e17c2240b9efe53bc09fc728b6671f1f7dfe
2020-09-15 17:39:41 -07:00
Alex Buynytskyy
bc0a7e6cbf Wait for APK to be fully downloaded for full APK digests.
Bug: 160605420
Test: atest ChecksumsTest
Change-Id: Ib9fd591c67290786268b6dcdc57c7db153612e01
2020-08-27 10:33:46 -07:00
Songchun Fan
425862f3e7 [incremental] storage is regarded as fully loaded if there's no file or only empty files
Test: atest service.incremental_test
BUG: 165799231
Change-Id: I3b6be3918d53ce4ab0a30a4da493d9730a31ed3c
2020-08-25 13:17:28 -07:00
Songchun Fan
374f7654df [IncrementalService] getLoadingProgress (v1)
This is to unblock Launcher's work on progress ring. Currently it uses
incfs getFilledBlocks(). Will switch to the new incfs progress reporting
API once it is ready.

Test: unit test
Test: adb shell dumpsys incremental
BUG: 165799231
Change-Id: Icd68124806454f888826294da36f109bca9771ac
2020-08-22 04:17:02 +00:00
Jooyung Han
16bac8513e Switch aidl::nullable to std::optional
AIDL's @nullable is represented as std::optional in C++ backend.
The previous aidl::nullable was a temporary solution to avoid
merge-conflicts between rvc-dev and master branches.

Bug: 151275535
Test: m
Change-Id: I8fe5bfc1763087e7ff364b4408f862436492d1db
2020-08-10 08:36:35 +00:00
Songchun Fan
c8975312c5 [IncrementalService] fix native lib check after OTA
The lib dir should always be at the same dir level as the APK file.

Test: manual with OTA
Test: atest CtsExtractNativeLibsHostTestCases
BUG: 160826412
Change-Id: I87d715c4739c78bb56b85d375838db08bafa1594
2020-07-13 12:14:37 -07:00
Alex Buynytskyy
0403545508 Don't provide read logs for shell-initiated installations.
Only if the application is profileable.

Bug: 158238023
Fixes: 158238023
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageParserTest
Change-Id: I8575830ec3f29850297fdbfbaa157072d6350a28
Merged-In: I8575830ec3f29850297fdbfbaa157072d6350a28
2020-06-12 13:30:45 -07:00
Alex Buynytskyy
46d3ddb311 Healthcheck: proper job allocation and test.
Bug: 153874006
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest

Change-Id: Iede1f2297cc4f8e3c3f0acd43cee597f75dff179
2020-06-01 00:25:23 +00:00
Alex Buynytskyy
4760d8ff75 Lifecycle: detecting blocked and unhealthy, part 2.
Part 2: continuous health checking, blocked and unhealthy states.

Bug: 153874006
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: Ifdbff7dfa24b3bd9b96c534a40bb1226082749ca
2020-05-29 10:32:59 -07:00
Songchun Fan
f2a6af4b8a Merge "[incremental] respect extractNativeLibs in native lib config" into rvc-dev 2020-05-27 22:17:23 +00:00
Songchun Fan
14f6c3c735 [incremental] respect extractNativeLibs in native lib config
Makes sure the behavior is consistent with legacy installs:

When the flag is on, the native libs will be extracted to subdirs under
lib/.
When the flag is off, the lib/ subdirs will be created but the native
libs are not extracted.
When the flag is off, check if the native libs are uncompressed and well
aligned.

Test: atest android.extractnativelibs.cts.CtsExtractNativeLibsHostTest
BUG: 157173358
Change-Id: Idb57fd7ca1115f787faf5cde3056c32ff3f60890
2020-05-27 22:16:57 +00:00
Alex Buynytskyy
8ef61aebee Lifecycle: detecting blocked and unhealthy.
Part 1: interfaces and PM implementation.

Bug: 153874006
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: I312dd919d2bb552bea3d72fb49fd1579882da14b
2020-05-26 19:34:39 +00:00
Alex Buynytskyy
ea96c1f624 Don't restart non-system DataLoaders on reboot.
This saves system's resources.
Healthcheck will automatically start data loader on pending reads.

Bug: 153874006
Test: test PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: I160153f1d3e7e653db5de06b59d6fbbc3667fdf2
2020-05-18 11:40:43 -07:00
Alex Buynytskyy
4dbc0607ee Don't try to recreate IncrementalFileStorages on re-commit.
Bug: 156287164
Fixes 156287164
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest

Change-Id: I75e392a1fa84df8b6ac0b855233f9a2662998e96
2020-05-12 13:49:29 -07:00
Alex Buynytskyy
d0855a3ad8 Addressing CL comments.
Bug: 153874006
Test: test PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: Ie8b86cdc382955840cc3718d79a45991cf89b879
2020-05-08 10:35:58 -07:00
Alex Buynytskyy
cca2c1110c Lifecycle: detecting pending reads.
Once pending read detected, try to start the dataloader.

Bug: 153874006
Test: test PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest

Change-Id: Ia8169ccbb0f710317715e6fddb9bc6a718543766
2020-05-07 21:22:14 +00:00
Alex Buynytskyy
b0ea448eeb Additional lock to avoid access to deleted object.
Bug: b/155692497
Fixes: 155692497
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: Ie02012bd5a6c0640d54c5390d3726a5405042408
2020-05-04 18:41:47 -07:00
Alex Buynytskyy
7e0a1a8ec7 Unavailable DataLoader status.
This is a temporary failure, does not fail the session, but requires
caller to re-commit. E.g. there are connectivity issues which can be
fixed later.

Bug: b/153874006
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: I02791a2963130dbecb510c4a7cafcf04f6245761
2020-04-29 04:23:49 +00:00
Alex Buynytskyy
64067b25b1 Fix for Binder one-way deadlock.
Dispatch session destroy to be executed later.
We can't do anything blocking during one-way transaction processing.
This blocks all one-way transactions to this Binder, including other
status updates.

Bug: b/153874006
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Test: atest IncrementalInstallerTest#testIncrementalInstallFails_onPrepareImageReturnsFalse
Change-Id: Ie4385d603fca8b5205775e849e92846cff672f56
2020-04-27 11:06:29 -07:00
Alex Buynytskyy
0bdbccf37c Corner cases handling.
- crash on disappearing dataloaders,
- more robust binder callbacks processing,
- heavy unbind lifting moved to a separate thread to unblock Binder.

Bug: b/153874006
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: Ie06c3c4dbecbdd552dd868e2092bf0ee48f8547a
2020-04-23 22:28:36 -07:00
Alex Buynytskyy
ea1390ff32 Keep only service binding in DataLoaderManager.
This simplifies:
- resource management - no extra copies of controls,
- state management - all states in one place, no more hidden (bound but
not created) state.

Bug: b/153874006
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: I3d16a099c7f42fcf14637c5a8e96bd6f99e073d1
2020-04-23 19:38:44 +00:00
Yurii Zubrytskyi
510037b064 [cleanup] Enable clang-tidy and fix warnings
Moving out of a const variable doesn't really work, apparently

Bug: 153704006
Test: builds & boots
Change-Id: Id7322f7045c08a20d72dda50808c0be427064cdd
2020-04-22 22:34:24 -07:00
Yurii Zubrytskyi
efebb45ad7 [incfs] Use the new IncFs_MakeDirs() function
This gets rid of annoying warnings in logcat about not being
able to create a directory and then directory already existing

Bug: 153704006
Test: atest IncrementalServiceTest
Test: adb install megacity.apk
Change-Id: Ib718960287f93cb383c06c9b9e3d0abf1ec42916
2020-04-22 14:00:55 -07:00
Yurii Zubrytskyi
629051fd65 [incfs] Use MountRegistry to import existing mounts on start
This is a big cleanup in IncrementalService that makes it behave
nicely on runtime restart, and more:

- fixed a bunch of threading issues in createStorage/bind
- made public functions correctly accept any path in any
  bind mount and translate it to the proper root mount
- got rid of "using namespace" in headers, cleaned includes
- removed all unused functions
- set CLOEXEC bit on all duped FDs

Bug: 151241369
Test: atest PackageManagerShellCommandTest \
  PackageManagerShellCommandIncrementalTest \
  IncrementalServiceTest

Change-Id: Ided4415aabfbfca3925b5e71c91896055886ac4a
2020-04-22 00:49:02 -07:00
Alex Buynytskyy
c2a645d717 Unblock the unmount on uninstallation.
Bug: b/153874006
Test: test PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: I540ed36b87c40b8b58f0de727936c3efeb221198
2020-04-20 14:11:55 -07:00
Alex Buynytskyy
9a54579ac5 Cleaning up resources on mount destruction.
DataLoaderStub's lifetime is controlled externally, but we want to
release resources ASAP.

Bug: b/153874006
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: I34035f36d1fe4ed0e4916014d859feb7fe2c0a09
2020-04-17 22:07:10 -07:00
Alex Buynytskyy
ab65cb1824 Switching to FSM-based DL lifecycle.
This is a pure refactoring.

Bug: b/153874006
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: Ieda2be08d7359fa69b2d328c85b3606de6d02b3d
2020-04-17 11:02:16 -07:00
TreeHugger Robot
3e188ae81d Merge "[incfs] Fix the block size calculation in .so extraction" into rvc-dev 2020-04-15 00:52:43 +00:00
Yurii Zubrytskyi
6c65a56d39 [incfs] Fix the block size calculation in .so extraction
Bug: 154044009
Test: adb install megacity.apk && adb shell cat *.so
Change-Id: I0069c8cfa74c83810152d917a2bac9cdaf24d6a7
2020-04-14 15:25:49 -07:00
Alex Buynytskyy
0b20266c12 DataLoader lifecycle.
- mark disconnected DataLoaders as non-existent (destroyed),
- remove storage destroy on DataLoader destroy,
- more robust destroy on destruct handling,
- less race-conditions.

Bug: b/153874006
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: Ib77302aac546d66ce40c5345417c7b424a1d601b
2020-04-14 09:25:37 -07:00
Yurii Zubrytskyi
721ac4d350 [incfs] Correctly wait for async .so's extraction
StorageID for an installation changes as we go from a staging
directory to the final one. That's why the only correct way
of waiting for the extraction to complete is by the whole
MountID, even if later it would cause the call to wait on all
instances of the app.

+ measure the waiting timing

Bug: 153513507
Test: adb install megacity.apk
Change-Id: I83558f155867ae5503719ecb63d591fc969c3995
2020-04-13 11:34:32 -07:00
Alex Buynytskyy
6994166a09 Reconnect to DataLoaders on restart.
Bug: b/151241369
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest
Change-Id: I1532df63a2a36104dbd9cd90ee4e5f40bae00a6a
2020-04-11 21:40:37 -07:00