Commit Graph

124 Commits

Author SHA1 Message Date
Alex Buynytskyy
e76e1ef510 Avoid sysfs name clashes.
Bug: 187437944
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest ApkVerityInstallTest
Change-Id: Idc8cb4d170b0bfd98ee3a10bdf57439801739757
2021-05-08 14:10:33 +00:00
Songchun Fan
e749b4b2c4 Merge "[incremental] report new metrics from Incremental Service" into sc-dev 2021-05-05 00:09:59 +00:00
Songchun Fan
0dc7772746 [incremental] report new metrics from Incremental Service
BUG: 184844615
Test: atest service.incremental_test
Change-Id: Ibbfe68851a4adfd620db9652f1fda550a3c6fb2c
2021-05-04 15:14:34 -07:00
Songchun Fan
30253adb74 Merge "[IncrementalService] add last read error in dumpsys" into sc-dev 2021-05-04 04:56:04 +00:00
Songchun Fan
7a76d7f713 Merge "[IncrementalService] use sysfs_name in mount options and dumpsys" into sc-dev 2021-05-04 00:14:40 +00:00
Songchun Fan
d48a25ee64 [IncrementalService] add last read error in dumpsys
BUG: 184844615
Test: dumpsys
Change-Id: I3e8178c8a62dc5d09c718b140bb48bda85a3bae5
2021-05-03 14:58:18 -07:00
Yurii Zubrytskyi
878714ab82 Add more info to IncrementalService::onDump()
+ suppress a couple lint warnings

Bug: 184844615
Test: manual + check the dump output
Change-Id: Ic1cf2a2fb519cc9e746c73816b1cb55f14d11279
2021-04-30 16:09:52 -07:00
Songchun Fan
f949c37e14 [IncrementalService] use sysfs_name in mount options and dumpsys
BUG: 184844615
Test: adb shell dumpsys incremental
Change-Id: Ib6bbbede6d4fbed0bcef8d37f377e706b9d1af28
2021-04-30 12:19:58 -07:00
Songchun Fan
03c3c1bbad Merge "[incremental] expose more metrics fields" into sc-dev 2021-04-27 00:34:08 +00:00
Songchun Fan
9471be56b4 [incremental] expose more metrics fields
Exposing more data loader states as per
go/incremental-crash-reports-1-pager.

BUG: 184844615
Test: atest service.incremental_test
Test: atest android.cts.statsdatom.incremental.AppErrorAtomTests
Change-Id: I532513453411b2ccdb21311d0bc3dee0641837db
2021-04-26 14:17:36 -07:00
Alex Buynytskyy
de4b8230ab Retry on unavailable.
Bug: 182214420
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: Iaf61b6825ced45ffdc7e9c87dfea830e50633476
2021-04-26 11:21:01 -07:00
Songchun Fan
5ada1508c2 [pm] remove incremental startable/unstartable code
Also remove streaming health status reporting which could cause
startable state change because it is also not needed any more.

BUG: 171920377
Test: builds
Change-Id: I7284e7a63df79da7dbf3d16ff64302b3d1ce1348
2021-04-16 02:29:24 +00:00
Alex Buynytskyy
f2af4d8806 Add storage state and installation ts to dump.
Bug: 160635296
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: I524a92aa8c6a7b6961722dc3004a4bb323232297
2021-04-08 18:56:00 -07:00
Alex Buynytskyy
c144cc453a Installation hardening: reducing read timeout during installation.
And then restoring post-install.

Bug: 160635296
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest

Change-Id: I0821458bf92db162518a2cbcb7499cd7544e64f2
2021-04-08 09:57:54 +00:00
Alex Buynytskyy
5ac5553879 More logging and better retry logic.
Bug: 183551663
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: I1b6da82cd4472cc5671b0b60fb12222b4ed08e17
2021-03-25 13:23:33 -07:00
Yurii Zubrytskyi
4cd2492576 [incfs] Space trimming for IncFS v1
Run a manual timed job that trims all files one by one on the
old version of IncFS, where it didn't do it automatically.

Bug: 183435580
Fixes: 183436717
Test: atest libincfs-test service.incremental_test
Change-Id: I57885b2826e383814822c767802f837135fd8464
2021-03-24 15:05:17 +00:00
Yurii Zubrytskyi
9acc9acea5 [incfs] Fix the mount state callbacks processing
- use a never-existing storage ID as a job key
- order the jobs in the map to not skip them on changes, or,
  worse, never hang in a loop
- clear the local callbacks vector before moving to the next
  storage ID
- try to resume from the closest place on the next processing
  iteration

Bug: 183435580
Test: atest service.incremental_test
Change-Id: I36cd5d30c656bed62c20bd7a7f84fb58046a0933
2021-03-24 12:22:39 +00:00
Alex Buynytskyy
50d83ff1b3 Reduce ifs lock scope.
Bug: 182214420
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: Ic36002205c93b316b28ba10ea8f5fbc50dcff70a
2021-03-23 22:50:15 -07:00
Alex Buynytskyy
bcb2fe0cbb Add properties to test readlogs timeout.
Bug: 182477087
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: I02bc11b11f9158c31a80b845adb0703a27459332
2021-03-23 16:50:26 -07:00
Alex Buynytskyy
7b3e06e1c2 Add a first chance retry delay of 1s.
Bug: 182214420
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: Iade9e1355ed0e5e761a3a98f3a7bca8215982315
2021-03-23 12:59:22 -07:00
Alex Buynytskyy
cb163f9bb6 Unbind from DataLoader when not needed anymore.
+ simplify adding new callbacks on storage state
+ streamline lock story for ifs members

Bug: 183101753
Fixes: 183101753
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: I86fffa7101eeb42ebccca67ae7f5d133c1ab9dfa
2021-03-23 02:01:06 +00:00
Yurii Zubrytskyi
f4769e2f5b Untangle listeners mess in IncrementalService
Listeners and some binder call parameters were using several
different styles when passed around - copy, move, pointer,
pointer to pointer. This CL tries to 'normalize' that.

Bug: 183067554
Test: atest IncrementalServiceTest
Change-Id: Ia28089aa9e4491b0f28e3e747489199cfccb5a1b
2021-03-19 04:29:08 +00:00
Yurii Zubrytskyi
883a27a373 [incfs] Call the new isFullyLoaded api where possible
Replace the remaining calls to getFilledRanges() with
isFullyLoaded() where we don't care about the progress

Bug: 183067554
Test: atest IncrementalService

Change-Id: Ic8dc2e3a0ef078353883feef7969b29e11dfa2d0
2021-03-19 04:28:47 +00:00
Yurii Zubrytskyi
4375a74535 [incfs] Use a more efficient getBlockCount() for incfs v2
v2 IncFS driver gives a very lightweight function to check the
loading progress on a file, use it instead of counting the
filled ranges

+ remove the unused mockable toString(IncFsFileId)

Bug: 183067554
Test: atest IncrementalServiceTest
Change-Id: Icd3bd891d671b27654f4194787a15a00cba1eb80
2021-03-19 03:46:40 +00:00
Yurii Zubrytskyi
256a1a4504 [incfs] Use the new libincfs API for file status checking
libincfs got a new set of functions for checking the file loaded
status, which works more efficiently than getting filled ranges.

Bug: 183067554
Test: atest IncrementalServiceTest
Change-Id: I3b96bf409f1778c5a89e4802e2005197f70ce0cb
2021-03-18 14:24:21 -07:00
Yurii Zubrytskyi
65fc38a029 [incfs] Preallocate space for IncFS files
Use the new libincfs APIs to preallocate space for all files
created via the public makeFile() API. This way we ensure
the device won't run out of space much later

Bug: 182185202
Test: atest libincfs-test PackageManagerShellCommandTest \
 PackageManagerShellCommandIncrementalTest \
 IncrementalServiceTest
Change-Id: I70af97949b29ff5db63201b0e3487fe026e23160
2021-03-17 13:21:10 -07:00
Alex Buynytskyy
d7aa3464da Limit read log reporting to 2hrs for non-system DLs (non-adb).
Bug: 182477087
Test: atest IncrementalServiceTest
Change-Id: I98c9ed3a2e8a91d26bcb879ab7073903ff7bb2c5
2021-03-16 04:12:21 +00:00
Alex Buynytskyy
7e06d712d2 DL lifecycle: handle slow DL binding.
Bug: 182214420
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: I5959e01177ab702de1f754f4ba433004925ce98b
2021-03-11 16:35:35 -08:00
Songchun Fan
1b76ccfa7c [incremental] expose duration since oldest pending read
As requested by go/incremental-disablement-metrics, we will expose the
duration since oldest pending read as part of the crash/ANR metrics.
This is the first step that exposes the value to Incremental Service.

BUG: 180951530
Test: unit test
Change-Id: Ic67460072556ef01780a1794b40924ca2092060d
2021-03-09 23:06:57 +00:00
Yurii Zubrytskyi
3fde572afc Fix the progress getting for mapped files
Mapped files don't support querying their loading progress,
so we should simply skip them - they are already a part of
some other file, and will get accounted for loading when
that file's progress get queried

+ a bunch of small improvements

Bug: 180535478
Test: atest service.incremental_test, adb install --incremental
  with mapped native libs

Change-Id: Ifc8a402144f2f3669a0419124fb0f35d7002190a
(cherry picked from commit 7731ebd1d8)
2021-02-22 05:15:50 +00:00
Yurii Zubrytskyi
a5946f7056 [incremental] Use file range mapping for .so if available
Bug: 180535478
Test: IncrementalService unit tests
Change-Id: I663dcdce337c289cacc5dc7224dedf5a55605c86
2021-02-22 04:05:11 +00:00
Alex Buynytskyy
060c9d6869 Potential deadlock mitigation.
+better error logging

DataLoaders might report user statuses from lifecycle callbacks.
Immediate processing of such might introduce infinite loops/deadlocks e.g.
DataLoader_OnStop -> reportStatus(UNRECOVERABLE) -> fsmStep -> DataLoader_OnStop

Bug: 160634487
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: Ic68657d7a8cd6c6855b6f5295276a42b3cb09117
2021-02-20 06:21:19 +00:00
Alex Buynytskyy
b19ee3ebd8 DataLoader lifecycle.
- restarting DataLoaders for not fully downloaded mounts,
- exponential backoff retry on DataLoader restart.

Bug: 173223115
Bug: 160634487
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: I5a93fd515ef0fec09452ce7bb4cd79fbd283d45f
2021-02-11 08:47:27 +00:00
Alex Buynytskyy
2b2f5f7c5f Per UID DataLoader API.
Bug: 160634487
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest

Change-Id: Id423d838ac7950696b5f75bec2d1610fbc929210
2021-02-02 17:25:04 +00:00
Alex Buynytskyy
07694ed49b Inherited installation support for Incremental.
Bug: 162345970
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest ChecksumsTest
Change-Id: I360f44bc52e05553eacc448faa26f603d9eaae59
Merged-In: I360f44bc52e05553eacc448faa26f603d9eaae59
2021-01-29 11:16:02 -08:00
Yurii Zubrytskyi
5f69292dcf Incfs: Add the new .blocks_written control file
Bug: 170231230
Test: incremental and PackageManager unit tests
Change-Id: Ia26306a4b1ca70ce70d55ce807fea62540f89334
2021-01-14 22:01:47 -08:00
Alex Buynytskyy
42d4ba418e Require INTERACT_ACROSS_USERS to enable readlogs.
As requested by PWG.

Bug: 177330562
Fixes: 177330562
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest
Change-Id: I1b03bc1c5b4c3cffaa0f97d864cc088782290bc2
2021-01-12 12:35:53 -08:00
Alex Buynytskyy
aa8e95ea33 Per package read timeouts.
Bug: 162345970
Test: atest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest PackageManagerServiceTest

Change-Id: I2599db1ed8827fff16387c11254a5d607f27ea46
2021-01-12 05:19:38 +00:00
TreeHugger Robot
af764c805b Merge "incfs: Create incomplete folder" 2020-11-21 01:09:21 +00:00
Paul Lawrence
87a92e1bc5 incfs: Create incomplete folder
incfs 2 also creates a .incomplete folder, so we need to set correct
permissions on that folder too

Bug: 169084168
Test: With updated kernel, incfs_test works and adb install
	--incremental works
Change-Id: Id7b9d017973777e37628664530dfed18c55c269f
2020-11-20 13:18:01 -08:00
Songchun Fan
6944f1e484 [incremental] add last pending reads info in dumpsys
Example section of dumpsys output:
        lastPendingReads:
          fileId: 03300000000000000000000000000000
          metadataHex: 0330
          blockIndex: 857738
          bootClockTsUs: 166877488

Test: manual
BUG: 162600251
Change-Id: I37b04751c2a11972450ce7e9082116f8047e87df
2020-11-09 22:07:54 +00:00
Songchun Fan
2570ec01c6 [incremental/pm] set health listener on commit and on reboot
This changes allow Incremental Service to directly report health status
to package manager service.

A health listener is created during package installation session to
monitor incremental storage health. After commit, a new listener is
created and will overwrite the old one.

The new listener will listen to incremental storage health and report
the status to package manager service, which will then send the status
to IncrementalStates, where the startability state and unstartable
reason might change, based on the health status code.

During reboot, for each incremental package, if it is not fully loaded,
the package manager service will register a health status listener to
continue monitor the health status of this package.

Test: unit test
Test: manual
BUG: 170435166
Change-Id: I220f230c523cfaf2c96019f9478554665e6af486
2020-10-20 01:13:07 +00:00
Songchun Fan
3309398cbf [incremental/pm] app states and transitions
Based on go/incremental-states-design with basic
setter/getters.

Defines IncrementalStates class which handles state transitions.

New (internal) Intent actions: PACKAGE_FULLY_LOADED, PACKAGE_STARTABLE,
PACKAGE_UNSTARTABLE.

BUG: 168043976
Test: unit tests
Change-Id: I7b0ec2dd9f028ee620a9307a1e71ddf12ea5a9af
2020-10-01 16:36:25 -07:00
Songchun Fan
2ff2a488e6 [incremental] fix x86 native lib extraction
Test: atest CtsIncrementalInstallHostTestCases
BUG: 168138334
Change-Id: I7b76afbd88cf3f38c1527c9c917ce179e0bde589
2020-09-29 18:53:18 +00:00
Alex Buynytskyy
b65a77f7b5 Using wake to forcefully wake up the looper instead of 1s timeout.
Bug: 168156055
Test: atest ApkVerityTest:com.android.apkverity.ApkVerityTest PackageManagerShellCommandTest PackageManagerShellCommandIncrementalTest IncrementalServiceTest ChecksumsTest
Change-Id: If65f65f9214b34bf057ed5f1877348d835251d14
2020-09-22 11:45:21 -07:00
Alex Buynytskyy
b39d13ed7f Installer digests for Incremental installations.
Bug: 160605420
Test: atest ChecksumsTest
Change-Id: I9d46c218cccf87781e9b33711c4d02d94bf824f5
2020-09-21 12:40:56 -07:00
Songchun Fan
a70985945b [incremental/pm] register progress listener
Incremental Serivce periodically polls loading progress and sends to
Package Manager Service. Package Manager provides APIs for other
interested parties to listen to the loading progress.

BUG: 165841827
Test: unit test
Change-Id: I44b9e17c2240b9efe53bc09fc728b6671f1f7dfe
2020-09-15 17:39:41 -07:00
Alex Buynytskyy
bc0a7e6cbf Wait for APK to be fully downloaded for full APK digests.
Bug: 160605420
Test: atest ChecksumsTest
Change-Id: Ib9fd591c67290786268b6dcdc57c7db153612e01
2020-08-27 10:33:46 -07:00
Songchun Fan
425862f3e7 [incremental] storage is regarded as fully loaded if there's no file or only empty files
Test: atest service.incremental_test
BUG: 165799231
Change-Id: I3b6be3918d53ce4ab0a30a4da493d9730a31ed3c
2020-08-25 13:17:28 -07:00
Songchun Fan
374f7654df [IncrementalService] getLoadingProgress (v1)
This is to unblock Launcher's work on progress ring. Currently it uses
incfs getFilledBlocks(). Will switch to the new incfs progress reporting
API once it is ready.

Test: unit test
Test: adb shell dumpsys incremental
BUG: 165799231
Change-Id: Icd68124806454f888826294da36f109bca9771ac
2020-08-22 04:17:02 +00:00