Commit Graph

15 Commits

Author SHA1 Message Date
Ryan Mitchell
1a48fa659b Dependency injection of AssetProviders into ApkAssets
Creates ApkAssets creation methods that allow an AssetsProvider to be
specified.

During idmap verification and creation, idmap2 currently opens the
target package and overlay package several times:
1) When the crc of the package is calculated in idmap2 verify
2) When the manifest of an overlay is parsed
3) When an ApkAssets is opened.

Opening large zip files (like framework-res.apk) is slow. If we opened
the zip one time as an ApkAssets, the resources.arsc would have to be
parsed (which means mmaping/unmapping and touching a lot of
resources.arsc pages). This would cause idmap2 to preform unnecessary
work just to check the crc of some files.

This change allows a ZipAssetsProvider to be created and then moved
for the creation of an ApkAssets. The zip file only needs to be opened
once and the resources.arsc is not parsed until reading resources is
actually necessary.

Bug: 172471315
Test: libandroidfw_tests
Test: CtsResourcesLoaderTests
Change-Id: I940bb2c13844c7f028776a623a9ecef45a4813bf
2021-02-03 10:38:24 -08:00
Ryan Mitchell
db21f09a8e Revert^2 "libandroidfw hardening for IncFs"
55ef6167a2

Change-Id: I02d4890d181655dfd0a14c188468db512559d27b
2020-11-17 23:01:35 +00:00
Ryan Mitchell
55ef6167a2 Revert "libandroidfw hardening for IncFs"
Revert "Move map_ptr to incfs namspace"

Revert submission 12787270

Reason for revert: b/173250495
Reverted Changes:
I5cd1bc8a2:libandroidfw hardening for IncFs
Ice5dbcfb2:Move map_ptr to incfs namspace
I29ccdc8ed:Do not cache bag parent stack until requested
I1e9e9acaa:Cache resolved theme values

Change-Id: Ib90ef68339710086df41e9abe0833a542d03a74f
2020-11-13 23:55:20 +00:00
Ryan Mitchell
c75c2e0922 libandroidfw hardening for IncFs
Migrate libandroifw to using incfs::util::map_ptr to prevent processes
from crashing when parsing the resources.arsc, parsing compiled xml,
files, and retrieving resource values.

This change propagates incremental failures to the JNI level where they
are raised as ResourcesNotFoundException.

Performance of ResourcesPerfWorkloads without change (time in
nanoseconds):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.883s)
    youtube_ns_median: 93812805
    youtube_ns_standardDeviation: 4387062
    youtube_ns_mean: 94455597
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.265s)
    maps_ns_standardDeviation: 2997543
    maps_ns_mean: 83480371
    maps_ns_median: 82210941
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.963s)
    gmail_ns_median: 266141091
    gmail_ns_standardDeviation: 3492043
    gmail_ns_mean: 267472765

With change and verification forcibly enabled for all apks
(including the framework-res.apk):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.646s)
    youtube_ns_median: 101999396
    youtube_ns_standardDeviation: 4625782
    youtube_ns_mean: 102631770
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.286s)
    maps_ns_standardDeviation: 2692088
    maps_ns_mean: 91326538
    maps_ns_median: 90519884
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.694s)
    gmail_ns_median: 290284442
    gmail_ns_standardDeviation: 5764632
    gmail_ns_mean: 291660464

With change and verification disabled:
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.748s)
    youtube_ns_median: 95490747
    youtube_ns_standardDeviation: 7282249
    youtube_ns_mean: 98442515
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (10.862s)
    maps_ns_standardDeviation: 4484213
    maps_ns_mean: 87912988
    maps_ns_median: 86325549
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.034s)
    gmail_ns_median: 282175838
    gmail_ns_standardDeviation: 6560876
    gmail_ns_mean: 282869146

These tests were done on a Pixel 3 and with cpu settings configured by
libs/hwui/tests/scripts/prep_generic.sh:

 Locked CPUs 4,5,6,7 to 1459200 / 2803200 KHz
 Disabled CPUs 0,1,2,3

Bug: 160635104
Bug: 169423204
Test: boot device && atest ResourcesPerfWorkloads

Change-Id: I5cd1bc8a2257bffaba6ca4a1c96f4e6640106866
2020-11-12 08:13:44 -08:00
Ryan Mitchell
a90930528d Invalidate idmap when target updates
When the target package update, check if the idmap file must change.
If so, propagate the idmap changes to the targets overlay paths, and
invalidate cached overlay ApkAssets in ResourcesManager.

Bug: 147794117
Bug: 150877400
Test: OverlayRemountedTest
Test: libandroidfw_tests
Change-Id: I6115c30bae3672b188a5ff270720a0eea15b43b5
2020-04-07 12:14:51 -07:00
Ryan Mitchell
ef40d2e832 Refactor ApkAsset loading APIs
To add the partner requested ResourcesProvider#loadFromDir APIs, this
change adds format type integer that allows us to reduce the number of
ApkAssets loading overrides.

This change also adds hidden offset and length based ResourcesProvider
APIs that could not make R.

Bug: 142716192
Test: atest FrameworksResourceLoaderTests
Change-Id: I926fde257cae701901dcd4ca408024feae8c90a6
Merged-In: I926fde257cae701901dcd4ca408024feae8c90a6
2020-03-19 18:33:55 -07:00
Ryan Mitchell
8a891d86ab Allow for RRO internal referencing
This change allows RROs to reference their own internal resources as
expected.

Overlays are loaded as shared libraries so they can have their own
resource id space that does not conflict with the resource id space of
the target or other overlays.

References to overlay resources that override target resources now
appear as references to the target resources.

Overlay values that are inlined into the xml file specified using
android:overlayResources are now able to be used at runtime.

See go/rro-references for more information.

Bug: 135943783
Test: idmap2_tests
Test: libandroidfw_tests
Change-Id: Ie349c56d7fd3f7d94b7d595ed6d01dc6b59b6178
2019-10-17 13:09:33 -07:00
Mårten Kongstad
67d5c938e9 idmap: optimize time to create idmap data
Change idmap to iterate over the resources in the overlay package
instead of the target package when scanning for resources defined in
both packages. This cuts down the runtime cost of creating an idmap
considerably since the algorithm now scales with the number of resources
in the overlay package (a handful) and not the number of resources in
the target package (android: 10k, SystemUI: 8k) at a minor cost to code
complexity.

Improvements on the runtime of ResTable::createIdmap (systrace on an
emulator running aosp_x86_64-eng):

  - target=android: 12.5 ms -> 3.0 ms
  - target=SystemUI: 8.6 ms -> 1.0 ms

The bulk of the cost of creating an idmap from installd is now the fork
and execl to call "idmap --fd ..." which weigh in at 16 ms.

Bug: 80150169
Test: make libandroidfw_tests
Test: atest OverlayHostTests OverlayDeviceTests
Change-Id: I98e18d5958c0cd835a73055b714f5bf0f4f95a09
2018-11-21 06:48:19 -08:00
Ryan Mitchell
c2b0782c37 Fix flaky idmap test
The entry pool is encoded in UTF-8 not UTF-16. This would cause stringAt
to return a pointer to a cache that holds the UTF-8 string converted to
UTF-16. If the cache becomes invalidated, the UTF-16 pointer is no
longer valid and causes seg faults when looked up.

Test: libandroidfw_tests
Change-Id: I0f13dbeadd0d148de2805efd6cadb8e220368e3b
2018-11-07 10:28:56 -08:00
Adam Lesinski
970bd8d283 AssetManager2: Implement IDMAP support
This enables RRO (runtime resource overlays) with AssetManager2

Test: make libandroidfw_tests
Test: out/host/<platform>/nativetest64/libandroidfw_tests/libandroidfw_tests --testdata=frameworks/base/libs/androidfw/tests/data
Change-Id: Id8079104faefbfaa3f4017d8f7ee1a8968f151a2
2017-10-13 10:23:34 -07:00
Adam Lesinski
ed69ce84bd libandroidfw: Fix mass logspam of ResourceTypes warnings
An overlay was incorrectly leaking its own resources into the
framework resource package, which caused warnings for every app
that tried to access framework resources (all of them).

This change skips including any resources that are not overlaying
anything (not present in IDMAP).

Bug: 36256974
Test: make libandroidfw_tests
Change-Id: I8c710af6849bb848938825aacca02799ee96c003
2017-03-20 14:51:13 -07:00
Adam Lesinski
4c67a475a3 Make tests use APKs instead of exploded APKs
Tests would expect parts of the APK to be unzipped and
maintained. Instead, we now decompress the required files
from the test APKs on test setup. This simplifies
test maintenance substantially.

Test: make libandroidfw_tests && libandroidfw_tests --testdata=frameworks/base/libs/androidfw/tests/data
Change-Id: I3d2100af22df913e02401dedcf9842cdb32b2a3b
2016-12-05 19:03:47 -08:00
Adam Lesinski
ccf25c7bf6 Fix shared library bug in bag attributes
A ResTable_map entry has a name attribute, which
could be a dynamic reference if it comes from
a shared library. It was not being patched with
the correct package id.

Bug:16795890
Change-Id: Ia8df6a943269b2fefb2132c3ed74eb1997d7701b
2014-08-11 09:16:01 -07:00
Adam Lesinski
833f3ccbc8 AAPT support for feature splits
This change allows the developer to add a base package for
which to build a feature split. The generated resource types
will begin after the base APK's defined types so as not
to collide or override resources.

Multiple features can be generated by first choosing an
arbitrary order for the features. Then for each feature,
the base APK and any preceding features are specified
with the --feature-of flags.

So with a base APK 'A' and features, 'B', and 'C',
'B' would be built with

aapt package [...] --feature-of A [...]

and 'C' would be built with

aapt package [...] --feature-of A --feature-of B [...]

Change-Id: I1be66e3f8df9a737b21c71f8a93685376c7e6780
2014-08-04 18:48:14 -07:00
Adam Lesinski
f90f2f8dc3 Support multiple resource tables with same package
In order to support APK split features, the resource
table needs to support loading multiple resource
tables with the same package but potentially new set
of type IDs.

This adds some complexity as the type ID space changes
from dense and ordered to potentially sparse.

A ByteBucketArray is used to store the type IDs in
a memory efficient way that allows for fast retrieval.

In addition, the IDMAP format has changed. We no longer
need random access to the type data, since we store the
types differently. However, random access to entries of
a given type is still required.

Change-Id: If6f5be680b405b368941d9c1f2b5d2ddca964160
2014-06-18 19:20:08 +00:00