Commit Graph

17 Commits

Author SHA1 Message Date
Ryan Mitchell
2ed8bfa7fd Add fabricated RRO generation to libidmap2
Fabricated Runtime Resource Overlays are overlays that are generated
at runtime and are stored in the data/ partition.

The system can fabricate RROs at runtime to dynamically theme the
device. Idmaps can now be created from APK RROs and fabricated RROs.

Rather than operating on ApkAssets, libidmap2 now operates on abstract
resource "containers" that supply resource values. Target resource
containers implement methods needed to query overlayable and target
overlay information. Currently only APKs can be loaded as target
resource containers. Overlay resource containers implement methods to
supply the mapping of target resource to overlay value and other
overlay information.

The format of a fabricated RRO is as follows:
0x00 - 0x04 : fabricated overlay magic (always FRRO)
0x04 - 0x08 : file format version
0x08 - 0x0c : crc of version + proto data
0x0c - EOF  : proto fabricated overlay data

The magic is used to quickly detect if the file is a fabricated overlay.
The version is incremented whenever backwards incompatible changes are
made to the proto file format. Idmap must always be able to upgrade
fabricated overlays from previous versions to new versions, so all
previous versions must be checked into the tree.

Bug: 172471315
Test: libidmap2_tests && libandroidfw_tests
Change-Id: I4c9f29da278672e5695fb57d131a44c11a835180
2021-02-09 20:13:50 -08:00
Ryan Mitchell
1a48fa659b Dependency injection of AssetProviders into ApkAssets
Creates ApkAssets creation methods that allow an AssetsProvider to be
specified.

During idmap verification and creation, idmap2 currently opens the
target package and overlay package several times:
1) When the crc of the package is calculated in idmap2 verify
2) When the manifest of an overlay is parsed
3) When an ApkAssets is opened.

Opening large zip files (like framework-res.apk) is slow. If we opened
the zip one time as an ApkAssets, the resources.arsc would have to be
parsed (which means mmaping/unmapping and touching a lot of
resources.arsc pages). This would cause idmap2 to preform unnecessary
work just to check the crc of some files.

This change allows a ZipAssetsProvider to be created and then moved
for the creation of an ApkAssets. The zip file only needs to be opened
once and the resources.arsc is not parsed until reading resources is
actually necessary.

Bug: 172471315
Test: libandroidfw_tests
Test: CtsResourcesLoaderTests
Change-Id: I940bb2c13844c7f028776a623a9ecef45a4813bf
2021-02-03 10:38:24 -08:00
Ryan Mitchell
831b072936 Add idmap completely parsed assertion
If LoadedIdmap::Load finishes parsing the idmap file but has not
parsed the entire file, then it parsed the file incorrectly. Log
an error if the idmap fails to parse.

Bug: 174676094
Test: libandroidfw_tests
Change-Id: I8ae7ed525715ab907d5813975c1ef5d1260bbd33
2021-01-13 10:11:18 -08:00
Ryan Mitchell
30dc2e0171 Accept --overlay-name flag in idmap2
To support allowing for multiple <overlay> tags in one package, idmap2
must be able to generate an idmap for an individual <overlay> tag.

`idmap2 create` now accepts a --overlay-name flag that specifies which
tag to use to generate the idmap. The value of --overlay-name should be
set to the value of the android:name attribute on the <overlay> tag to
use.

If the flag is not present, idmap2 will look for an <overlay> tag with
no value for android:name.

Bug: 162841629
Test: libandroidfw_tests
Test: libidmap2_tests
Change-Id: I02316d0b88773f02c04a5d462be9825016fa496d
2021-01-06 16:19:53 -08:00
Ryan Mitchell
0699f1de6a Remove idmap path 256 length limit
Overlay and target package paths can be longer than 256 characters.
Currently, the idmap will fail to be generated if either path
is longer than 256 characters.

This change removes the 256 character limit and makes parsing variable
length strings easier in libandroidfw.

Bug: 174676094
Test: idmap2_tests && libandroidfw_tests
Change-Id: Ic240cdb8700566b2ac2ade08da58bea852e4ae0c
2021-01-06 16:19:50 -08:00
Ryan Mitchell
db21f09a8e Revert^2 "libandroidfw hardening for IncFs"
55ef6167a2

Change-Id: I02d4890d181655dfd0a14c188468db512559d27b
2020-11-17 23:01:35 +00:00
Ryan Mitchell
55ef6167a2 Revert "libandroidfw hardening for IncFs"
Revert "Move map_ptr to incfs namspace"

Revert submission 12787270

Reason for revert: b/173250495
Reverted Changes:
I5cd1bc8a2:libandroidfw hardening for IncFs
Ice5dbcfb2:Move map_ptr to incfs namspace
I29ccdc8ed:Do not cache bag parent stack until requested
I1e9e9acaa:Cache resolved theme values

Change-Id: Ib90ef68339710086df41e9abe0833a542d03a74f
2020-11-13 23:55:20 +00:00
Ryan Mitchell
c75c2e0922 libandroidfw hardening for IncFs
Migrate libandroifw to using incfs::util::map_ptr to prevent processes
from crashing when parsing the resources.arsc, parsing compiled xml,
files, and retrieving resource values.

This change propagates incremental failures to the JNI level where they
are raised as ResourcesNotFoundException.

Performance of ResourcesPerfWorkloads without change (time in
nanoseconds):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.883s)
    youtube_ns_median: 93812805
    youtube_ns_standardDeviation: 4387062
    youtube_ns_mean: 94455597
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.265s)
    maps_ns_standardDeviation: 2997543
    maps_ns_mean: 83480371
    maps_ns_median: 82210941
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.963s)
    gmail_ns_median: 266141091
    gmail_ns_standardDeviation: 3492043
    gmail_ns_mean: 267472765

With change and verification forcibly enabled for all apks
(including the framework-res.apk):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.646s)
    youtube_ns_median: 101999396
    youtube_ns_standardDeviation: 4625782
    youtube_ns_mean: 102631770
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.286s)
    maps_ns_standardDeviation: 2692088
    maps_ns_mean: 91326538
    maps_ns_median: 90519884
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.694s)
    gmail_ns_median: 290284442
    gmail_ns_standardDeviation: 5764632
    gmail_ns_mean: 291660464

With change and verification disabled:
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.748s)
    youtube_ns_median: 95490747
    youtube_ns_standardDeviation: 7282249
    youtube_ns_mean: 98442515
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (10.862s)
    maps_ns_standardDeviation: 4484213
    maps_ns_mean: 87912988
    maps_ns_median: 86325549
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.034s)
    gmail_ns_median: 282175838
    gmail_ns_standardDeviation: 6560876
    gmail_ns_mean: 282869146

These tests were done on a Pixel 3 and with cpu settings configured by
libs/hwui/tests/scripts/prep_generic.sh:

 Locked CPUs 4,5,6,7 to 1459200 / 2803200 KHz
 Disabled CPUs 0,1,2,3

Bug: 160635104
Bug: 169423204
Test: boot device && atest ResourcesPerfWorkloads

Change-Id: I5cd1bc8a2257bffaba6ca4a1c96f4e6640106866
2020-11-12 08:13:44 -08:00
Ryan Mitchell
bf1f45b071 Remove malloc/free for inline overlay values
Remove malloc/free of android::ResTable_entry for inline overlay
values.

Add `target_entry_inline` to the idmap format to encode inline overlay
values separate from direct mapping of target resource to overlay
resource. This reduces the number of bytes needed to represent a direct
mapping of target resource to overlay resource from 9 bytes to 8 bytes
per entry.

Fixed all idmap alignment issues that required the framework to use
"#pragma pack(push, 1)" when loading idmaps.

Bug: 170341022
Test: idmap2_tests and libandroidfw_tests
Change-Id: Iab4d3902508f02773464724913e0ee966e3689e4
2020-10-14 14:52:52 -07:00
Ryan Mitchell
c4142d91f5 libafw: Use std::free explicitly for overlays
Currently we rely on libc to call std::free on the malloced pointer to
the overlay table entry. Rather than relying on the libc implementation
to call std::free instead of trying to delete the pointer as if it was
created using "new", explicitly pass std::free as the deleter to the
data held by the std::shared_ptr.

Bug: 159562360
Test: Observed no heap growth while looping infinitely and retrieving
      a string resource overlaid using an inline xml value
Change-Id: I9a2ebaf9a993ad9c44fab8f052430c8142d4347d
2020-06-22 15:50:14 -07:00
Ryan Mitchell
a90930528d Invalidate idmap when target updates
When the target package update, check if the idmap file must change.
If so, propagate the idmap changes to the targets overlay paths, and
invalidate cached overlay ApkAssets in ResourcesManager.

Bug: 147794117
Bug: 150877400
Test: OverlayRemountedTest
Test: libandroidfw_tests
Change-Id: I6115c30bae3672b188a5ff270720a0eea15b43b5
2020-04-07 12:14:51 -07:00
TreeHugger Robot
5a9109789e Merge "Add tests for layout overlaying" 2019-12-20 21:40:41 +00:00
Mårten Kongstad
d7e8a534d0 idmap2: add debug information to idmap file format
Add a new variable length string to the idmap file format. This string will
hold debug information like fulfilled policies and any warnings triggered while
generating the file.

Bump the idmap version to 3.

Adjust the idmap header definition in ResourceType.h to take the new string
into account.

Example debug info:

  $ idmap2 create \
        --target-apk-path frameworks/base/cmds/idmap2/tests/data/target/target.apk \
        --overlay-apk-path frameworks/base/cmds/idmap2/tests/data/overlay/overlay.apk \
        --idmap-path /tmp/a.idmap \
        --policy public \
        --policy oem

  $ idmap2 dump --idmap-path /tmp/a.idmap
  target apk path  : frameworks/base/cmds/idmap2/tests/data/target/target.apk
  overlay apk path : frameworks/base/cmds/idmap2/tests/data/overlay/overlay.apk
  I fulfilled_policies=oem|public enforce_overlayable=true
  W failed to find resource "integer/not_in_target" in target resources
  0x7f010000 -> 0x7f010000 integer/int1
  0x7f02000c -> 0x7f020000 string/str1
  [...]

  $ idmap2 dump --idmap-path /tmp/a.idmap --verbose
  00000000: 504d4449  magic
  00000004: 00000003  version
  00000008: 76a20829  target crc
  0000000c: c054fb26  overlay crc
  00000010: ........  target path: frameworks/base/cmds/idmap2/tests/data/target/target.apk
  00000110: ........  overlay path: frameworks/base/cmds/idmap2/tests/data/overlay/overlay.apk
  00000210: ........  debug info: ...
  00000294:       7f  target package id
  00000295:       7f  overlay package id
  [...]

Also, tell cpplint to accept non-const references as function parameters:
they make more sense as out-parameters than pointers that are assumed to
be non-null.

Also, switch to regular expressions in the RawPrintVisitorTests: no more
manual fixups of the stream offsets! Tell cpplint that the <regex>
header is OK to use.

Bug: 140790707
Test: idmap2_tests
Change-Id: Ib94684a3b4001240321801e21af8e132fbcf6609
2019-12-19 16:51:51 +00:00
Ryan Mitchell
df9e732962 Add tests for layout overlaying
Add tests to ensure that findViewbyId retrieves the correct view when
an id is overlaid in a RRO.

Also fixes a bug that made strings hardcoded in the overlays.xml
file unable to be retrieved from the string pool.

Bug: 135943783
Test: atest OverlayDeviceTests
Change-Id: I2bf03f151cb696d28f6bb9018eb319af29ba48f4
2019-12-12 10:55:33 -08:00
Ryan Mitchell
73bfe41bab Do not look for R.class of overlays
Now that RROs are loaded as shared libraries,
LoadedApk#makeApplication is attempting to find the onResourcesLoaded
method of the overlays. This is a performance hit and causes more
memory than necessary to be allocated during application start up.

Bug: 143314947
Test: com.android.performance.tests.HermeticMemoryTest
Change-Id: I3b8cd22dae83e0164d6678c80279f9fffceb34e6
2019-11-13 14:51:14 -08:00
Ryan Mitchell
8a891d86ab Allow for RRO internal referencing
This change allows RROs to reference their own internal resources as
expected.

Overlays are loaded as shared libraries so they can have their own
resource id space that does not conflict with the resource id space of
the target or other overlays.

References to overlay resources that override target resources now
appear as references to the target resources.

Overlay values that are inlined into the xml file specified using
android:overlayResources are now able to be used at runtime.

See go/rro-references for more information.

Bug: 135943783
Test: idmap2_tests
Test: libandroidfw_tests
Change-Id: Ie349c56d7fd3f7d94b7d595ed6d01dc6b59b6178
2019-10-17 13:09:33 -07:00
Adam Lesinski
970bd8d283 AssetManager2: Implement IDMAP support
This enables RRO (runtime resource overlays) with AssetManager2

Test: make libandroidfw_tests
Test: out/host/<platform>/nativetest64/libandroidfw_tests/libandroidfw_tests --testdata=frameworks/base/libs/androidfw/tests/data
Change-Id: Id8079104faefbfaa3f4017d8f7ee1a8968f151a2
2017-10-13 10:23:34 -07:00