With the introduction of ResourcesProviders, not all ApkAssets have
paths on disk. Theme::SetTo and various AssetManager methods use the
path to perform equality checking on ApkAssets. This equality check
will be performed on the debug string of an ApkAssets if it does not
have a path on disk. This causes ApkAssets with the same debug name
(like "<empty>") to be seen as the same ApkAssets. Rather than using
path, the pointer to the ApkAssets should be used for equality checking
since ResourcesManager caches and reuses ApkAssets when multiple
AssetManagers request the same assets.
Bug: 177101983
Test: atest CtsResourcesLoaderTests
Change-Id: I11f6a2a3a7cc8febe3f976236792f78e41cf07e6
Fabricated Runtime Resource Overlays are overlays that are generated
at runtime and are stored in the data/ partition.
The system can fabricate RROs at runtime to dynamically theme the
device. Idmaps can now be created from APK RROs and fabricated RROs.
Rather than operating on ApkAssets, libidmap2 now operates on abstract
resource "containers" that supply resource values. Target resource
containers implement methods needed to query overlayable and target
overlay information. Currently only APKs can be loaded as target
resource containers. Overlay resource containers implement methods to
supply the mapping of target resource to overlay value and other
overlay information.
The format of a fabricated RRO is as follows:
0x00 - 0x04 : fabricated overlay magic (always FRRO)
0x04 - 0x08 : file format version
0x08 - 0x0c : crc of version + proto data
0x0c - EOF : proto fabricated overlay data
The magic is used to quickly detect if the file is a fabricated overlay.
The version is incremented whenever backwards incompatible changes are
made to the proto file format. Idmap must always be able to upgrade
fabricated overlays from previous versions to new versions, so all
previous versions must be checked into the tree.
Bug: 172471315
Test: libidmap2_tests && libandroidfw_tests
Change-Id: I4c9f29da278672e5695fb57d131a44c11a835180
Creates ApkAssets creation methods that allow an AssetsProvider to be
specified.
During idmap verification and creation, idmap2 currently opens the
target package and overlay package several times:
1) When the crc of the package is calculated in idmap2 verify
2) When the manifest of an overlay is parsed
3) When an ApkAssets is opened.
Opening large zip files (like framework-res.apk) is slow. If we opened
the zip one time as an ApkAssets, the resources.arsc would have to be
parsed (which means mmaping/unmapping and touching a lot of
resources.arsc pages). This would cause idmap2 to preform unnecessary
work just to check the crc of some files.
This change allows a ZipAssetsProvider to be created and then moved
for the creation of an ApkAssets. The zip file only needs to be opened
once and the resources.arsc is not parsed until reading resources is
actually necessary.
Bug: 172471315
Test: libandroidfw_tests
Test: CtsResourcesLoaderTests
Change-Id: I940bb2c13844c7f028776a623a9ecef45a4813bf
ResTable_config of every ResTable_type is read from device every time
AssetManager::RebuildFilterList is invoked. For large APKs (like
framework-res.apk), this causes a large number of page faults
when accessing the config from disk. The configs are also used in the
slow path of AssetManager::FindEntryInternal, which makes it even
slower. Instead cache the config on the TypeSpec of its ApkAsset.
Bug: 177247024
Test: libandroidfw_tests
Change-Id: I66d507c4eeb2399f7558f3d9dfc53c157129ada0
To support allowing for multiple <overlay> tags in one package, idmap2
must be able to generate an idmap for an individual <overlay> tag.
`idmap2 create` now accepts a --overlay-name flag that specifies which
tag to use to generate the idmap. The value of --overlay-name should be
set to the value of the android:name attribute on the <overlay> tag to
use.
If the flag is not present, idmap2 will look for an <overlay> tag with
no value for android:name.
Bug: 162841629
Test: libandroidfw_tests
Test: libidmap2_tests
Change-Id: I02316d0b88773f02c04a5d462be9825016fa496d
Overlay and target package paths can be longer than 256 characters.
Currently, the idmap will fail to be generated if either path
is longer than 256 characters.
This change removes the 256 character limit and makes parsing variable
length strings easier in libandroidfw.
Bug: 174676094
Test: idmap2_tests && libandroidfw_tests
Change-Id: Ic240cdb8700566b2ac2ade08da58bea852e4ae0c
When calling Resources#obtainStyledAttributes, if a value for an
attribute is supplied from the theme, and the value in the theme
is a reference to a resource, the reference will be resolved using
AssetManager2::ResolveReference each time the value from the theme
is selected. This causes Resources#obtainStyledAttributes to do
repeated work every time the same attribute is supplied from the
theme in multiple invocations.
Caching the result of ResolveReference reduces the cost of this
repeated work and reduces the amount of time needed to inflate views.
Before:
com.android.resources.perf (3 Tests)
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.748s)
youtube_ns_median: 95490747
youtube_ns_standardDeviation: 7282249
youtube_ns_mean: 98442515
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (10.862s)
maps_ns_standardDeviation: 4484213
maps_ns_mean: 87912988
maps_ns_median: 86325549
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.034s)
gmail_ns_median: 282175838
gmail_ns_standardDeviation: 6560876
gmail_ns_mean: 282869146
After:
com.android.resources.perf (3 Tests)
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.245s)
youtube_ns_median: 92292347
youtube_ns_standardDeviation: 5899906
youtube_ns_mean: 93045239
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (10.583s)
maps_ns_standardDeviation: 7567929
maps_ns_mean: 81895979
maps_ns_median: 78647883
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (21.439s)
gmail_ns_median: 229185043
gmail_ns_standardDeviation: 8770133
gmail_ns_mean: 232561234
These tests were done on a Pixel 3 and with cpu settings configured by
libs/hwui/tests/scripts/prep_generic.sh:
Locked CPUs 4,5,6,7 to 1459200 / 2803200 KHz
Disabled CPUs 0,1,2,3
Bug: 170232288
Test: atest ResourcesPerfWorkloads
Change-Id: I1e9e9acaa40fa60475a0e55230e11243f5b69b39
We recently rewrote CursorWindow, so let's get a fuzzer wired up
to see if it has any bugs.
This change creates a separate "libandroidfw_fuzz" library, since we
can't link to libbinder when building Windows host-side binaries;
the fuzzer doesn't need Window support.
And fix our first vulnerability where getFieldSlot() could be
tricked into reading out of bounds data.
The included corpus seed was generated using this example code:
CursorWindow* w = nullptr;
CursorWindow::create(android::String8("test"), 1 << 21, &w);
w->setNumColumns(3);
w->allocRow();
w->putLong(0,0,0xcafe);
w->putLong(0,1,0xcafe);
w->putLong(0,2,0xcafe);
// Row purposefully left empty
w->allocRow();
w->allocRow();
w->putNull(2,0);
w->putNull(2,1);
w->putNull(2,2);
w->allocRow();
w->putString(3,0,"cafe",5);
w->putString(3,1,"cafe",5);
w->putString(3,2,"cafe",5);
w->allocRow();
w->putDouble(4,0,3.14159f);
w->putDouble(4,1,3.14159f);
w->putDouble(4,2,3.14159f);
Parcel p;
w->writeToParcel(&p);
Bug: 169251528
Test: atest libandroidfw_tests:CursorWindowTest
Test: SANITIZE_HOST=address make ${FUZZER_NAME} && ${ANDROID_HOST_OUT}/fuzz/$(get_build_var HOST_ARCH)/${FUZZER_NAME}/${FUZZER_NAME}
Change-Id: I405d377900943de0ad732d3f1a1a0970e17d5140
The original CursorWindow implementation was created in Android 1.0
and has remained relatively unchanged since then. Unfortunately that
design results in very poor performance on large windows, since
reading or writing each FieldSlot is O(row/100) to traverse through
a chain of RowSlotChunks. It's also memory-inefficient due to how
it allocates RowSlotChunks in 404 byte chunks, even when there's only
a single row to store.
This change is a complete redesign of the CursorWindow internals to
use a "heap-and-stack" style approach, where a "heap" of strings
and blobs increment up from the bottom of the window while a "stack"
of FieldSlots increment down from the top of the window.
The included benchmarks show the following improvements, ensuring
no regressions for small windows, while offering very dramatic
improvements for larger windows:
Big cores Little cores
4x4 cursor no regression no regression
1024x4 cursor 2.2x faster 2.0x faster
16384x4 cursor 48.5x faster 24.4x faster
Detailed unit testing is also included to ensure that the rewrite
behaves correctly.
Bug: 169251528
Test: atest libandroidfw_tests
Test: atest CtsDatabaseTestCases
Test: atest FrameworksCoreTests:android.database
Test: ./frameworks/base/libs/hwui/tests/scripts/prep_generic.sh little && atest libandroidfw_benchmarks
Test: ./frameworks/base/libs/hwui/tests/scripts/prep_generic.sh little && atest CorePerfTests:android.database.CrossProcessCursorPerfTest
Change-Id: I90dff31fd550130dae917a33e0e1fa684e15c107
Remove malloc/free of android::ResTable_entry for inline overlay
values.
Add `target_entry_inline` to the idmap format to encode inline overlay
values separate from direct mapping of target resource to overlay
resource. This reduces the number of bytes needed to represent a direct
mapping of target resource to overlay resource from 9 bytes to 8 bytes
per entry.
Fixed all idmap alignment issues that required the framework to use
"#pragma pack(push, 1)" when loading idmaps.
Bug: 170341022
Test: idmap2_tests and libandroidfw_tests
Change-Id: Iab4d3902508f02773464724913e0ee966e3689e4
Alongside SIGNATURE and ACTOR_SIGNATURE policies, add CONFIG_SIGNATURE
policy to overlayable that overlay fulfills if it is signed with the
same certificate as the reference package whose package name is
declared in 'config-signature' tag of SystemConfig and is vetted by
OMS that it's a system pre-installed package.
BUG: 158726924
TEST: regular aapt2, idmap2, OMS tests
Merged-In: I645ee72271496008742886274be0d63a2985201b
Change-Id: I645ee72271496008742886274be0d63a2985201b
It's reasonable for a zero-length field to have its start offset
placed exactly at on the edge of the underlying buffer; we'll catch
any buffer overflows moments later when we verify the end offset
calculated from bufferSize.
Bug: 169547603
Test: atest libandroidfw_tests
Test: atest CtsDatabaseTestCases
Test: atest FrameworksCoreTests:android.database
Change-Id: I3d955f222343bd7ae63eaba7e367126dc136ecdf
Currently each CursorWindow allocates a 2MiB ashmem region to store
data, but this ends up being quite wasteful since the majority of
windows only end up storing a small handful of rows/columns. In
addition, creating and mmap'ing these ashmem regions requires
acquiring the mmap semaphore in the kernel, which can significantly
impact P95/P99 metrics when the system is under heavy load.
To mitigate the issues described above, this change adjusts
CursorWindow to send small windows (under 16KiB in size) directly
inline in Parcel responses without requiring an ashmem region.
CursorWindows also offer to gracefully "inflate" themselves into an
ashmem region when filled with more than 16KiB of data. This
requires some bugfixes around alloc() call sites to ensure that any
pointers are converted to offsets during a potential inflation.
The benchmarks referenced below show the following improvements
after this change is applied:
* Small cursor (1 row): 36% performance improvement
* Medium cursor (100 rows): no difference
* Large cursor (10k rows): no difference
Bug: 169251528
Test: atest CtsDatabaseTestCases
Test: atest FrameworksCoreTests:android.database
Test: ./frameworks/base/libs/hwui/tests/scripts/prep_generic.sh little && atest CorePerfTests:android.database.CrossProcessCursorPerfTest
Change-Id: Ie0fd149299f9847bf59a39f2855ed201bca4cdf6
Configurations differing only in density value will match the same
device configuration at runtime; therefore, they should be considered
sibling configurations in the dominator tree.
So, given configurations A and B:
1) If A has a density defined and B does not, A must never dominate B.
2) If A and B both have densities defined, neither must dominate the
other.
Previous behavior:
<default>
sw600dp-v13
sw600dp-hdpi-v13
sw800dp-hdpi-v13
sw800dp-xxhdpi-v13
New behavior:
<default>
sw600dp-v13
sw600dp-hdpi-v13
sw800dp-hdpi-v13
sw800dp-xxhdpi-v13
Bug: 167944889
Test: aapt2_tests
Merged-In: Ie9fba4c4b74af2b3bbf8fc6432539a99f8647634
Change-Id: Ie9fba4c4b74af2b3bbf8fc6432539a99f8647634
Configurations differing only in density value will match the same
device configuration at runtime; therefore, they should be considered
sibling configurations in the dominator tree.
So, given configurations A and B:
1) If A has a density defined and B does not, A must never dominate B.
2) If A and B both have densities defined, neither must dominate the
other.
Previous behavior:
<default>
sw600dp-v13
sw600dp-hdpi-v13
sw800dp-hdpi-v13
sw800dp-xxhdpi-v13
New behavior:
<default>
sw600dp-v13
sw600dp-hdpi-v13
sw800dp-hdpi-v13
sw800dp-xxhdpi-v13
Bug: 167944889
Test: aapt2_tests
Change-Id: Ie9fba4c4b74af2b3bbf8fc6432539a99f8647634
Alongside SIGNATURE and ACTOR_SIGNATURE policies, add CONFIG_SIGNATURE
policy to overlayable that overlay fulfills if it is signed with the
same certificate as the reference package whose package name is
declared in 'config-signature' tag of SystemConfig and is vetted by
OMS that it's a system pre-installed package.
BUG: 158726924
TEST: regular aapt2, idmap2, OMS tests
Change-Id: I645ee72271496008742886274be0d63a2985201b
Currently we rely on libc to call std::free on the malloced pointer to
the overlay table entry. Rather than relying on the libc implementation
to call std::free instead of trying to delete the pointer as if it was
created using "new", explicitly pass std::free as the deleter to the
data held by the std::shared_ptr.
Bug: 159562360
Test: Observed no heap growth while looping infinitely and retrieving
a string resource overlaid using an inline xml value
Change-Id: I9a2ebaf9a993ad9c44fab8f052430c8142d4347d
backupToTar function used in fullbackup currently uses off_t to
represent file sizes which overflows for files having size >= 2 GiB
on 32 bit target architectures.
This CL replaces off_t with off64_t to overcome this limitation.
Fixes: 157633626
Test: atest libandroidfw_tests
Test: Locally running a backup with a test app having a file of size
2GiB and verifying the size variable in backupToTar doesn't overflow
on a 32 bit target.
Change-Id: I404cced810f331ecb9327495ce1f092f1fa2d378
If the fulfilled policies change without the contents of the target
and overlay APKs changing, the idmap for the overlay should be
regenerated. This change adds fulfilled policies and enforce
overlayable to the idmap header so that idmap2d can determine if the
polices or enforce overlayable changed from what was used to generate
the idmap.
Bug: 119328308
Test: idmap2_tests
Test: atest RegenerateIdmapTest
Change-Id: I96f970e82b5243be01b205ac2cb6ab249c6100bc
If an application targets R+, prevent the application from being
installed if the app has a compressed resources.arsc or if the
resources.arsc is not aligned on a 4-byte boundary. Resources tables
that cannot be memory mapped have to be read into a buffer in RAM
and exert unnecessary memory pressure on the system.
Bug: 132742131
Test: manual (adding CTS tests)
Change-Id: Ieef764c87643863de24531fac12cc520fe6d90d0
When the target package update, check if the idmap file must change.
If so, propagate the idmap changes to the targets overlay paths, and
invalidate cached overlay ApkAssets in ResourcesManager.
Bug: 147794117
Bug: 150877400
Test: OverlayRemountedTest
Test: libandroidfw_tests
Change-Id: I6115c30bae3672b188a5ff270720a0eea15b43b5
Before overlays could reference internal resources, overlays were
added to AssetManagers before shared libraries. Overlays are now
loaded as shared libraries so they must be assigned package ids
after non-overlay shared libraries so enabling and disabling
overlays does not affect the package ids of the non-overlay shared
libraries.
Since overlays are added to the end of the AssetManager by
ResourcesManager, enabling and disabling overlays will not change
the assets cookie of shared libraries loaded through ResourcesManager,
but it will change the apk assets cookie or overlays added through
AssetManager#addAssetPathAsSharedLibrary.
The package ids of shared libraries added through
AssetManager#addAssetPathAsSharedLibrary will not be affected by
overlays since overlay package ids are assigned last.
Bug: 146685730
Test: CtsHostsideWebViewTests
Change-Id: If7ea17d51b18769bf2465e29af3ae6a71004d06c
This change must be reverted because it broke packages with the same
package name but different package ids loaded at once.
Bug: 146685730
Test: MultiSplitInstallTest
This reverts commit fe50d739f7.
Change-Id: If6364fd660c76284452f77e7d4f09a3df9dede1d
Querying in the native layer for assets provided through
AssetsProviders does not currently work. This change refactors the
AssetProvider API to return a file descriptor that is read in the
native layer and can bubble up to the java layer.
This change also removes the InputStream API to favor of developers
using memfd_create.
Bug: 142716192
Test: atest ResourceLoaderValuesTest
Change-Id: I1a7eca0994c3b7cc32008d9a72bf91086ff0e816
This API allows a directory to be loaded as if it was a zipped APK.
This is a substitute for the DirectoryAssetProvider API that
currently does not work in the native layer.
Bug: 142716192
Test: atest FrameworksResourceLoaderTests
Change-Id: Ia13e15653e75b421423dd56f9fe89e183ab4cb9a
To add the partner requested ResourcesProvider#loadFromDir APIs, this
change adds format type integer that allows us to reduce the number of
ApkAssets loading overrides.
This change also adds hidden offset and length based ResourcesProvider
APIs that could not make R.
Bug: 142716192
Test: atest FrameworksResourceLoaderTests
Change-Id: I926fde257cae701901dcd4ca408024feae8c90a6
Merged-In: I926fde257cae701901dcd4ca408024feae8c90a6
* normally a copy constructor should call base copy constructor
Bug: 151109313
Test: WITH_TIDY=1 make
Change-Id: I9677889973178c26cd787a457333877619956b51
