It isn't enough to only dump the theme id and name because there is
no whole theme inheritance map. Developers need the map to figure out
why the attribute value is so strange.
A theme is a kind of style. This patch creates an API to retrieve the
parent id for any style. And, it dumps the whole inheritance between
themes until it can't find the parent style.
Test: manual test described in b/209708201#comment5
Bug: 209708201
Change-Id: I9de0e6e9aaeff7cb9fc89dc9b8a04925e84f975c
Print the best matching configuration from the given resource while
looking up for better understanding why an overlay isn't taken.
Bug: 193701453
Test: atest AssetManager2Test libandroidfw_tests
Test: manually install a target package has values and values-en,
and an overlay package only has values. Then, use overlay
lookup command to check if the output shows related info.
Change-Id: I0cb415b9e932e1d4bfd0be2e1768adeb78e7b7d0
stdout and stderr are macros in musl, which causes build failures
when they are used as field names. Rename the fields to stdout_str
and stderr_str.
Test: builds
Change-Id: I04770d2d6e82ca8393308ab12a304838a80f0ae5
Memory churn is high when swapping the ResourcesImpl of a Resources
object. Each time Resources#setImpl is invoked, all themes based on
that Resources object are assigned new ThemeImpl objects that are
created using the new ResourcesImpl.
ThemeImpls can only belong to one Theme object, so the old
implementation is discarded and the theme takes ownership of the new
ThemeImp.
This creates performance problems when framework overlays are toggled.
Toggling overlays targeting the framework causes all themes across all
processes to recreate and reallocate all of their themes. By rebasing
the ThemeImpl on the new ResourcesImpl without deallocating the native
theme memory, we reduce churn and produce less garbage that needs to
be garbage collected.
Bug: 141198925
Test: atest libandroidfw_tests
Test: atest ResourcesPerfWorkloads
Change-Id: I03fb31ee09c9cfdbd3c41bcf0b605607dab54ed7
Themes are represented in the native layer using an array where the
entry id of resource ids are used to index into the array. This causes
native allocation size of a theme to correlate with the largest
attribute resource id in the styles applied to the theme.
From manual testing, I determined that on average in 1P apps and
system_server only 10-20% of the space allocated for themes actually
hold theme attribute values and the rest is empty/unused space.
Using std::vector and std::lower_bound to create a sparse array
representation will reduce amount of memory allocated by themes while
having a minimal impact on the performance of querying the attributes
defined in a theme.
From testing with ResourcesPerfWorkloads, this increased time spent in
the resources synthetic benchmarks by ~1%.
Bug: 141198925
Test: atest libandroidfw_tests
Test: atest ResourcesPerfWorkloads
Change-Id: Iec512b31b0545b0898ff248cd23f074a20fff45d
Rather than attempt to remove all users of Asset::getBuffer (which
includes AssetManager(1), aapt(1), and other places) and migrate them
to use Asset::getIncFsBuffer, verify the presence of all the data in
the buffer before returning a raw pointer to the buffer data to
guarantee callers will not unexpectedly get a SIGBUS due to incremental
installation.
Bug: 179254882
Test: builds
Change-Id: I24fd9036bc53a8c23166b5471862ee542630fb56
When an application is incrementally installed, and a resources
operation fails due to the resources not being fully present,
the app should crash instead of swallowing the error and
returning default values to not alter the experience of
using the application.
Disable IncFsFileMap protections on ApkAssets that are a part of the
application that is running (base and splits).
Bug: 187220960
Test: atest ResourcesHardeningTest
Change-Id: Ibc67aca688720f983c7c656f404593285a54999b
Fabricated RROs do not provide assets but are stored on disk. Ensure
that the path to the frro is returned when querying for the ApkAssets
path (which is mostly for debug purposes).
Bug: 181338216
Test: enable frro, use cmd overlay lookup to see path in resolution
Change-Id: Ibf9b1bf0a995325affbf084c71b1e87c5682e734
To allow apps that compiled against a pre-release SDK to continue
working for a period of time after API finalization, a new tag,
<staging-public-group-final>, has been added to aapt2.
When finalizing the framework resource API, converting
<staging-public-group> tags to <staging-public-group-final> will
cause aapt2 to generate the resource table so that there is a resource
entry for the old non-finalized (staged) resource ID and another entry
for the finalized resource ID of newly finalized resources. This allows
an application that compiled against the pre-release SDK to continue
resolving resources using pre-release resource IDs.
All references to pre-release resource IDs will be rewritten to their
finalized resource IDs through the information stored in the new staged
alias chunk. This allows applications compiled against
<staging-public-group> resources to use the newly finalized
resource ID without re-compilation.
When an application is re-compiled against the SDK with
<staging-public-group-final> tags, the application will use the
finalized resource IDs.
This change limits the use of the alias chunk to the framework for S.
Bug: 183411356
Test: aapt2_test
Change-Id: Iba1c3033c3c2f32de8e4a19b58d3921c971092c4
When an APK defines multiple "uses-sdk" tags, the Android runtime only
uses the minSdkVersion and targetSdkVersion values from the last
occurrence of the "uses-sdk" tag.
For example an application with the following tags:
<uses-sdk android:minSdkVersion="21" android:targetSdkVersion="29"/>
<uses-sdk android:maxSdkVersion="28"/>
Will have the following version codes at runtime:
minSdk=1 targetSdk=1
Another example:
<uses-sdk android:minSdkVersion="5" android:targetSdkVersion="28"/>
<uses-sdk android:minSdkVersion="5" android:targetSdkVersion="19"/>
Will have the following version codes at runtime:
minSdk=5 targetSdk=19
AAPT must print the version data from only the last tag, skipping other
occurrences of the tag.
Bug: 175789289
Test: manual
Change-Id: Ic855ff920d0b7abedd250d977bfa55189f4c8946
If an ApkAssets is created and destroyed before being put into an
AssetManager, the native layer may not have been synchronized in the
thread that does Java object finalization. So when the Java object gets
deleted on the finalization thread, it may see old/unallocated native
data and not delete the native object correctly.
Similar to how we already lock on the native AssetManager whenever
entering the JNI layer, we should also do the same for ApkAssets.
Bug: 186528775
Test: boot and no crashes
Change-Id: I5fcb9e788a9e3cb8baeb92fa6efc882d5bbbf0b0
staging-public-group is a tag for putting resources that have been
added during platform development, but have not yet been finalized,
into a separate resource id namespace.
R.java fields of staged resources are non-final, so when the SDK is
finalized, applications using the android R.java will automatically
use the new finalized resource id without having to recompile.
Staged resources can exist either in the same type id as the type's
non-staged counterpart or in a separate type id. Multiple
staging-public-group tags each with a different type id can exist
simultaneously, which allows for multiple versions of the platform
to be developed at once.
Bug: 183411093
Test: aapt2_tests
Change-Id: Ibb6c84c3626751e33c6097f35a03e306bb85616a
With the introduction of ResourcesProviders, not all ApkAssets have
paths on disk. Theme::SetTo and various AssetManager methods use the
path to perform equality checking on ApkAssets. This equality check
will be performed on the debug string of an ApkAssets if it does not
have a path on disk. This causes ApkAssets with the same debug name
(like "<empty>") to be seen as the same ApkAssets. Rather than using
path, the pointer to the ApkAssets should be used for equality checking
since ResourcesManager caches and reuses ApkAssets when multiple
AssetManagers request the same assets.
Bug: 177101983
Test: atest CtsResourcesLoaderTests
Change-Id: I11f6a2a3a7cc8febe3f976236792f78e41cf07e6
Fabricated Runtime Resource Overlays are overlays that are generated
at runtime and are stored in the data/ partition.
The system can fabricate RROs at runtime to dynamically theme the
device. Idmaps can now be created from APK RROs and fabricated RROs.
Rather than operating on ApkAssets, libidmap2 now operates on abstract
resource "containers" that supply resource values. Target resource
containers implement methods needed to query overlayable and target
overlay information. Currently only APKs can be loaded as target
resource containers. Overlay resource containers implement methods to
supply the mapping of target resource to overlay value and other
overlay information.
The format of a fabricated RRO is as follows:
0x00 - 0x04 : fabricated overlay magic (always FRRO)
0x04 - 0x08 : file format version
0x08 - 0x0c : crc of version + proto data
0x0c - EOF : proto fabricated overlay data
The magic is used to quickly detect if the file is a fabricated overlay.
The version is incremented whenever backwards incompatible changes are
made to the proto file format. Idmap must always be able to upgrade
fabricated overlays from previous versions to new versions, so all
previous versions must be checked into the tree.
Bug: 172471315
Test: libidmap2_tests && libandroidfw_tests
Change-Id: I4c9f29da278672e5695fb57d131a44c11a835180
Creates ApkAssets creation methods that allow an AssetsProvider to be
specified.
During idmap verification and creation, idmap2 currently opens the
target package and overlay package several times:
1) When the crc of the package is calculated in idmap2 verify
2) When the manifest of an overlay is parsed
3) When an ApkAssets is opened.
Opening large zip files (like framework-res.apk) is slow. If we opened
the zip one time as an ApkAssets, the resources.arsc would have to be
parsed (which means mmaping/unmapping and touching a lot of
resources.arsc pages). This would cause idmap2 to preform unnecessary
work just to check the crc of some files.
This change allows a ZipAssetsProvider to be created and then moved
for the creation of an ApkAssets. The zip file only needs to be opened
once and the resources.arsc is not parsed until reading resources is
actually necessary.
Bug: 172471315
Test: libandroidfw_tests
Test: CtsResourcesLoaderTests
Change-Id: I940bb2c13844c7f028776a623a9ecef45a4813bf
ResTable_config of every ResTable_type is read from device every time
AssetManager::RebuildFilterList is invoked. For large APKs (like
framework-res.apk), this causes a large number of page faults
when accessing the config from disk. The configs are also used in the
slow path of AssetManager::FindEntryInternal, which makes it even
slower. Instead cache the config on the TypeSpec of its ApkAsset.
Bug: 177247024
Test: libandroidfw_tests
Change-Id: I66d507c4eeb2399f7558f3d9dfc53c157129ada0
To support allowing for multiple <overlay> tags in one package, idmap2
must be able to generate an idmap for an individual <overlay> tag.
`idmap2 create` now accepts a --overlay-name flag that specifies which
tag to use to generate the idmap. The value of --overlay-name should be
set to the value of the android:name attribute on the <overlay> tag to
use.
If the flag is not present, idmap2 will look for an <overlay> tag with
no value for android:name.
Bug: 162841629
Test: libandroidfw_tests
Test: libidmap2_tests
Change-Id: I02316d0b88773f02c04a5d462be9825016fa496d
Overlay and target package paths can be longer than 256 characters.
Currently, the idmap will fail to be generated if either path
is longer than 256 characters.
This change removes the 256 character limit and makes parsing variable
length strings easier in libandroidfw.
Bug: 174676094
Test: idmap2_tests && libandroidfw_tests
Change-Id: Ic240cdb8700566b2ac2ade08da58bea852e4ae0c
When calling Resources#obtainStyledAttributes, if a value for an
attribute is supplied from the theme, and the value in the theme
is a reference to a resource, the reference will be resolved using
AssetManager2::ResolveReference each time the value from the theme
is selected. This causes Resources#obtainStyledAttributes to do
repeated work every time the same attribute is supplied from the
theme in multiple invocations.
Caching the result of ResolveReference reduces the cost of this
repeated work and reduces the amount of time needed to inflate views.
Before:
com.android.resources.perf (3 Tests)
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.748s)
youtube_ns_median: 95490747
youtube_ns_standardDeviation: 7282249
youtube_ns_mean: 98442515
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (10.862s)
maps_ns_standardDeviation: 4484213
maps_ns_mean: 87912988
maps_ns_median: 86325549
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.034s)
gmail_ns_median: 282175838
gmail_ns_standardDeviation: 6560876
gmail_ns_mean: 282869146
After:
com.android.resources.perf (3 Tests)
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.245s)
youtube_ns_median: 92292347
youtube_ns_standardDeviation: 5899906
youtube_ns_mean: 93045239
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (10.583s)
maps_ns_standardDeviation: 7567929
maps_ns_mean: 81895979
maps_ns_median: 78647883
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (21.439s)
gmail_ns_median: 229185043
gmail_ns_standardDeviation: 8770133
gmail_ns_mean: 232561234
These tests were done on a Pixel 3 and with cpu settings configured by
libs/hwui/tests/scripts/prep_generic.sh:
Locked CPUs 4,5,6,7 to 1459200 / 2803200 KHz
Disabled CPUs 0,1,2,3
Bug: 170232288
Test: atest ResourcesPerfWorkloads
Change-Id: I1e9e9acaa40fa60475a0e55230e11243f5b69b39
We recently rewrote CursorWindow, so let's get a fuzzer wired up
to see if it has any bugs.
This change creates a separate "libandroidfw_fuzz" library, since we
can't link to libbinder when building Windows host-side binaries;
the fuzzer doesn't need Window support.
And fix our first vulnerability where getFieldSlot() could be
tricked into reading out of bounds data.
The included corpus seed was generated using this example code:
CursorWindow* w = nullptr;
CursorWindow::create(android::String8("test"), 1 << 21, &w);
w->setNumColumns(3);
w->allocRow();
w->putLong(0,0,0xcafe);
w->putLong(0,1,0xcafe);
w->putLong(0,2,0xcafe);
// Row purposefully left empty
w->allocRow();
w->allocRow();
w->putNull(2,0);
w->putNull(2,1);
w->putNull(2,2);
w->allocRow();
w->putString(3,0,"cafe",5);
w->putString(3,1,"cafe",5);
w->putString(3,2,"cafe",5);
w->allocRow();
w->putDouble(4,0,3.14159f);
w->putDouble(4,1,3.14159f);
w->putDouble(4,2,3.14159f);
Parcel p;
w->writeToParcel(&p);
Bug: 169251528
Test: atest libandroidfw_tests:CursorWindowTest
Test: SANITIZE_HOST=address make ${FUZZER_NAME} && ${ANDROID_HOST_OUT}/fuzz/$(get_build_var HOST_ARCH)/${FUZZER_NAME}/${FUZZER_NAME}
Change-Id: I405d377900943de0ad732d3f1a1a0970e17d5140
The original CursorWindow implementation was created in Android 1.0
and has remained relatively unchanged since then. Unfortunately that
design results in very poor performance on large windows, since
reading or writing each FieldSlot is O(row/100) to traverse through
a chain of RowSlotChunks. It's also memory-inefficient due to how
it allocates RowSlotChunks in 404 byte chunks, even when there's only
a single row to store.
This change is a complete redesign of the CursorWindow internals to
use a "heap-and-stack" style approach, where a "heap" of strings
and blobs increment up from the bottom of the window while a "stack"
of FieldSlots increment down from the top of the window.
The included benchmarks show the following improvements, ensuring
no regressions for small windows, while offering very dramatic
improvements for larger windows:
Big cores Little cores
4x4 cursor no regression no regression
1024x4 cursor 2.2x faster 2.0x faster
16384x4 cursor 48.5x faster 24.4x faster
Detailed unit testing is also included to ensure that the rewrite
behaves correctly.
Bug: 169251528
Test: atest libandroidfw_tests
Test: atest CtsDatabaseTestCases
Test: atest FrameworksCoreTests:android.database
Test: ./frameworks/base/libs/hwui/tests/scripts/prep_generic.sh little && atest libandroidfw_benchmarks
Test: ./frameworks/base/libs/hwui/tests/scripts/prep_generic.sh little && atest CorePerfTests:android.database.CrossProcessCursorPerfTest
Change-Id: I90dff31fd550130dae917a33e0e1fa684e15c107
Remove malloc/free of android::ResTable_entry for inline overlay
values.
Add `target_entry_inline` to the idmap format to encode inline overlay
values separate from direct mapping of target resource to overlay
resource. This reduces the number of bytes needed to represent a direct
mapping of target resource to overlay resource from 9 bytes to 8 bytes
per entry.
Fixed all idmap alignment issues that required the framework to use
"#pragma pack(push, 1)" when loading idmaps.
Bug: 170341022
Test: idmap2_tests and libandroidfw_tests
Change-Id: Iab4d3902508f02773464724913e0ee966e3689e4
Alongside SIGNATURE and ACTOR_SIGNATURE policies, add CONFIG_SIGNATURE
policy to overlayable that overlay fulfills if it is signed with the
same certificate as the reference package whose package name is
declared in 'config-signature' tag of SystemConfig and is vetted by
OMS that it's a system pre-installed package.
BUG: 158726924
TEST: regular aapt2, idmap2, OMS tests
Merged-In: I645ee72271496008742886274be0d63a2985201b
Change-Id: I645ee72271496008742886274be0d63a2985201b
It's reasonable for a zero-length field to have its start offset
placed exactly at on the edge of the underlying buffer; we'll catch
any buffer overflows moments later when we verify the end offset
calculated from bufferSize.
Bug: 169547603
Test: atest libandroidfw_tests
Test: atest CtsDatabaseTestCases
Test: atest FrameworksCoreTests:android.database
Change-Id: I3d955f222343bd7ae63eaba7e367126dc136ecdf
Currently each CursorWindow allocates a 2MiB ashmem region to store
data, but this ends up being quite wasteful since the majority of
windows only end up storing a small handful of rows/columns. In
addition, creating and mmap'ing these ashmem regions requires
acquiring the mmap semaphore in the kernel, which can significantly
impact P95/P99 metrics when the system is under heavy load.
To mitigate the issues described above, this change adjusts
CursorWindow to send small windows (under 16KiB in size) directly
inline in Parcel responses without requiring an ashmem region.
CursorWindows also offer to gracefully "inflate" themselves into an
ashmem region when filled with more than 16KiB of data. This
requires some bugfixes around alloc() call sites to ensure that any
pointers are converted to offsets during a potential inflation.
The benchmarks referenced below show the following improvements
after this change is applied:
* Small cursor (1 row): 36% performance improvement
* Medium cursor (100 rows): no difference
* Large cursor (10k rows): no difference
Bug: 169251528
Test: atest CtsDatabaseTestCases
Test: atest FrameworksCoreTests:android.database
Test: ./frameworks/base/libs/hwui/tests/scripts/prep_generic.sh little && atest CorePerfTests:android.database.CrossProcessCursorPerfTest
Change-Id: Ie0fd149299f9847bf59a39f2855ed201bca4cdf6
Configurations differing only in density value will match the same
device configuration at runtime; therefore, they should be considered
sibling configurations in the dominator tree.
So, given configurations A and B:
1) If A has a density defined and B does not, A must never dominate B.
2) If A and B both have densities defined, neither must dominate the
other.
Previous behavior:
<default>
sw600dp-v13
sw600dp-hdpi-v13
sw800dp-hdpi-v13
sw800dp-xxhdpi-v13
New behavior:
<default>
sw600dp-v13
sw600dp-hdpi-v13
sw800dp-hdpi-v13
sw800dp-xxhdpi-v13
Bug: 167944889
Test: aapt2_tests
Merged-In: Ie9fba4c4b74af2b3bbf8fc6432539a99f8647634
Change-Id: Ie9fba4c4b74af2b3bbf8fc6432539a99f8647634
Configurations differing only in density value will match the same
device configuration at runtime; therefore, they should be considered
sibling configurations in the dominator tree.
So, given configurations A and B:
1) If A has a density defined and B does not, A must never dominate B.
2) If A and B both have densities defined, neither must dominate the
other.
Previous behavior:
<default>
sw600dp-v13
sw600dp-hdpi-v13
sw800dp-hdpi-v13
sw800dp-xxhdpi-v13
New behavior:
<default>
sw600dp-v13
sw600dp-hdpi-v13
sw800dp-hdpi-v13
sw800dp-xxhdpi-v13
Bug: 167944889
Test: aapt2_tests
Change-Id: Ie9fba4c4b74af2b3bbf8fc6432539a99f8647634
Alongside SIGNATURE and ACTOR_SIGNATURE policies, add CONFIG_SIGNATURE
policy to overlayable that overlay fulfills if it is signed with the
same certificate as the reference package whose package name is
declared in 'config-signature' tag of SystemConfig and is vetted by
OMS that it's a system pre-installed package.
BUG: 158726924
TEST: regular aapt2, idmap2, OMS tests
Change-Id: I645ee72271496008742886274be0d63a2985201b
Currently we rely on libc to call std::free on the malloced pointer to
the overlay table entry. Rather than relying on the libc implementation
to call std::free instead of trying to delete the pointer as if it was
created using "new", explicitly pass std::free as the deleter to the
data held by the std::shared_ptr.
Bug: 159562360
Test: Observed no heap growth while looping infinitely and retrieving
a string resource overlaid using an inline xml value
Change-Id: I9a2ebaf9a993ad9c44fab8f052430c8142d4347d
backupToTar function used in fullbackup currently uses off_t to
represent file sizes which overflows for files having size >= 2 GiB
on 32 bit target architectures.
This CL replaces off_t with off64_t to overcome this limitation.
Fixes: 157633626
Test: atest libandroidfw_tests
Test: Locally running a backup with a test app having a file of size
2GiB and verifying the size variable in backupToTar doesn't overflow
on a 32 bit target.
Change-Id: I404cced810f331ecb9327495ce1f092f1fa2d378
If the fulfilled policies change without the contents of the target
and overlay APKs changing, the idmap for the overlay should be
regenerated. This change adds fulfilled policies and enforce
overlayable to the idmap header so that idmap2d can determine if the
polices or enforce overlayable changed from what was used to generate
the idmap.
Bug: 119328308
Test: idmap2_tests
Test: atest RegenerateIdmapTest
Change-Id: I96f970e82b5243be01b205ac2cb6ab249c6100bc
