There are 2 copy codes for handling the obfuscating resources names
between serializing to pb format file and writing to apks. The
obfuscator also needs to dump resources names. It means there are
3 places to handle the obfuscating resources names.
So, using C++ lambda to apply the callback mechanism refactors the
codes.
Obfuscator
* Initial a Obfuscator according to Optimizer's options
* Add Obfuscator.IsEnabled() function.
return true either shorten_resource_paths_ is true or
collapse_key_stringpool_ is true.
Bug: 228192695
Test: atest aapt2_test idmap2_test
Change-Id: Idd2442beecf41e9392620ff801a36fd1285e06f9
Bug: 237583012
Most offsets to the entries can be well encoded in 16-bit,
and given entries are 4-byte aligned, this gives us a range
of entry offsets from 0x00000 to 0xfffe * 4u, with 0xffffu
to represent ResTable_type::NO_ENTRY.
For now, 16-bit entry offset will be enabled only when:
* all the entry offsets can be represented in 16-bit
* --enable-compact-entries switch is turned on
Change-Id: I1c815c052aa5fba6eab2529434d31d7714c13694
Bug: 237583012
Given the large number of simple resources such as strings in
Android resources, their ResTable_entry and Res_value can be
encoded together in a compact way. This allows a significant
saving in both storage and memory footprint.
The basic observations for simple resources are:
* ResTable_entry.size will always be sizeof(ResTable_entry)
unless it's a complex entry
* ResTable_entry.key is unlikely to exceed 16-bit
* ResTable_entry.flags only uses 3 bits for now
* Res_value.size will always be sizeof(Res_value)
Given the above, we could well encode the information into
a compact/compatible structure.
struct compact {
uint16_t key;
uint16_t flags;
uint32_t data;
};
The layout of this structure will allow maximum backward
compatibility. e.g. the flags will be at the same offset,
and a
`dtohs((ResTable_entry *)entry->flags) & FLAG_COMPACT`
would tell if this entry is a compact one or not. For a
compact entry:
struct compact *entry;
entry_size == sizeof(*entry)
entry_key == static_cast<uint32_t>(dtohs(entry->key))
entry_flags == dtohs(entry->flags) & 0xff // low 8-bit
data_type == dtohs(entry->flags) >> 8 // high 8-bit
data_size == sizeof(Res_value)
data_value == dtohl(entry->data)
To allow minimum code change and backward compatibility,
we change 'struct ResTable_entry' to 'union ResTable_entry',
with an anonymous structure inside that's fully backward
compatible. Thus, any existing reference such as:
ResTable_entry *entry = ...
if (dtohs(entry->flags) & ResTable_entry::FLAG_COMPLEX) ...
would still work.
However, special care needs to be taken after an entry is
obtained, and when value needs to be extracted.
A compact entry will not encode a complex value, and hence
complex entries/values are handled the same way.
Change-Id: I15d97a4f5e85fab28c075496f7f0cf6b1fcd73e3
This CL introduces DeduplicateItemsResEntryWriter which stores unique
pairs of entry and value and deduplicates a new pair based of this data
before writing it into the buffer.
Bug: b/249793372
Test: ResEntriesWriter_test, TableFlattener_test
Change-Id: I938a3425a87bfad9853e59193680de2050a95bd2
ResEntryWriter is responsible to write pairs of entry+value into binary
buffers and allows easy customization how and where entries are written.
This is no-op refactoring and all existing tests should pass.
Bug: b/249793372
Test: Existing tests
Change-Id: Ica98a25aeb49cc96cad28547c462b36316a8adb6
This allows to use sparse encoding for APKs that do not specify minSdk
version inside. This is common for split APKs generated by bundletool
when Android App Bundle (AAB) is split into multiple APKs.
Bug: b/223394605
Test: TableFlattener_test
Change-Id: I2a99a611f855cffbcea20018d869e1eb54d1b3d5
aapt2 disable sparse encoding when sdk < 32
Bug: 197642721
Test: Updated and verified affected atests pass
Change-Id: I839978207eb30f958f5d695cf8dea7f0dad1312f
aapt2 enable sparse encoding when sdk is not set.
Bug: 197642721
Test: Added new unit test and verified all tests pass
Change-Id: Ib58941d98eafe8e543d50054c9d36a51d7977c71
Once the build process finds one overlayable resource entry is not in
the list of collapse name exemption, the build process outputs the
resource name automatically and gives a warning message to notify the
developers.
Bug: 228192695
Test: atest aapt2_test
Change-Id: I989b75343d803b059c902baa088590fce7237211
Treat it as Stylable, same as XML parsing works
Bug: 203821692
Test: aapt2 dump resources CtsContentTestCases.apk succeeds
Change-Id: I836495743aea1a19af9ed273a6b33f15494b2e97
The code used to expect all original staged resources to get
parsed before their finalized version. Looks like that's not the
actual case, and parsing code needs to expect those to come
later, and preserve the information that they aren't needed
anymore
Bug: 203779955
Test: manual, aapt2 dump resources <finalized framework SDK>
Change-Id: I6d9efe2eb7b406d05ca49a0a6ed745a1c509e2ef
With c++17, std::optional provides the functionality that Maybe
provided.
Bug: 183215655
Test: aapt2_tests
Change-Id: I62bb9c2fa4985dc5217a6ed153df92b85ad9a034
Fixes BinaryResourceParser loading of alias chunk and makes changes
that did not get committed in 2fedba9a32.
Bug: 183411356
Test: aapt2_test
Change-Id: Ieff9166100019f38ddcfe900014709b15db24e43
To allow apps that compiled against a pre-release SDK to continue
working for a period of time after API finalization, a new tag,
<staging-public-group-final>, has been added to aapt2.
When finalizing the framework resource API, converting
<staging-public-group> tags to <staging-public-group-final> will
cause aapt2 to generate the resource table so that there is a resource
entry for the old non-finalized (staged) resource ID and another entry
for the finalized resource ID of newly finalized resources. This allows
an application that compiled against the pre-release SDK to continue
resolving resources using pre-release resource IDs.
All references to pre-release resource IDs will be rewritten to their
finalized resource IDs through the information stored in the new staged
alias chunk. This allows applications compiled against
<staging-public-group> resources to use the newly finalized
resource ID without re-compilation.
When an application is re-compiled against the SDK with
<staging-public-group-final> tags, the application will use the
finalized resource IDs.
This change limits the use of the alias chunk to the framework for S.
Bug: 183411356
Test: aapt2_test
Change-Id: Iba1c3033c3c2f32de8e4a19b58d3921c971092c4
AAPT2 Macros are compile-time resources definitions that are expanded
when referenced during the link phase.
A macro must be defined in the res/values.xml directory. A macro
definition for a macro named "foo" looks like the following:
<macro name="foo">contents</macro>
When "@macro/foo" is used in the res/values directory or in a compiled
XML file, the contents of the macro replace the macro reference and
then the substituted contents are compiled and linked. If the macro
contents reference xml namespaces from its original definition, the
namespaces of the original macro definition will be used to determine
which package is being referenced.
Macros can be used anywhere resources can be referenced using the
@package:type/entry syntax.
Macros are not included in the final resource table or the R.java since
they are not actual resources.
Bug: 175616308
Test: aapt2_tests
Change-Id: I48b29ab6564357b32b4b4e32bff7ef06036382bc
For future macro support, aapt2 must be able to convert Reference
values into other Value types. Currently a DescendingValueVisitor is
used to visit all of the References in a ResourceTable or a compiled
XML file to set their resource ids during the link phase. This was fine
since we were only mutating the resource id of the visited Reference.
A macro may reference a String, BinaryPrimitive, or any other Item
type. During the link phase, we will need to transform references to
macros into the values of the macros.
The only parameter in the methods of the ValueVisitor interface is a
raw pointer to the type being visited. The visitor interface does not
support reassigning the visited type to a different type.
ValueTransformer is a new interface for consuming a Value type and
transforming it into a compatible Value type. This change refactors
Value::Clone to use this interface.
Bug: 175616308
Test: aapt2_tests
Change-Id: Ic1b9d718b932c208764114cd9c74d880e189ccb0
To make S finalization easier, this changes the framework SDK so that
apps linking against it will be able to continue working as expected
after the first phase of SDK finalization.
During the first phase of SDK finalization, the resource ids of
resources that have not been removed are finalized.
staging-public-group tags are converted to staging-public-group-final
tags in order to encode into the framework what the staged resource id
of a finalized resource was. When an app recompiles, it will use the
finalized resource id. Then after all apps recompile, phase 2 of
finalization begins, in which the staging-public-group-final tags are
removed so apps can no longer use the staged resource ids.
Apps that link against the SDK (provided they are using a recent
version of aapt) will encode references to staged resources as
TYPE_DYNAMIC_REFERENCE and TYPE_DYNAMIC_ATTRIBUTE. The values of R
fields for staged resources are defined out-of-line to prevent them
from being inlined into apps linking agsint the SDK. This allows the
resource ids to change during phase 1 of API finalization.
Bug: 183413192
Test: `aapt2 diff` and resource ids stayed the same
Test: `aapt2 dump` of framework-res.apk and observe staged resources
Change-Id: Ie2275c608297a5f63dde8b1cf795415112cbcc24
staging-public-group is a tag for putting resources that have been
added during platform development, but have not yet been finalized,
into a separate resource id namespace.
R.java fields of staged resources are non-final, so when the SDK is
finalized, applications using the android R.java will automatically
use the new finalized resource id without having to recompile.
Staged resources can exist either in the same type id as the type's
non-staged counterpart or in a separate type id. Multiple
staging-public-group tags each with a different type id can exist
simultaneously, which allows for multiple versions of the platform
to be developed at once.
Bug: 183411093
Test: aapt2_tests
Change-Id: Ibb6c84c3626751e33c6097f35a03e306bb85616a
This changes refactors tests to use the NewResourceBuilder class
that makes it easier to construct resource entries.
Bug: 183102797
Test: aapt2_tests
Change-Id: I851f9fb99a003769f8df8c1876997eee0864822a
For the SDK finalization changes, aapt2 must be able to handle
resources of the same type having different type ids. The
ResourceTable data structure currently stores package ids and type ids
on ResourceTablePackage and ResourceTableType respectively. This
prevents resource entries of the same type from having different type
ids without having to create another ResourceTableType structure.
JavaClassGenerator assumes each type only appears once in the
ResourceTable and it would need to dedupe the types to ensure one class
containing all the resource types ids is generated. TableFlattener on
the other hand needs a separate ResourceTableType for each type/id
combination so that the types are flattened into separate
ResTable_types.
This change simplifies aapt2's ResourceTable data structure:
- Resource ids are stored exclusively on ResourceEntry structures
meaning multiple entries can have different type ids while being
stored in the same ResourceTableType. Classes like JavaClassGenerator
can simply iterate over a type to see all the resources of the type
regardless of what their type id is.
- ResourceTable::GetPartitionedView() retrieves a list of resources
sorted and partitioned by package id, type id, and entry id. Classes
like TableFlattener can use this view to get separate
ResourceTavleTypes for each different type id that a type has.
These changes will also make it easy to have a resource span multiple
type ids if it exhausts all of the entry ids in one type id.
The new NewResourcesBuilder replaces the numerous setter methods on
ResourceTable.
Bug: 183102797
Test: aapt2_tests
Change-Id: I60dbcb24143bb958333899cafa7d41faa226d203
If a shared library exposes an attribute and a client uses the
attribute in its own styleable, the value of the lib attribute
resource id in the client styleable must be fixed with the correct
package id at runtime. Since the client will not have an
onResourcesLoaded to call, the client should directly reference the
attribute resource field in its styleable.
Bug: 147674078
Test: aapt2_tests
Change-Id: I8e64bb2d3165a7072e2604fe1730b248545978f4
When loading binary APKs into aapt2 for dumping purposes, aapt2 allows
the presence of custom resource types in the resources.arsc. Since
apapt2 may not be able to correctly interpret the data of custom types,
ignore resources from custom types completely when constructing the
ResourceTable data structure for the APK.
Bug: 168247188
Bug: 36051266
Test: run aapt2 dump bading/resources on APKs from both bugs
Change-Id: Ia3c0b4e07457eb8634b5a256ef77270e99b8bd72
Alongside SIGNATURE and ACTOR_SIGNATURE policies, add CONFIG_SIGNATURE
policy to overlayable that overlay fulfills if it is signed with the
same certificate as the reference package whose package name is
declared in 'config-signature' tag of SystemConfig and is vetted by
OMS that it's a system pre-installed package.
BUG: 158726924
TEST: regular aapt2, idmap2, OMS tests
Change-Id: I645ee72271496008742886274be0d63a2985201b
This cl enables aapt2 optimize and convert to handle collapsed resource
names optimization.
Test: make aapt2_test
Change-Id: I160d7e5bbd94580b52c00b648918e47beb4674f1
There are cases where an app can ship overlays for itself,
but the "signature" policy as described would open up
a vulnerability by allowing the system actor to create
and sign any arbitrary overlay that will apply to the target.
To prevent this, redefine "signature" as target package only,
and introduce "actor" for checking against the actor signature.
Any app that wishes to use both can include both policies.
Bug: 130563563
Test: m aapt2_tests idmapt2_tests and run from host test output
Test: atest libandroidfw_tests
Change-Id: I1c583a5b37f4abbeb18fc6a35c502377d8977a41
To make it easier to add the actor policy in a follow up CL,
move most of the policy handling to a central location.
The strings and transformation between strings and flags is
now handled in libidmap2policies, with libandroidfw
containing the single source of policy flags.
This also extracts all the test resource IDs into an R.h
so they can be swapped without having to edit a dozen files
each time.
Bug: 130563563
Test: m aapt2_tests idmapt2_tests and run from host test output
Test: atest libandroidfw_tests
Change-Id: Ie533c9cebf938215df7586f00c38763ae467e606
AssetManager2.cpp expects style attribute IDs to be in sorted order when
applying a style (see AssetManager2::GetBag). Shared libraries have a
package ID of 0x00, which will mean any attribute defined in a shared
library will be put before all other attributes. Once the attribute ID
is looked up in the dynamic ref table, the package ID is no longer 0x00,
which means this ID is no longer in sorted order. This messes up the
logic in AssetManager2::GetBag, and results in some style attributes
getting dropped from shared libraries.
This change modifies how aapt2 sorts the style entries, sorting entries
with dynamic IDs after entries with the android framework ID. This means
the entries will still be in sorted order when the IDs are looked up.
Bug: 147674078
Test: TableFlattenerTest.FlattenSharedLibraryWithStyle
Change-Id: Ic4f4004b6d9cecde9325dcdb37f71138857f8236
The is_dynamic bit on references was getting lost when converting to
proto and back. This was preventing bundles from being used as shared
libraries, since layout inflation would fail when it hit a dynamic
reference.
Bug: 146491000
Test: Build a shared library with a layout that has a dynamic reference,
and attempt to inflate that layout.
Change-Id: Ia0e615670d2ac52f9266e3eec8813af7687d3323
When the protobuf header (or data payload) size are a multiple of 4,
"entry_length" (called "aligned size" in the code) would have counted bytes that
are never actually written.
Bug: 139418052
Change-Id: Ia688a82a67f3807f7feb0be03670bf2827b1d6a1
This change allows RROs to reference their own internal resources as
expected.
Overlays are loaded as shared libraries so they can have their own
resource id space that does not conflict with the resource id space of
the target or other overlays.
References to overlay resources that override target resources now
appear as references to the target resources.
Overlay values that are inlined into the xml file specified using
android:overlayResources are now able to be used at runtime.
See go/rro-references for more information.
Bug: 135943783
Test: idmap2_tests
Test: libandroidfw_tests
Change-Id: Ie349c56d7fd3f7d94b7d595ed6d01dc6b59b6178
Removed the --whitelist-path flag, which is poorly named and we can
already specify these resources with --resources-config-path.
Renamed TableFlattenerOptions.whitelisted_resources to keep_resources.
It holds ResourceName instead of std::string. This lets us include type
when specifying what gets exempted, for correctness.
Bug: 111115201
Test: make aapt2_tests
Change-Id: Ifa5df924b5e2265c32cdcf8ca7dfa4a3992a0468
