Commit Graph

57 Commits

Author SHA1 Message Date
Ryan Mitchell
326e35ffaf Add <macro> tag to aapt2
AAPT2 Macros are compile-time resources definitions that are expanded
when referenced during the link phase.

A macro must be defined in the res/values.xml directory. A macro
definition for a macro named "foo" looks like the following:
 <macro name="foo">contents</macro>

When "@macro/foo" is used in the res/values directory or in a compiled
XML file, the contents of the macro replace the macro reference and
then the substituted contents are compiled and linked. If the macro
contents reference xml namespaces from its original definition, the
namespaces of the original macro definition will be used to determine
which package is being referenced.

Macros can be used anywhere resources can be referenced using the
@package:type/entry syntax.

Macros are not included in the final resource table or the R.java since
they are not actual resources.

Bug: 175616308
Test: aapt2_tests
Change-Id: I48b29ab6564357b32b4b4e32bff7ef06036382bc
2021-04-28 14:58:23 -07:00
Ryan Mitchell
db21f09a8e Revert^2 "libandroidfw hardening for IncFs"
55ef6167a2

Change-Id: I02d4890d181655dfd0a14c188468db512559d27b
2020-11-17 23:01:35 +00:00
Ryan Mitchell
55ef6167a2 Revert "libandroidfw hardening for IncFs"
Revert "Move map_ptr to incfs namspace"

Revert submission 12787270

Reason for revert: b/173250495
Reverted Changes:
I5cd1bc8a2:libandroidfw hardening for IncFs
Ice5dbcfb2:Move map_ptr to incfs namspace
I29ccdc8ed:Do not cache bag parent stack until requested
I1e9e9acaa:Cache resolved theme values

Change-Id: Ib90ef68339710086df41e9abe0833a542d03a74f
2020-11-13 23:55:20 +00:00
Ryan Mitchell
c75c2e0922 libandroidfw hardening for IncFs
Migrate libandroifw to using incfs::util::map_ptr to prevent processes
from crashing when parsing the resources.arsc, parsing compiled xml,
files, and retrieving resource values.

This change propagates incremental failures to the JNI level where they
are raised as ResourcesNotFoundException.

Performance of ResourcesPerfWorkloads without change (time in
nanoseconds):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.883s)
    youtube_ns_median: 93812805
    youtube_ns_standardDeviation: 4387062
    youtube_ns_mean: 94455597
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.265s)
    maps_ns_standardDeviation: 2997543
    maps_ns_mean: 83480371
    maps_ns_median: 82210941
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.963s)
    gmail_ns_median: 266141091
    gmail_ns_standardDeviation: 3492043
    gmail_ns_mean: 267472765

With change and verification forcibly enabled for all apks
(including the framework-res.apk):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.646s)
    youtube_ns_median: 101999396
    youtube_ns_standardDeviation: 4625782
    youtube_ns_mean: 102631770
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.286s)
    maps_ns_standardDeviation: 2692088
    maps_ns_mean: 91326538
    maps_ns_median: 90519884
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.694s)
    gmail_ns_median: 290284442
    gmail_ns_standardDeviation: 5764632
    gmail_ns_mean: 291660464

With change and verification disabled:
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.748s)
    youtube_ns_median: 95490747
    youtube_ns_standardDeviation: 7282249
    youtube_ns_mean: 98442515
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (10.862s)
    maps_ns_standardDeviation: 4484213
    maps_ns_mean: 87912988
    maps_ns_median: 86325549
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.034s)
    gmail_ns_median: 282175838
    gmail_ns_standardDeviation: 6560876
    gmail_ns_mean: 282869146

These tests were done on a Pixel 3 and with cpu settings configured by
libs/hwui/tests/scripts/prep_generic.sh:

 Locked CPUs 4,5,6,7 to 1459200 / 2803200 KHz
 Disabled CPUs 0,1,2,3

Bug: 160635104
Bug: 169423204
Test: boot device && atest ResourcesPerfWorkloads

Change-Id: I5cd1bc8a2257bffaba6ca4a1c96f4e6640106866
2020-11-12 08:13:44 -08:00
Ryan Mitchell
4ea9075124 Make aapt2 terminology more inclusive
Update language to comply with Android's inclusive language guidance

See https://source.android.com/setup/contribute/respectful-code
for reference

Bug: 162364131
Bug: 162536543
Test: aapt2_tests
Change-Id: I4eef925d6319b40a395fc8d59a3f2912a02ca03b
2020-07-31 21:03:54 +00:00
Clark DuVall
e9fedbe223 Fix serializing dynamic references to proto
The is_dynamic bit on references was getting lost when converting to
proto and back. This was preventing bundles from being used as shared
libraries, since layout inflation would fail when it hit a dynamic
reference.

Bug: 146491000
Test: Build a shared library with a layout that has a dynamic reference,
      and attempt to inflate that layout.

Change-Id: Ia0e615670d2ac52f9266e3eec8813af7687d3323
2020-01-13 09:55:53 -08:00
Ryan Mitchell
cd78febeac Recognize dynamic res ids as valid
Shared libraries are assigned package id 0. Resource ids that start
with 0x00 are not invalid. This change changes is_valid to accept
dynamic resource ids and adds an is_valid_static method for when
an id must have a non-zero package id.

This also fixes an issue that made layouts in shared libraries that
use internal attributes exclude compiled resource ids from the binay
xml output.

Bug: 146491000
Test: Build a shared library with a layout that uses app attributes
      as well as android attribute and verify that all attributes
      have assigned resource ids using `aapt2 dump xmltree`

Change-Id: Ibc0407c610ffc98d7aaf233c37c065912ab0d516
2019-12-19 18:22:45 +00:00
Donald Chai
4abc82859d Fix deserialization of @id aliases
I didn't realize that this was actually a thing, but:

  <resources>
    <item name="id1" type="id"></item>
    <item name="id2" type="id">@id/id1</item>
  </resources>

leads to a resource table which 'aapt dump resources' will render as:

    resource 0x7f030000 com.pkg:id/id1: t=0x03 d=0x00000001 (s=0x0008 r=0x00)
    resource 0x7f030001 com.pkg:id/id2: t=0x01 d=0x7f030000 (s=0x0008 r=0x00)

while 'aapt2 dump resources' printed:

  type id id=03 entryCount=2
    resource 0x7f030000 id/id1
      () (id)
    resource 0x7f030001 id/id2
      () (id)

After this change, it prints:

  type id id=03 entryCount=2
    resource 0x7f030000 id/id1
      () (id)
    resource 0x7f030001 id/id2
      () @id/id1

Bug: 69445910
Change-Id: I0001ff09345434577ac4c1d32f96a781f4190d8d
Tested: manual, see above
2019-12-14 07:29:01 +00:00
Ryan Mitchell
c1676807f4 Retain parsed attribute type
If the value of an attribute enum is defined as a hexadecimal integer,
flatten uses of the attribute as with the
android::Res_value::TYPE_INT_HEX type.

This change adds a "type" field to pb::Attribute::Symbol, which if left
unset, will have a default value of android::Res_value::TYPE_INT_DEC
when deserialized by aapt2.

Bug: 124474141
Test: aapt2_tests and manual compilation of files and inspection using
`aapt2 dump chunks`
Change-Id: Ibf12394284fdbe3a8047f7ecf4fe68517dfc3abb
2019-06-06 14:19:26 +00:00
Ryan Mitchell
48002f6313 qt-dev-plus-aosp designated for R. Update MAX_PLATFORM_VERSION, etc.
Test: aapt2_tests
Bug: 128934651
Change-Id: I179f7b92dcf65e77f039e6cd42d91bf5b1461e35
2019-04-11 14:29:25 -07:00
Ryan Mitchell
0f32675909 Do not convert whitespace chars above max char to regular space
Using isspace and iswspace on characters above the maximum char value is
undefined. This change makes aapt2 use isspace like aapt and only trims
whitespace characters at or below the maximum char value like aapt2.

Bug: 121017795
Test: aapt2_tests
Change-Id: I015e4b77f0ff53e409e880fcf9ae104ba3444d1a
2019-03-11 11:00:25 -07:00
Ryan Mitchell
1d358ff5bb Fix aapt2 whitespace diffs from aapt(1)
CDATA blocks were being processed differently in aapt2 so this change
fixes aapt2 to not treat cdata blocks differently and still trime
whitespace.

Also, aapt did not process escapes when compiling xml files. This
change removes over-processing of xml text nodes.

All test strings are what aapt(1) would output.

Test: aapt2_tests
Bug: 124470332
Change-Id: I90ee0c1e5e9208f8a5c60cee93e3ba02712c9b2c
2019-03-06 15:06:49 -08:00
Ryan Mitchell
a55dc2ed6b Fix long paths on Windows
util::mkdirs iteratively creates each directory of a specified path. For
windows, Calling mkdir on only the drive letter or on the extended path
prefix (\?\\) will result in an error. Start after the long path prefix
and the drive letter.

This also changes AAPT2 to use AssetMaanager2 to retrieve symbols from
the symbol table. AssetManager2's zip library uses _wopen to open
windows files.

Bug:123251200
Test: aapt2_tests.exe
Change-Id: I26169d83b22d441485de3c49d63a6c4ed710e292
2019-02-20 09:46:00 -08:00
Ryan Mitchell
90b7a08aaf Revert "Fix loaded apk string pool order"
This reverts commit 4e9a922ede.

Reason for revert: <b/122518436>

Change-Id: I3650b2c6c9bdfa69a3034f9ca49e95a9698c3cdd
2019-02-15 14:37:45 -08:00
Nicholas Lativy
79f0396898 AAPT2: Add support for parsing codeNames with fingerprints.
In addition to supporting manifest declared codenames of the form
"[codename]", also support codenames of the form "[codename].[fingerprint]".

Matches the behaviour of PackageParser as of ag/6056697.

Test: ResourceUtils_test
Change-Id: I814330eba9d383e4549e35da791fcfa9bd0cdf57
2019-01-16 16:49:31 +00:00
Chih-Hung Hsieh
2491af24a8 Merge "Fix performance-for-range-copy warnings" am: bc29242288 am: 7adb73f332
am: 2296036a40

Change-Id: Ia99ea6c9633986e68340bc88983d06baa7a1c597
2018-12-17 14:31:14 -08:00
Chih-Hung Hsieh
a1b644e88c Fix performance-for-range-copy warnings
Bug: 30413223
Test: make with WITH_TIDY=1 DEFAULT_GLOBAL_TIDY_CHECKS=-*,performance*
Change-Id: Ie481e88025a7a1f3abde8ff63420d5ccd8577e52
2018-12-17 20:00:55 +00:00
Ryan Mitchell
4e9a922ede Fix loaded apk string pool order
Loading in an APk changed the ordering of strings in the string pool.
When loading an apk, assign the strings to the same index as they
are in the ResStringPool.

Bug: 118831219
Test: "aapt2 dump strings left.apk" prints in the correct order,
"aapt2 convert left.apk --output-format binary -o left_binary.apk" has
entries in the correct order, and aapt2_tests

Change-Id: I00014c02195f39c1152a110e90083d9b14e9216e
2018-11-15 11:37:01 -08:00
Mårten Kongstad
5c541f6e36 libandroidfw: move ConfigDescription from aapt2 to libandroidfw
This is to allow idmap2 to access ConfigDescription.

Test: libandroidfw_tests
Test: aapt2_tests
Change-Id: I54210bbbd8dad5903cb7100807df977efa394ad5
Merged-In: I54210bbbd8dad5903cb7100807df977efa394ad5
2018-10-08 09:56:09 -07:00
Mårten Kongstad
24c9aa6541 libandroidfw: move ConfigDescription from aapt2 to libandroidfw
This is to allow idmap2 to access ConfigDescription.

Test: libandroidfw_tests
Test: aapt2_tests
Change-Id: I54210bbbd8dad5903cb7100807df977efa394ad5
2018-10-08 06:50:22 -07:00
Ryan Mitchell
0ce89734d1 Fix failing Windows tests
Some tests were not written to run on Windows correctly. Compile also
has a bug that caused using the --zip flag to fail on Windows.
iswspace does count the non breaking space as Whiespace on Windows but
not on Unix based systems

Bug: 117156986
Change-Id: I999375162bdfdf86fb25992ee88e2962ab90b577
Test: aapt2_tests and wine aapt2_tests.exe
2018-10-03 11:35:37 -07:00
Ryan Mitchell
cb76d734fe AAPT2: Fix unrecognized CDATA
This change adds support for resources that have CDATA blocks within
their values. The blocks should allow any character to occur without
being escaped. It also should not effect the current state of quote
processing.

Bug: 80326349
Test: Created tests in aapt2_tests
Change-Id: Ie1a00e50cffc877e2eb5f788f8d7a1bda839c0cf
2018-06-22 14:06:02 -07:00
Ryan Mitchell
9beaa9cfe3 Changed AAPT2 to abide by AAPT resource whitespace triming.
Bug: b/74331008
Test: Created tests in ResourceParser_test.cpp
Change-Id: Id7b387692b795774cd77452ca8cf06a8447bf3be
(cherry picked from commit a048807712)
2018-04-04 15:54:05 +00:00
Ryan Mitchell
a048807712 Changed AAPT2 to abide by AAPT resource whitespace triming.
Bug: b/74331008
Test: Created tests in ResourceParser_test.cpp
Change-Id: Id7b387692b795774cd77452ca8cf06a8447bf3be
2018-03-29 09:05:40 -07:00
Adam Lesinski
2eed52ecc0 AAPT2: Fix styled string whitespace processing
Change styled string whitespace processing to be like AAPT's was.

Main changes:
- whitespace around tags is preserved.
- tags start exactly where they are supposed to, not off by one.

Bug: 72406283
Test: make aapt2_tests
Change-Id: I4d12728c493efd8c978e2e3d2718b56534ff52ef
2018-02-27 11:39:10 -08:00
Shane Farmer
d05b9130fd AAPT2: Fix compatible-screens element in AndroidManifest.xml
Fix an issue where the compatible-screens element was not being
populated correctly. The previous version was missing the screenSize
attribute which is mandatory. The attributes were also missing the
resource ID from the framework library as these are a part of the public
API.

Change-Id: I2f594c2259831dbbd96c58db4ba55e8288d4231e
Test: unit tests
Test: manually split an APK and dumped with aapt
2018-02-14 15:44:42 -08:00
Pierre Lecesne
70fdf76c52 AAPT2 - Make raw files of type unknown.
Bug: 69791607
Test: Manual.

Change-Id: I6ee5ef37fb447ab79d46b57e0dc4eaad84eec723
2017-11-27 19:33:47 +00:00
Adam Lesinski
e59f0d80ec AAPT2: Enable building proto artifacts
- This will allow the bundle tool to ingest the outputs of the aapt2 link
  phase.

Bug: 64143208
Test: manual
Change-Id: I1a4b3ce5c2ffbbdc4bc642c3371a9ef2e7e9ad71
2017-10-19 13:22:48 -07:00
Adam Lesinski
4670805ea4 AAPT2: Move format related files under same directory
Test: make aapt2_tests
Change-Id: Id72cdfc12ba3add294048e60c55f2461344464bf
2017-10-04 14:26:55 -07:00
Adam Lesinski
060b53d028 AAPT2: Ensure style strings are always first in StringPool
Move the styled strings to a separate section of the StringPool so
that sorting can never mess up the order of Styles.

Bug: 63570514
Test: make aapt2_tests
Change-Id: Id2ce1355b92be1bb31ce0daa7e54ae9b5b6c2ffe
2017-07-31 17:46:28 -07:00
Adam Lesinski
8a3bffea49 AAPT2: Fix issues with parsing integers + floats
If whitespace was present at the end of an integer or float, parsing
would fail.

Bug: 62902869
Test: make aapt2_tests
Change-Id: I6c54f25ad73913d8ea90969fca9de24f726deb96
2017-06-27 12:45:00 -07:00
Adam Lesinski
bab4ef56d7 AAPT2: Allow undefined resources (placeholders)
A resource defined like so:

<item type="drawable" name="foo" />

should be assigned the value @null.

The only exception is for <string> resources, which are given the
empty string value (since <string></string> is ambiguous). The decision
to use "" is based off the fact that old AAPT used to assign "" to all
undefined resources, even non-string ones.

Bug: 38425050
Test: make aapt2_tests
Change-Id: Ib3e0f6f83d16ddd8b279c9fd44a07a37867b85e9
2017-06-02 16:48:38 -07:00
Adam Lesinski
5924d8c9ab AAPT2: Allow merging of Style attributes from overlays
Previously style overlays would completely override an existing style.
To be compatible with AAPT, styles now merge with the overlay, allowing
the overlay's attributes and parent to take precedence.

Bug: 38355988
Test: make aapt2_tests
Change-Id: Id25c7240050a43e6a4a177c6e3d51e048d0cceb5
2017-05-31 10:09:06 -07:00
Adam Lesinski
d0f492db03 AAPT2: Share split functionality between link and optimize
Generating splits should be possible to do from the optimize command.
This means that a lot of infrastructure around split APKs can be
shared by both the optimize and link phase.

Bug: 35925830
Change-Id: Ia88b9e4bff300a56353b2f7a4a2547c8eb43a299
Test: manual
2017-04-10 11:59:18 -07:00
Adam Lesinski
b5dc4bd49a AAPT2: Few tweaks to get shared-libraries working
Test: manual (building shared support library demo)
Change-Id: I4730645aa92ba1893baf67ffe35fbd4aac0f8e46
2017-02-22 19:54:43 -08:00
Adam Lesinski
ceb9b2f80f AAPT2: Shared library support
Test: make aapt2_tests
Change-Id: I98dddf1367e6c0ac425bb20be46e6ff05f4f2f45
2017-02-22 11:16:13 -08:00
Adam Lesinski
929d6517df AssetManager2: Add GetResourceId
Add ability to lookup a resource by name.

Test: make libandroidfw_tests
Change-Id: I262ba5ce4c9892458226fbdb44cf21f9877fb92d
2017-02-08 06:04:52 -08:00
Adam Lesinski
d5083f6f6b Move StringPiece to libandroidfw
libandroidfw needs to make use of StringPiece, so
move it to libandroidfw and update all code referencing
StringPiece in aapt2.

Test: make libandroidfw_tests libaapt2_tests
Change-Id: I68d7f0fc7c651b048d9d1f5e7971f10ef5349fa1
2017-01-17 18:55:51 -08:00
Adam Lesinski
ce5e56e243 AAPT2: Rename to match new style
Use Google3 naming style to match new
projects' and open source google projects' style.

Preferred to do this in a massive CL so as to avoid
style inconsistencies that plague legacy code bases.
This is a relatively NEW code base, may as well keep
it up to date.

Test: name/style refactor - existing tests pass
Change-Id: Ie80ecb78d46ec53efdfca2336bb57d96cbb7fb87
2016-10-26 19:30:23 -07:00
Adam Lesinski
cacb28f2d6 Use Google3 style guide with .clang-format
Test: style change only, builds ok
Change-Id: I885180e24cb2e7b58cfb4967c3bcb40058ce4078
2016-10-19 12:18:14 -07:00
Adam Lesinski
5eeaaddffd AAPT2: Add Inline Complex XML support
See: https://developer.android.com/guide/topics/resources/complex-xml-resources.html

Change-Id: I8274c85e25cabf90423141c228697e873167d136
2016-08-31 18:32:34 -07:00
TreeHugger Robot
1d740812e6 Merge "AAPT2: Expose split support to command line" 2016-08-15 20:46:38 +00:00
Adam Lesinski
36c73a5959 AAPT2: Expose split support to command line
Bug:30445078
Change-Id: If4b8530dba71b9059b8e62c04757da99c1119d22
2016-08-15 12:14:26 -07:00
Chih-Hung Hsieh
9b8528fee4 Fix clang-tidy warnings in aapt and aapt2.
* Add explicit keyword to conversion constructors.
* Add NOLINT(implicit) comments for implicit conversion constructors.
Bug: 28341362
* Use const reference type for read-only parameters.
Bug: 30407689
* Use const reference type to avoid unnecessary copy.
Bug: 30413862
Test: build with WITH_TIDY=1

Change-Id: Id6d21961f313a1ad92b15a37fdaa5be9e8ab48e1
2016-08-11 15:16:41 -07:00
Adam Lesinski
bf0bd0f9ac AAPT2: Add support to specify stable IDs
The --stable-ids flag allows the user to specify a file containing
a list of resource name and resource ID pairs in the form of:

package:type/name = 0xPPTTEEEE

This assigns the given resource the specified ID. It helps ensure
that when adding or removing resources, IDs are assigned in a stable
fashion.

If a package, type, or name is not found, no error or warning is
raised.

Change-Id: Ibc2f4e05cc924be255fedd862d835cb5b18d7584
2016-07-25 16:02:18 -07:00
Adam Lesinski
d0f116b619 AAPT2: Remove usage of u16string
For legacy reasons, we kept around the use of UTF-16 internally
in AAPT2. We don't need this and this CL removes all instances of
std::u16string and StringPiece16. The only places still needed
are when interacting with the ResTable APIs that only operate in
UTF16.

Change-Id: I492475b84bb9014fa13bf992cff447ee7a5fe588
2016-07-13 17:45:28 -07:00
Adam Lesinski
fb6312fe93 AAPT2: Add version collapsing
When an app specifies (or imports) resources with various
configurations for different SDK versions, specifying
a minSdk will make many of those resources unreachable.

Version collapsing will prune out the resources specified
for SDK versions less than the minSdk.

If, however, there is no exact matching resource for the
minSdk version, the next smallest SDK version is kept.

Change-Id: Ic7bcab6c59d65c97c67c8767358abb57cdec60a4
2016-06-29 17:02:19 -07:00
Adam Lesinski
458b877488 AAPT2: Add diff command
Adds the diff command and various small fixes to issues
discovered when diffing old AAPT built APKs with new AAPT2
built APKS.

Bug:22775504
Change-Id: I682a7fe1cf4b3efa7cbd5d18b333cf2d1046fe1b
2016-04-30 01:36:12 +00:00
Adam Lesinski
59e04c6f92 AAPT2: Switch to protobuf for intermediate format
Without needing to conform to the runtime data format,
it is much easier to add new features such as debugging symbols
and carrying over product data to link time.

This also simplifies the runtime format parser and serializer,
which will change much less frequently than the protobuf intermediate
format.

Change-Id: I209787bbf087db0a58a534cb8511c51d21133e00
2016-02-09 19:59:17 +00:00
Adam Lesinski
52364f7ae3 AAPT2: Variety of small fixes to get the build working
- Add option to rename package in AndroidManifest.xml
- Support default versionName and versionCode
- Accept True and False as valid booleans

Change-Id: I400e350b9dcd0fd1c197d1929144299c7823617d
2016-01-13 18:48:24 -08:00