Commit Graph

252 Commits

Author SHA1 Message Date
Ryan Mitchell
2fedba9a32 Add <staging-public-group-final> to aapt2
To allow apps that compiled against a pre-release SDK to continue
working for a period of time after API finalization, a new tag,
<staging-public-group-final>, has been added to aapt2.

When finalizing the framework resource API, converting
<staging-public-group> tags to <staging-public-group-final> will
cause aapt2 to generate the resource table so that there is a resource
entry for the old non-finalized (staged) resource ID and another entry
for the finalized resource ID of newly finalized resources. This allows
an application that compiled against the pre-release SDK to continue
resolving resources using pre-release resource IDs.

All references to pre-release resource IDs will be rewritten to their
finalized resource IDs through the information stored in the new staged
alias chunk. This allows applications compiled against
<staging-public-group> resources to use the newly finalized
resource ID without re-compilation.

When an application is re-compiled against the SDK with
<staging-public-group-final> tags, the application will use the
finalized resource IDs.

This change limits the use of the alias chunk to the framework for S.

Bug: 183411356
Test: aapt2_test
Change-Id: Iba1c3033c3c2f32de8e4a19b58d3921c971092c4
2021-05-10 17:22:33 -07:00
Ryan Mitchell
326e35ffaf Add <macro> tag to aapt2
AAPT2 Macros are compile-time resources definitions that are expanded
when referenced during the link phase.

A macro must be defined in the res/values.xml directory. A macro
definition for a macro named "foo" looks like the following:
 <macro name="foo">contents</macro>

When "@macro/foo" is used in the res/values directory or in a compiled
XML file, the contents of the macro replace the macro reference and
then the substituted contents are compiled and linked. If the macro
contents reference xml namespaces from its original definition, the
namespaces of the original macro definition will be used to determine
which package is being referenced.

Macros can be used anywhere resources can be referenced using the
@package:type/entry syntax.

Macros are not included in the final resource table or the R.java since
they are not actual resources.

Bug: 175616308
Test: aapt2_tests
Change-Id: I48b29ab6564357b32b4b4e32bff7ef06036382bc
2021-04-28 14:58:23 -07:00
Ryan Mitchell
31031a91f1 Merge "Aapt2 ValueTransformer" into sc-dev 2021-04-21 17:44:55 +00:00
Ryan Mitchell
efcdb95fa7 Aapt2 ValueTransformer
For future macro support, aapt2 must be able to convert Reference
values into other Value types. Currently a DescendingValueVisitor is
used to visit all of the References in a ResourceTable or a compiled
XML file to set their resource ids during the link phase. This was fine
since we were only mutating the resource id of the visited Reference.

A macro may reference a String, BinaryPrimitive, or any other Item
type. During the link phase, we will need to transform references to
macros into the values of the macros.

The only parameter in the methods of the ValueVisitor interface is a
raw pointer to the type being visited. The visitor interface does not
support reassigning the visited type to a different type.

ValueTransformer is a new interface for consuming a Value type and
transforming it into a compatible Value type. This change refactors
Value::Clone to use this interface.

Bug: 175616308
Test: aapt2_tests
Change-Id: Ic1b9d718b932c208764114cd9c74d880e189ccb0
2021-04-21 08:32:30 -07:00
Ryan Mitchell
d4f0220d29 Merge "Use staging-public-group in framework SDK" into sc-dev 2021-04-01 18:36:33 +00:00
Ryan Mitchell
ca3b4f7633 Use staging-public-group in framework SDK
To make S finalization easier, this changes the framework SDK so that
apps linking against it will be able to continue working as expected
after the first phase of SDK finalization.

During the first phase of SDK finalization, the resource ids of
resources that have not been removed are finalized.
staging-public-group tags are converted to staging-public-group-final
tags in order to encode into the framework what the staged resource id
of a finalized resource was. When an app recompiles, it will use the
finalized resource id. Then after all apps recompile, phase 2 of
finalization begins, in which the staging-public-group-final tags are
removed so apps can no longer use the staged resource ids.

Apps that link against the SDK (provided they are using a recent
version of aapt) will encode references to staged resources as
TYPE_DYNAMIC_REFERENCE and TYPE_DYNAMIC_ATTRIBUTE. The values of R
fields for staged resources are defined out-of-line to prevent them
from being inlined into apps linking agsint the SDK. This allows the
resource ids to change during phase 1 of API finalization.

Bug: 183413192
Test: `aapt2 diff` and resource ids stayed the same
Test: `aapt2 dump` of framework-res.apk and observe staged resources
Change-Id: Ie2275c608297a5f63dde8b1cf795415112cbcc24
2021-03-31 23:04:57 -07:00
Ryan Mitchell
87d30ddcc0 Fix aapt2 feature split package name generation
Before flattening the resource table of a feature split, if the package
name of the feature split is the same as the base package name, the
split name is appended to the end of the feature's package name.

A bug was causing the rewritten package name to not be applied to the
feature package which resulted in the feature split being flattened
under the same package name as the base.

Bug: 184034454
Test: `m CtsSplitAppFeatureWarm_v23` && `aapt2 dump resources`
      to observe that package name is correct
Change-Id: I0a0ac57764f599c4dd05326a222056a3c52a4ae3
2021-03-31 22:09:04 +00:00
Ryan Mitchell
1499f504e4 Update iterators during aapt2 diff
A recent change broke aapt2 diff. We need to update the iterators when
an package, type, or entry is not missing from either APK.

Bug: 184066458
Test: `aapt2 diff` with framework before and after staging changes
Change-Id: I9138b9bd80aec24488feca6e06be1d7cf5775290
2021-03-30 14:13:07 -07:00
Ryan Mitchell
2e9bec1154 Add staging-public-group to aapt2
staging-public-group is a tag for putting resources that have been
added during platform development, but have not yet been finalized,
into a separate resource id namespace.

R.java fields of staged resources are non-final, so when the SDK is
finalized, applications using the android R.java will automatically
use the new finalized resource id without having to recompile.

Staged resources can exist either in the same type id as the type's
non-staged counterpart or in a separate type id. Multiple
staging-public-group tags each with a different type id can exist
simultaneously, which allows for multiple versions of the platform
to be developed at once.

Bug: 183411093
Test: aapt2_tests

Change-Id: Ibb6c84c3626751e33c6097f35a03e306bb85616a
2021-03-29 16:33:16 -07:00
Ryan Mitchell
1d008d1d2a Refactor aapt2 tests ResourceTable changes
This changes refactors tests to use the NewResourceBuilder class
that makes it easier to construct resource entries.

Bug: 183102797
Test: aapt2_tests
Change-Id: I851f9fb99a003769f8df8c1876997eee0864822a
2021-03-29 10:22:10 -07:00
Ryan Mitchell
9634efb6f4 Prepare aapt2 for multiple ids per type
For the SDK finalization changes, aapt2 must be able to handle
resources of the same type having different type ids. The
ResourceTable data structure currently stores package ids and type ids
on ResourceTablePackage and ResourceTableType respectively. This
prevents resource entries of the same type from having different type
ids without having to create another ResourceTableType structure.

JavaClassGenerator assumes each type only appears once in the
ResourceTable and it would need to dedupe the types to ensure one class
containing all the resource types ids is generated. TableFlattener on
the other hand needs a separate ResourceTableType for each type/id
combination so that the types are flattened into separate
ResTable_types.

This change simplifies aapt2's ResourceTable data structure:
- Resource ids are stored exclusively on ResourceEntry structures
  meaning multiple entries can have different type ids while being
  stored in the same ResourceTableType. Classes like JavaClassGenerator
  can simply iterate over a type to see all the resources of the type
  regardless of what their type id is.

- ResourceTable::GetPartitionedView() retrieves a list of resources
  sorted and partitioned by package id, type id, and entry id. Classes
  like TableFlattener can use this view to get separate
  ResourceTavleTypes for each different type id that a type has.

These changes will also make it easy to have a resource span multiple
type ids if it exhausts all of the entry ids in one type id.

The new NewResourcesBuilder replaces the numerous setter methods on
ResourceTable.

Bug: 183102797
Test: aapt2_tests
Change-Id: I60dbcb24143bb958333899cafa7d41faa226d203
2021-03-29 10:22:08 -07:00
Ryan Mitchell
5855de7ed6 Use R field directly for shared lib styleable attrs
If a shared library exposes an attribute and a client uses the
attribute in its own styleable, the value of the lib attribute
resource id in the client styleable must be fixed with the correct
package id at runtime. Since the client will not have an
onResourcesLoaded to call, the client should directly reference the
attribute resource field in its styleable.

Bug: 147674078
Test: aapt2_tests
Change-Id: I8e64bb2d3165a7072e2604fe1730b248545978f4
2021-02-24 15:57:34 -08:00
Rhed Jao
b9ccb8a49b aapt2: Add a new flag '--revision-code'
A new flag to inject revisionCode into the manifest for cts splits tests.

Bug: 174338944
Test: atest aapt2_tests
Change-Id: If5a089f37233f53af3012ca5eab17fab21eafd9c
2020-11-30 21:48:34 +08:00
Rhed Jao
61bcee0cd1 Merge "aapt2: Limit length of package name and shared user id" 2020-11-20 02:18:53 +00:00
Rhed Jao
2c4344223d aapt2: Limit length of package name and shared user id
Package name and shared user id could be used as part of filename
as prefix by other modules. Limits the length to 223 and reserves
32 for the OS.

Bug: 118768971
Test: atest aapt2_tests
Test: aapt2 link -I android.jar --manifest ManifestLongPackageName.xml
Test: aapt2 link -I android.jar --manifest ManifestLongSharedUserId.xml
Change-Id: Ic4b5b4647b9e253b79b663f4d7a9050f43bb8cf0
2020-11-19 15:34:14 +08:00
Ryan Mitchell
db21f09a8e Revert^2 "libandroidfw hardening for IncFs"
55ef6167a2

Change-Id: I02d4890d181655dfd0a14c188468db512559d27b
2020-11-17 23:01:35 +00:00
Ryan Mitchell
55ef6167a2 Revert "libandroidfw hardening for IncFs"
Revert "Move map_ptr to incfs namspace"

Revert submission 12787270

Reason for revert: b/173250495
Reverted Changes:
I5cd1bc8a2:libandroidfw hardening for IncFs
Ice5dbcfb2:Move map_ptr to incfs namspace
I29ccdc8ed:Do not cache bag parent stack until requested
I1e9e9acaa:Cache resolved theme values

Change-Id: Ib90ef68339710086df41e9abe0833a542d03a74f
2020-11-13 23:55:20 +00:00
Ryan Mitchell
c75c2e0922 libandroidfw hardening for IncFs
Migrate libandroifw to using incfs::util::map_ptr to prevent processes
from crashing when parsing the resources.arsc, parsing compiled xml,
files, and retrieving resource values.

This change propagates incremental failures to the JNI level where they
are raised as ResourcesNotFoundException.

Performance of ResourcesPerfWorkloads without change (time in
nanoseconds):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.883s)
    youtube_ns_median: 93812805
    youtube_ns_standardDeviation: 4387062
    youtube_ns_mean: 94455597
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.265s)
    maps_ns_standardDeviation: 2997543
    maps_ns_mean: 83480371
    maps_ns_median: 82210941
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.963s)
    gmail_ns_median: 266141091
    gmail_ns_standardDeviation: 3492043
    gmail_ns_mean: 267472765

With change and verification forcibly enabled for all apks
(including the framework-res.apk):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.646s)
    youtube_ns_median: 101999396
    youtube_ns_standardDeviation: 4625782
    youtube_ns_mean: 102631770
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.286s)
    maps_ns_standardDeviation: 2692088
    maps_ns_mean: 91326538
    maps_ns_median: 90519884
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.694s)
    gmail_ns_median: 290284442
    gmail_ns_standardDeviation: 5764632
    gmail_ns_mean: 291660464

With change and verification disabled:
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.748s)
    youtube_ns_median: 95490747
    youtube_ns_standardDeviation: 7282249
    youtube_ns_mean: 98442515
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (10.862s)
    maps_ns_standardDeviation: 4484213
    maps_ns_mean: 87912988
    maps_ns_median: 86325549
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.034s)
    gmail_ns_median: 282175838
    gmail_ns_standardDeviation: 6560876
    gmail_ns_mean: 282869146

These tests were done on a Pixel 3 and with cpu settings configured by
libs/hwui/tests/scripts/prep_generic.sh:

 Locked CPUs 4,5,6,7 to 1459200 / 2803200 KHz
 Disabled CPUs 0,1,2,3

Bug: 160635104
Bug: 169423204
Test: boot device && atest ResourcesPerfWorkloads

Change-Id: I5cd1bc8a2257bffaba6ca4a1c96f4e6640106866
2020-11-12 08:13:44 -08:00
Ryan Mitchell
bf511ddcea Add ".webp" to default compress extensions
The .webp extension contains compressed image data so do not deflate
.webp files in APKs.

Bug: 166137617
Test: aapt2 link
Change-Id: I70181437e0b51122c7aa5047bd0a0791ac3b8df6
2020-09-14 12:10:29 -07:00
Ryan Mitchell
0f60ef818f Merge "Only move private attrs when building framework" 2020-08-07 16:18:31 +00:00
Ryan Mitchell
4ea9075124 Make aapt2 terminology more inclusive
Update language to comply with Android's inclusive language guidance

See https://source.android.com/setup/contribute/respectful-code
for reference

Bug: 162364131
Bug: 162536543
Test: aapt2_tests
Change-Id: I4eef925d6319b40a395fc8d59a3f2912a02ca03b
2020-07-31 21:03:54 +00:00
Luke Edgar
3e810aec45 Merge "Add --source-path flag to AAPT2 compile" 2020-07-23 14:41:29 +00:00
lukeedgar
19b8ebf109 Add --source-path flag to AAPT2 compile
This added flag uses a given value to replace the default absolute resource
file path in the compiled resource file. This allows for relative file
paths to be used instead of absolute file paths.

Test: Compile_test.cpp
Bug: 159611599
Change-Id: I5f1c99cf40e5d31fc54cda819c91640285986d2a
2020-07-22 15:14:07 +00:00
TreeHugger Robot
aed7bf43db Merge "Don't add API annotations in the internal R.java" into rvc-dev-plus-aosp am: 7c72b7f45e am: 780aeb116a am: 47d47a8ac5
Original change: https://googleplex-android-review.googlesource.com/c/platform/frameworks/base/+/11955280

Change-Id: I602f8515391d93daf1d538f965659dfdca62e550
2020-06-23 17:11:14 +00:00
Makoto Onuki
de6e6f2098 Don't add API annotations in the internal R.java
I'm trying to enable a check for the following structure:
```
/** @hide */
public class Class1 {
    /** @hide */
    @SystemApi // Invalid because the class is hidden.
    public void method1() { }
}
```

The internal R.java file violates this, which this change is going to fix.

Bug: 159162473
Test: build (treehugger)
Test: atest aapt2_tests

Change-Id: I613e8611ddaf5f8e4761d351d4cd0142d59c7cc9
2020-06-22 16:33:53 -07:00
Ryan Mitchell
22dc5310a4 Only move private attrs when building framework
When aapt was changed to move private attributes into a different type
(9b624c186c), attributes only were moved
when building the framework. When this functionality was ported to
aapt2, all apps using the public tag would have their private
attributes moved. The change does not appear to be intentional and
thus is a parity issue.

This change only moves private attributes when building a package with
an id of 0x01.

Bug: 151683505
Test: manual
Change-Id: Ic9843f99e428f94cb6a55988a3881b8a91c7d7e2
2020-06-01 19:28:48 +00:00
Ryan Mitchell
d7465318e6 Merge "Do not rename R.java package" into rvc-dev am: 5c8dd641ba am: 98b076b85d am: 1da72bfb6c am: 9bb6fa9e89
Change-Id: I128110df8aa4c7f0d9abd144be3f4078e6d87ad6
2020-05-08 18:41:04 +00:00
Ryan Mitchell
5c8dd641ba Merge "Do not rename R.java package" into rvc-dev 2020-05-08 17:37:46 +00:00
Ryan Mitchell
2b7e847ac9 Do not rename R.java package
When the --rename-resources-package flag is used, only rename the
package name within the resources.arsc. Generate the R.java under
the package name defined by the manifest unless --custom-package is
used.

Bug: 147434671
Test: manual
Change-Id: Id9d10e16f32533da1b67ab72ac846791526e7a76
2020-05-06 11:26:48 -07:00
Automerger Merge Worker
fafe29f8b2 Merge "Merge "aapt2: Add command to rename overlay "targetPackage" attribute" into rvc-dev am: 6839fc74b6 am: 5c47bceb5d" into rvc-d1-dev-plus-aosp am: 88786a6f61 am: b1b587c7fb
Change-Id: If200647b976a4d562f7bc2fcc0cc3b2b97c0d485
2020-04-27 23:14:46 +00:00
Roshan Pius
ae12ce4119 aapt2: Add command to rename overlay "targetPackage" attribute
Bug: 154960712
Test: atest ManifestFixerTest
Change-Id: I556fc11e271337de40c70d4ed56a0381a2f79712
2020-04-27 21:49:23 +00:00
TreeHugger Robot
76750266cd Merge "Fix aapt2 Maybe check" 2020-03-20 03:23:36 +00:00
Ryan Mitchell
60b74fb42f Add --rename-resources-package to aapt2
There is currently no way to change the package name encoded in the
resources table. This change adds a flag that controls rewriting the
package name encoded into the resources table. This flag does not
rewrite the package name in the manifest xml.

Bug: 147434671
Test: manual
Change-Id: I435ba07e29df0522c3a44cc5f2c460004a02d71c
2020-02-26 21:35:47 +00:00
Ryan Mitchell
30cd9b2a9a Fix aapt2 Maybe check
Fixed a bug in AliasAdaptiveIcon that occurs when building static
libraries.

Bug: 148455934
Test: build SysUIComposeLib && manual parity test
Change-Id: I6f29630ac3a92d431115885b1369f0541c5044cd
2020-02-20 12:06:49 -08:00
Ryan Mitchell
ee4a564d4f Allow for overlaying dynamic shared libraries
Overlays targeting shared libraries should be loaded into the
resources of every target that depends on the shared library.

Static shared libraries are currently not supported because overlays
should override all versions of static shared libraries and there is
not currently support for an overlay targeting multiple APKs.

Also created a test instrumentation and host test suite for testing
overlays and packages on the system image.

Bug: 140790224
Test: atest OverlayRemountedTest
Change-Id: I20a217b6368d6cf92b2b9f46908fd58012933f72
2020-01-10 23:20:19 +00:00
Donald Chai
94e4a01dec Add option to avoid clobbering visibility of <declare-styleable>
"aapt2 compile" marks styleables as public instead of preserving
information from <public/> or --visibility options.  This behavior can
now be disabled via --preserve-visibility-of-styleables.

Bug: 146649511
Change-Id: Ifb8ab396573d1393df737a59625e79e9cf2494a7
Tested: aapt2_tests
2020-01-07 12:55:52 -08:00
Jean-Luc Coelho
181cbfde80 Add "no-proguard-location-reference" flag to aapt2
This fixes relocatability issues with the code shrinkers.

Bug: 144529940
Test: None
Change-Id: I65e08e850193e67dcdf4c533d9e66332714f0c8c
2019-12-02 21:11:08 +00:00
Izabela Orlowska
5b89b2d320 Use case-sensitive regular expressions in --no-compress-regex
On some windows versions (plus wine) std::regex has issues with the
icase flag (making regex case insensitive). Put the responsibility of
the case sensitivity on the caller.

Fixes: 127793905
Test: wine aapt2_tests.exe -> Util_test
Change-Id: I6775514d3ef43c2c47465bf39ceaf2c195909724
2019-11-27 18:37:09 +00:00
Ryan Mitchell
e5981c075a Disable test broken on windows
RegularExperssions is broken on Windows, disable this test while a
fix is being worked on.

Bug: 127793905
Test: wine aapt2.exe
Change-Id: Ic78a7f12b004d2e8dbf5d79c5b52f00ca0ceb9e9
2019-11-26 18:28:01 +00:00
Mårten Kongstad
1d3b64855b aapt2: add 'dump overlayable' command
Add a command to print a resource table's <overlayable> resources. Given
the following input

  <overlayable name="TestResources">
    <policy type="system">
      <item type="string" name="a" />
    </policy>
    <policy type="sytem|vendor">
      <item type="string" name="b" />
      <item type="string" name="c" />
    </policy>
  </overlayable>

aapt2 dump overlayable will produce

  name="TestResources" actor=""
    policies="system"
      string/a
    policies="system|vendor"
      string/b
      string/c

Bug: 120609160
Test: manual (aapt2 dump overlayable $(gettop)/frameworks/base/cmds/idmap2/tests/data/target/target.apk)
Change-Id: I21041e6169c62d01f1a469624911ce7cad3e18a8
2019-09-18 07:35:44 -07:00
Brian Chang
3d1cde3fce Merge "Improve how we exempt resources from getting their names collapsed." 2019-09-17 17:31:18 +00:00
Brian Chang
dcef831e61 Improve how we exempt resources from getting their names collapsed.
Removed the --whitelist-path flag, which is poorly named and we can
already specify these resources with --resources-config-path.

Renamed TableFlattenerOptions.whitelisted_resources to keep_resources.
It holds ResourceName instead of std::string. This lets us include type
when specifying what gets exempted, for correctness.

Bug: 111115201
Test: make aapt2_tests
Change-Id: Ifa5df924b5e2265c32cdcf8ca7dfa4a3992a0468
2019-09-16 17:29:46 -07:00
Brian Chang
d882ba40db Rename flags in aapt2-optimize for more accuracy and consistency.
--enable-resource-path-shortening => --shorten-resource-paths
--enable-resource-obfuscation => --collapse-resource-names

Bug: 140883718
Test: Verified by running `aapt2 optimize
Change-Id: I697713f49b27ccf5f152c76e256234c041dc5f51
2019-09-16 13:09:56 -07:00
TreeHugger Robot
b13c6bbdf8 Merge "Allow compile output to be either a zip or dir" 2019-09-05 13:32:30 +00:00
Ryan Mitchell
c7db244cbe Allow compile output to be either a zip or dir
When passing a directory to compile using --zip or --dir, the -o flag
had to represent a zip file the compiled contents would be written to.
Now, if the path specified by the -o flag is a directory, the compiled
files will be written to the directory rather than erroring out.

Also when passing files to compile as file arguments, the -o flag had to
represent an existing directory. Now, if the path is not an existing
directory, the compiled files will create a zip file at the path that
contains the compiled files.

Test: manual
Change-Id: I92a8e124d53cdb653eb3a7ec549f09f9ad0ef04f
2019-08-28 11:21:47 -07:00
Ryan Mitchell
2814793744 Merge "Add --exclude-sources to AAPT2 link" 2019-08-14 21:29:16 +00:00
Ryan Mitchell
ef9e688393 Add --exclude-sources to AAPT2 link
Information about where resources are defined can make having
reproducible builds more difficult, makes the generates resources.pb
large, and can exposes details about the machines that compiled the
resources.

The --exclude-sources flags can only be used when building a proto APK
and prevents source information from being included in the generated
resources protobuf.

Bug: 134929532
Test: checked debug string with and without the string
Change-Id: Ia345f067fe781ea82a4bcad37eb55576c72c44d7
2019-08-14 19:28:32 +00:00
Donald Chai
d554f7c597 Fix uninitialized variable.
"aapt2 optimize" without --enable-resource-path-shortening should not do
resource-path-shortening.

Bug: 75965637
Change-Id: Ibcbf325e66ed7992bf81184260abd63711b2af2a
2019-07-10 18:50:15 -07:00
Udam Saini
1710e8370e Merge "Allows features to link to other feature splits without namespacing." 2019-06-27 16:05:22 +00:00
TreeHugger Robot
5e4c18b05c Merge "AAPT2: Merge-only mode for building static libraries" 2019-06-27 12:01:35 +00:00