Commit Graph

38 Commits

Author SHA1 Message Date
Ryan Mitchell
2e9bec1154 Add staging-public-group to aapt2
staging-public-group is a tag for putting resources that have been
added during platform development, but have not yet been finalized,
into a separate resource id namespace.

R.java fields of staged resources are non-final, so when the SDK is
finalized, applications using the android R.java will automatically
use the new finalized resource id without having to recompile.

Staged resources can exist either in the same type id as the type's
non-staged counterpart or in a separate type id. Multiple
staging-public-group tags each with a different type id can exist
simultaneously, which allows for multiple versions of the platform
to be developed at once.

Bug: 183411093
Test: aapt2_tests

Change-Id: Ibb6c84c3626751e33c6097f35a03e306bb85616a
2021-03-29 16:33:16 -07:00
Ryan Mitchell
9634efb6f4 Prepare aapt2 for multiple ids per type
For the SDK finalization changes, aapt2 must be able to handle
resources of the same type having different type ids. The
ResourceTable data structure currently stores package ids and type ids
on ResourceTablePackage and ResourceTableType respectively. This
prevents resource entries of the same type from having different type
ids without having to create another ResourceTableType structure.

JavaClassGenerator assumes each type only appears once in the
ResourceTable and it would need to dedupe the types to ensure one class
containing all the resource types ids is generated. TableFlattener on
the other hand needs a separate ResourceTableType for each type/id
combination so that the types are flattened into separate
ResTable_types.

This change simplifies aapt2's ResourceTable data structure:
- Resource ids are stored exclusively on ResourceEntry structures
  meaning multiple entries can have different type ids while being
  stored in the same ResourceTableType. Classes like JavaClassGenerator
  can simply iterate over a type to see all the resources of the type
  regardless of what their type id is.

- ResourceTable::GetPartitionedView() retrieves a list of resources
  sorted and partitioned by package id, type id, and entry id. Classes
  like TableFlattener can use this view to get separate
  ResourceTavleTypes for each different type id that a type has.

These changes will also make it easy to have a resource span multiple
type ids if it exhausts all of the entry ids in one type id.

The new NewResourcesBuilder replaces the numerous setter methods on
ResourceTable.

Bug: 183102797
Test: aapt2_tests
Change-Id: I60dbcb24143bb958333899cafa7d41faa226d203
2021-03-29 10:22:08 -07:00
Ryan Mitchell
5855de7ed6 Use R field directly for shared lib styleable attrs
If a shared library exposes an attribute and a client uses the
attribute in its own styleable, the value of the lib attribute
resource id in the client styleable must be fixed with the correct
package id at runtime. Since the client will not have an
onResourcesLoaded to call, the client should directly reference the
attribute resource field in its styleable.

Bug: 147674078
Test: aapt2_tests
Change-Id: I8e64bb2d3165a7072e2604fe1730b248545978f4
2021-02-24 15:57:34 -08:00
Ryan Mitchell
14e8ade9a9 Optimize FilterApkAssets by caching config
ResTable_config of every ResTable_type is read from device every time
AssetManager::RebuildFilterList is invoked. For large APKs (like
framework-res.apk), this causes a large number of page faults
when accessing the config from disk. The configs are also used in the
slow path of AssetManager::FindEntryInternal, which makes it even
slower. Instead cache the config on the TypeSpec of its ApkAsset.

Bug: 177247024
Test: libandroidfw_tests
Change-Id: I66d507c4eeb2399f7558f3d9dfc53c157129ada0
2021-01-13 21:14:31 -08:00
Ryan Mitchell
db21f09a8e Revert^2 "libandroidfw hardening for IncFs"
55ef6167a2

Change-Id: I02d4890d181655dfd0a14c188468db512559d27b
2020-11-17 23:01:35 +00:00
Ryan Mitchell
55ef6167a2 Revert "libandroidfw hardening for IncFs"
Revert "Move map_ptr to incfs namspace"

Revert submission 12787270

Reason for revert: b/173250495
Reverted Changes:
I5cd1bc8a2:libandroidfw hardening for IncFs
Ice5dbcfb2:Move map_ptr to incfs namspace
I29ccdc8ed:Do not cache bag parent stack until requested
I1e9e9acaa:Cache resolved theme values

Change-Id: Ib90ef68339710086df41e9abe0833a542d03a74f
2020-11-13 23:55:20 +00:00
Ryan Mitchell
c75c2e0922 libandroidfw hardening for IncFs
Migrate libandroifw to using incfs::util::map_ptr to prevent processes
from crashing when parsing the resources.arsc, parsing compiled xml,
files, and retrieving resource values.

This change propagates incremental failures to the JNI level where they
are raised as ResourcesNotFoundException.

Performance of ResourcesPerfWorkloads without change (time in
nanoseconds):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.883s)
    youtube_ns_median: 93812805
    youtube_ns_standardDeviation: 4387062
    youtube_ns_mean: 94455597
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.265s)
    maps_ns_standardDeviation: 2997543
    maps_ns_mean: 83480371
    maps_ns_median: 82210941
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.963s)
    gmail_ns_median: 266141091
    gmail_ns_standardDeviation: 3492043
    gmail_ns_mean: 267472765

With change and verification forcibly enabled for all apks
(including the framework-res.apk):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.646s)
    youtube_ns_median: 101999396
    youtube_ns_standardDeviation: 4625782
    youtube_ns_mean: 102631770
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.286s)
    maps_ns_standardDeviation: 2692088
    maps_ns_mean: 91326538
    maps_ns_median: 90519884
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.694s)
    gmail_ns_median: 290284442
    gmail_ns_standardDeviation: 5764632
    gmail_ns_mean: 291660464

With change and verification disabled:
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.748s)
    youtube_ns_median: 95490747
    youtube_ns_standardDeviation: 7282249
    youtube_ns_mean: 98442515
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (10.862s)
    maps_ns_standardDeviation: 4484213
    maps_ns_mean: 87912988
    maps_ns_median: 86325549
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.034s)
    gmail_ns_median: 282175838
    gmail_ns_standardDeviation: 6560876
    gmail_ns_mean: 282869146

These tests were done on a Pixel 3 and with cpu settings configured by
libs/hwui/tests/scripts/prep_generic.sh:

 Locked CPUs 4,5,6,7 to 1459200 / 2803200 KHz
 Disabled CPUs 0,1,2,3

Bug: 160635104
Bug: 169423204
Test: boot device && atest ResourcesPerfWorkloads

Change-Id: I5cd1bc8a2257bffaba6ca4a1c96f4e6640106866
2020-11-12 08:13:44 -08:00
Ryan Mitchell
ee4a564d4f Allow for overlaying dynamic shared libraries
Overlays targeting shared libraries should be loaded into the
resources of every target that depends on the shared library.

Static shared libraries are currently not supported because overlays
should override all versions of static shared libraries and there is
not currently support for an overlay targeting multiple APKs.

Also created a test instrumentation and host test suite for testing
overlays and packages on the system image.

Bug: 140790224
Test: atest OverlayRemountedTest
Change-Id: I20a217b6368d6cf92b2b9f46908fd58012933f72
2020-01-10 23:20:19 +00:00
Ryan Mitchell
cd78febeac Recognize dynamic res ids as valid
Shared libraries are assigned package id 0. Resource ids that start
with 0x00 are not invalid. This change changes is_valid to accept
dynamic resource ids and adds an is_valid_static method for when
an id must have a non-zero package id.

This also fixes an issue that made layouts in shared libraries that
use internal attributes exclude compiled resource ids from the binay
xml output.

Bug: 146491000
Test: Build a shared library with a layout that uses app attributes
      as well as android attribute and verify that all attributes
      have assigned resource ids using `aapt2 dump xmltree`

Change-Id: Ibc0407c610ffc98d7aaf233c37c065912ab0d516
2019-12-19 18:22:45 +00:00
Ryan Mitchell
c1676807f4 Retain parsed attribute type
If the value of an attribute enum is defined as a hexadecimal integer,
flatten uses of the attribute as with the
android::Res_value::TYPE_INT_HEX type.

This change adds a "type" field to pb::Attribute::Symbol, which if left
unset, will have a default value of android::Res_value::TYPE_INT_DEC
when deserialized by aapt2.

Bug: 124474141
Test: aapt2_tests and manual compilation of files and inspection using
`aapt2 dump chunks`
Change-Id: Ibf12394284fdbe3a8047f7ecf4fe68517dfc3abb
2019-06-06 14:19:26 +00:00
Fabien Sanglard
2d34e76dac Add --trace_folder to aapt2
Add a tracing API and instrument key functions in order to profile
aapt2 bottleneck. The API allows to generate systrace fragment files.

Impact on performance is neglibible with each Trace requiring less
than 1us and the final Flush operation at the end of a command
requiring around 40us.

Bug: None
Test: None
Change-Id: I51b564d3694e9384679f43b878b32295527dddf6
2019-03-05 15:09:27 -08:00
Ryan Mitchell
a55dc2ed6b Fix long paths on Windows
util::mkdirs iteratively creates each directory of a specified path. For
windows, Calling mkdir on only the drive letter or on the extended path
prefix (\?\\) will result in an error. Start after the long path prefix
and the drive letter.

This also changes AAPT2 to use AssetMaanager2 to retrieve symbols from
the symbol table. AssetManager2's zip library uses _wopen to open
windows files.

Bug:123251200
Test: aapt2_tests.exe
Change-Id: I26169d83b22d441485de3c49d63a6c4ed710e292
2019-02-20 09:46:00 -08:00
Mårten Kongstad
24c9aa6541 libandroidfw: move ConfigDescription from aapt2 to libandroidfw
This is to allow idmap2 to access ConfigDescription.

Test: libandroidfw_tests
Test: aapt2_tests
Change-Id: I54210bbbd8dad5903cb7100807df977efa394ad5
2018-10-08 06:50:22 -07:00
Treehugger Robot
c07d4e7118 Merge "Mark symbols as dynamic for shared libraries" 2018-08-20 21:31:28 +00:00
Todd Kennedy
196dbc379b Mark symbols as dynamic for shared libraries
Bug: 111425000
Change-Id: Ic1a68ac90d5e0443a536ee9ce75a72bc370cd673
Test: build, install and run the shared library test application
2018-07-24 21:55:12 +00:00
Todd Kennedy
325129964e Allow using reserved package IDs
Pre-O, the platform treats negative resource IDs [those with
a package ID of 0x80 or higher] as invalid.  In order to work
around this limitation, we allow the use of traditionally
reserved resource IDs [those between 0x02 and 0x7E].

Bug: 78041707
Test: ./out/host/linux-x86/nativetest64/aapt2_tests/aapt2_tests
Test: ./out/host/linux-x86/nativetest/libandroidfw_tests/libandroidfw_tests
Test: Manual. Create a feature split with a package ID of 0x7E and see it runs and can reference base resources.

Change-Id: I3d9782cc05d3a55e1a2467bf39566788847e1160
2018-04-26 10:27:30 -07:00
Adam Lesinski
73bff1e851 AAPT2: Allow compatible duplicate Attributes
If a resource XML file defines two compatible Attributes, they should
be merged without throwing an error. Ex:

<declare-styleable>
  <attr name="conflict" format="string" />
</declare-styleable>

<declare-styleable>
  <attr name="conflict" format="string|reference" />
</declare-styleable>

In this case, string|reference and string are the same, so these should
merge correctly.

Bug: 65699599
Test: make aapt2_tests
Test: make AaptBasicTest
Change-Id: I7b0f956d2332f7f0b458acd59ca0a606b2cfdf95
2018-01-11 13:54:11 -08:00
Adam Lesinski
71be70507d AAPT2: Propagate SPEC_OVERLAYABLE flag to final APK
Resources can be marked as overlayable, which means they can
be overlaid by runtime resource overlays.

This change propagates this state to the final resource table that
is installed on device.

Future work:
- Have the idmap tool respect the overlayable state and ignore
  entries that overlay anything else.

Bug: 64980941
Test: make aapt2_tests
Change-Id: Id45b1e141a281be2ee32a4ac3096fcf1114d523b
2017-12-18 14:16:02 -08:00
Adam Lesinski
dc785052d0 AAPT2: Enable mangled symbol lookup in AssetManagerSymbolTable
Bug: 70045583
Test: make AaptTestNamespace_Split
Test: make aapt2_tests
Change-Id: I25da9d58736fc9090d1527782391c9b2220d2f8d
2017-12-07 17:23:25 -08:00
Adam Lesinski
93190b79d1 AAPT2: Better debugging output
Test: make aapt2_tests
Change-Id: I7778b773201381538dc1f2e376abee4eb33e44c0
2017-11-07 11:21:10 -08:00
Adam Lesinski
c744ae8aca AAPT2: Implement attribute compat versioning
This change defines some hardcoded rules to degrade
attributes in newer SDKs to specific older attributes.

An attribute with a degrade rule will generate a new XML for the API
in which the attribute resulting from the degradation was introduced.

Since API 22 (Lollipop MR1), attributes are correctly ignored and do
not need to be versioned. In XML files defined for APIs 22+, the
original and degraded attributes coexist in the same XML file.

One such example is paddingHorizontal, introduced in API 26.
paddingHorizontal degrades to paddingLeft and paddingRight, which
were both introduced in API 1.

Bug: 35763493
Test: make aapt2_tests
Change-Id: I4aa8755a9ee2c0cc5afdc55c3d30093fd3a47f3d
2017-05-24 15:14:29 -07:00
Adam Lesinski
1e4b0e54a3 AAPT2: Workaround for findViewById with package ID > 0x7f
The entire View code base checks IDs against View.NO_ID except
findViewById(), which checks to see if the ID is negative.

Any package ID > 0x7f is interpreted as a negative number in Java
(no unsigned ints), so this check prevents the use of IDs > 0x7f.

findViewById is final, so support library workarounds are not possible.

Instead, IDs (@id/foo) are just sentinels, their values don't matter.
If building for pre-O devices, rewrite any references to these IDs of
the for 0xPPTTEEEE, where PP > 7f, to 0x7fPPEEEE.

The symbol table will check for potential collisions against the base
APK, so this should be safe.

Bug: 37498913
Test: manual
Change-Id: Ife3bbd29db287757ef8a2ffd83053d97f1db2613
2017-04-28 12:47:48 -07:00
Adam Lesinski
b5dc4bd49a AAPT2: Few tweaks to get shared-libraries working
Test: manual (building shared support library demo)
Change-Id: I4730645aa92ba1893baf67ffe35fbd4aac0f8e46
2017-02-22 19:54:43 -08:00
Adam Lesinski
ceb9b2f80f AAPT2: Shared library support
Test: make aapt2_tests
Change-Id: I98dddf1367e6c0ac425bb20be46e6ff05f4f2f45
2017-02-22 11:16:13 -08:00
Adam Lesinski
d5083f6f6b Move StringPiece to libandroidfw
libandroidfw needs to make use of StringPiece, so
move it to libandroidfw and update all code referencing
StringPiece in aapt2.

Test: make libandroidfw_tests libaapt2_tests
Change-Id: I68d7f0fc7c651b048d9d1f5e7971f10ef5349fa1
2017-01-17 18:55:51 -08:00
Adam Lesinski
ce5e56e243 AAPT2: Rename to match new style
Use Google3 naming style to match new
projects' and open source google projects' style.

Preferred to do this in a massive CL so as to avoid
style inconsistencies that plague legacy code bases.
This is a relatively NEW code base, may as well keep
it up to date.

Test: name/style refactor - existing tests pass
Change-Id: Ie80ecb78d46ec53efdfca2336bb57d96cbb7fb87
2016-10-26 19:30:23 -07:00
Adam Lesinski
cacb28f2d6 Use Google3 style guide with .clang-format
Test: style change only, builds ok
Change-Id: I885180e24cb2e7b58cfb4967c3bcb40058ce4078
2016-10-19 12:18:14 -07:00
Chih-Hung Hsieh
9b8528fee4 Fix clang-tidy warnings in aapt and aapt2.
* Add explicit keyword to conversion constructors.
* Add NOLINT(implicit) comments for implicit conversion constructors.
Bug: 28341362
* Use const reference type for read-only parameters.
Bug: 30407689
* Use const reference type to avoid unnecessary copy.
Bug: 30413862
Test: build with WITH_TIDY=1

Change-Id: Id6d21961f313a1ad92b15a37fdaa5be9e8ab48e1
2016-08-11 15:16:41 -07:00
Adam Lesinski
d0f116b619 AAPT2: Remove usage of u16string
For legacy reasons, we kept around the use of UTF-16 internally
in AAPT2. We don't need this and this CL removes all instances of
std::u16string and StringPiece16. The only places still needed
are when interacting with the ResTable APIs that only operate in
UTF16.

Change-Id: I492475b84bb9014fa13bf992cff447ee7a5fe588
2016-07-13 17:45:28 -07:00
Adam Lesinski
7656554f91 AAPT2: Add descriptions of Attributes in Styleables for R.java
Change-Id: I69e7b73cbdfe4baf502348397435c501ae29ff5e
2016-03-11 00:22:52 -08:00
Adam Lesinski
64587af817 AAPT2: Support building, linking, and merging static libraries
Android static libraries are like APKs but they contain much more debugging
and source information. We need to treat them differently in 3 ways:

1) When building a static library, we skip things like ID assignment and
   product/config stripping. Source information is emitted as well.
2) When building a static library and linking against another
   static library, we don't want to merge, we want to simply reference.
3) When building an app that uses static libraries, we want to merge
   the static library under the same package with or without mangling.

Bug:25958912
Change-Id: I425e032857936a3e83173c1edc2a6cdc6020b842
2016-03-04 14:39:01 -08:00
Adam Lesinski
e4bb9eb5af AAPT2: Introduce notion of 'product' to ResourceTable
This allows us to preserve the various product definitions during the compile
phase, and allows us to select the product in the link phase.

This allows compiled files to remain product-independent, so that they do not need
to be recompiled when switching targets.

Bug:25958912
Change-Id: Iaa7eed25c834b67a39cdc9be43613e8b5ab6cdd7
2016-02-12 22:21:48 -08:00
Adam Lesinski
a587065721 AAPT2: Verify min/max attr fields
Integers are now checked to see if they fall in the range
of min/max for the attribute they are assigned.

Change-Id: I42c435b15fd3f0bd23691c83efccce4ad5973276
2015-11-23 12:01:15 -08:00
Adam Lesinski
467f171315 AAPT2: Fail compiling when private symbols are referenced
Also moved some XML specific stuff into its own directory,
and refactored ReferenceLinker a bit.

Change-Id: I912247a82023c1bbf72dc191fbdaf62858cbec0c
2015-11-19 14:46:53 -08:00
Adam Lesinski
e352b990e1 AAPT2: Minor fixes to SymbolTable and diagnostic output
The SymbolTable lookup relied on the configuration, which we NEVER
want. Now we rely solely on the symbols defined in the ResTable
and no specific configuration or value of that symbol.

Also added some better source logging.

Change-Id: I983855c283493e924b2e92a9fd8e4cb841769349
2015-11-16 11:59:14 -08:00
Adam Lesinski
b274e35abf AAPT2: Fix inclusion of comments in R.java javadoc
Comments weren't being copied when merged from the various
resource tables.

Also refactored the JavaClassGenerator to omit a class
if no entries exist for it.

Change-Id: I6eaa89b7b3715bc05403635a2baf0d1db3efd142
2015-11-06 16:37:57 -08:00
Adam Lesinski
e78fd617ec AAPT2: Move comments and source into Value
Values are closely related to where they were defined, so
this information should live inside the Value.

This also enables comments to be attached to nested Values.

Change-Id: Ic7481b5a5f26d0ef248d638e2e29252f88154581
2015-10-22 16:35:47 -07:00
Adam Lesinski
1ab598f46c AAPT2: Separate out the various steps
An early refactor. Some ideas became clearer as
development continued. Now the various phases are much
clearer and more easily reusable.

Also added a ton of tests!

Change-Id: Ic8f0a70c8222370352e63533b329c40457c0903e
2015-10-16 12:13:08 -07:00