Commit Graph

36 Commits

Author SHA1 Message Date
Ryan Mitchell
55ef6167a2 Revert "libandroidfw hardening for IncFs"
Revert "Move map_ptr to incfs namspace"

Revert submission 12787270

Reason for revert: b/173250495
Reverted Changes:
I5cd1bc8a2:libandroidfw hardening for IncFs
Ice5dbcfb2:Move map_ptr to incfs namspace
I29ccdc8ed:Do not cache bag parent stack until requested
I1e9e9acaa:Cache resolved theme values

Change-Id: Ib90ef68339710086df41e9abe0833a542d03a74f
2020-11-13 23:55:20 +00:00
Ryan Mitchell
c75c2e0922 libandroidfw hardening for IncFs
Migrate libandroifw to using incfs::util::map_ptr to prevent processes
from crashing when parsing the resources.arsc, parsing compiled xml,
files, and retrieving resource values.

This change propagates incremental failures to the JNI level where they
are raised as ResourcesNotFoundException.

Performance of ResourcesPerfWorkloads without change (time in
nanoseconds):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.883s)
    youtube_ns_median: 93812805
    youtube_ns_standardDeviation: 4387062
    youtube_ns_mean: 94455597
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.265s)
    maps_ns_standardDeviation: 2997543
    maps_ns_mean: 83480371
    maps_ns_median: 82210941
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.963s)
    gmail_ns_median: 266141091
    gmail_ns_standardDeviation: 3492043
    gmail_ns_mean: 267472765

With change and verification forcibly enabled for all apks
(including the framework-res.apk):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.646s)
    youtube_ns_median: 101999396
    youtube_ns_standardDeviation: 4625782
    youtube_ns_mean: 102631770
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.286s)
    maps_ns_standardDeviation: 2692088
    maps_ns_mean: 91326538
    maps_ns_median: 90519884
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.694s)
    gmail_ns_median: 290284442
    gmail_ns_standardDeviation: 5764632
    gmail_ns_mean: 291660464

With change and verification disabled:
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.748s)
    youtube_ns_median: 95490747
    youtube_ns_standardDeviation: 7282249
    youtube_ns_mean: 98442515
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (10.862s)
    maps_ns_standardDeviation: 4484213
    maps_ns_mean: 87912988
    maps_ns_median: 86325549
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.034s)
    gmail_ns_median: 282175838
    gmail_ns_standardDeviation: 6560876
    gmail_ns_mean: 282869146

These tests were done on a Pixel 3 and with cpu settings configured by
libs/hwui/tests/scripts/prep_generic.sh:

 Locked CPUs 4,5,6,7 to 1459200 / 2803200 KHz
 Disabled CPUs 0,1,2,3

Bug: 160635104
Bug: 169423204
Test: boot device && atest ResourcesPerfWorkloads

Change-Id: I5cd1bc8a2257bffaba6ca4a1c96f4e6640106866
2020-11-12 08:13:44 -08:00
Ryan Mitchell
a707013b78 Add policies and enforce overlayable to header
If the fulfilled policies change without the contents of the target
and overlay APKs changing, the idmap for the overlay should be
regenerated. This change adds fulfilled policies and enforce
overlayable to the idmap header so that idmap2d can determine if the
polices or enforce overlayable changed from what was used to generate
the idmap.

Bug: 119328308
Test: idmap2_tests
Test: atest RegenerateIdmapTest

Change-Id: I96f970e82b5243be01b205ac2cb6ab249c6100bc
2020-06-01 16:15:35 -07:00
Winson
62ac8b56a9 Refactor overlayable policy
To make it easier to add the actor policy in a follow up CL,
move most of the policy handling to a central location.

The strings and transformation between strings and flags is
now handled in libidmap2policies, with libandroidfw
containing the single source of policy flags.

This also extracts all the test resource IDs into an R.h
so they can be swapped without having to edit a dozen files
each time.

Bug: 130563563

Test: m aapt2_tests idmapt2_tests and run from host test output
Test: atest libandroidfw_tests

Change-Id: Ie533c9cebf938215df7586f00c38763ae467e606
2020-02-26 15:59:43 -08:00
Mårten Kongstad
1d3b64855b aapt2: add 'dump overlayable' command
Add a command to print a resource table's <overlayable> resources. Given
the following input

  <overlayable name="TestResources">
    <policy type="system">
      <item type="string" name="a" />
    </policy>
    <policy type="sytem|vendor">
      <item type="string" name="b" />
      <item type="string" name="c" />
    </policy>
  </overlayable>

aapt2 dump overlayable will produce

  name="TestResources" actor=""
    policies="system"
      string/a
    policies="system|vendor"
      string/b
      string/c

Bug: 120609160
Test: manual (aapt2 dump overlayable $(gettop)/frameworks/base/cmds/idmap2/tests/data/target/target.apk)
Change-Id: I21041e6169c62d01f1a469624911ce7cad3e18a8
2019-09-18 07:35:44 -07:00
Donald Chai
e83c451060 Merge "[aapt2] Close empty lists in "dump resources"." 2019-05-08 20:51:10 +00:00
Donald Chai
42fe2b2ac7 [aapt2] Close empty lists in "dump resources".
Bug: 129134933
Test: manual (b/129134933#comment4) since there's no Debug_test.cpp
Change-Id: Ib31481457089eade6f1d380cff4eeba0ff0c499d
2019-05-07 21:08:36 -07:00
Ryan Mitchell
0c0cedff00 Check value in dump before printing
For applications that remove the names of resources from the string
pool, check that the attribute has a name before attempting to print it.

Test: manual
Bug: 130553900
Change-Id: I05e5d59f01b2c02c8a024d06fd896074d6bf465b
2019-04-16 12:00:30 -07:00
Ryan Mitchell
90b7a08aaf Revert "Fix loaded apk string pool order"
This reverts commit 4e9a922ede.

Reason for revert: <b/122518436>

Change-Id: I3650b2c6c9bdfa69a3034f9ca49e95a9698c3cdd
2019-02-15 14:37:45 -08:00
Chih-Hung Hsieh
3b3be3f10f Merge "Fix/suppress aapt/aapt2 google-explicit-constructor warnings" am: cdf6be573f am: d42fe48c93
am: 0bf62f1309

Change-Id: Iffb7e4a145b172a35dd58b9450d4a5184d8b07f7
2019-01-04 12:20:01 -08:00
Chih-Hung Hsieh
1fc78e1814 Fix/suppress aapt/aapt2 google-explicit-constructor warnings
* Add explicit to conversion constructors/operators
* Use NOLINT or NOLINTNEXTLINE to suppress warnings on intended converters

Bug: 28341362
Test: make with WITH_TIDY=1 DEFAULT_GLOBAL_TIDY_CHECKS=-*,google-explicit-constructor
Change-Id: Ie02101ea7c422e8add535c111a30a2f21ead0ace
2019-01-04 17:38:13 +00:00
Ryan Mitchell
1bb1fe068a Refactor policy parsing
This change removes the ability for an overlayable resource to be
defined in multiple policy blocks within the same overlayable. This
change also changes aapt2 to use a bit mask to keep track of the parsed
policies.

Bug: 110869880
Bug: 120298168
Test: aapt2_tests
Change-Id: Ie26cd913f94a16c0b312f222bccfa48f62feceaa
2018-12-11 13:48:45 -08:00
Ryan Mitchell
4e9a922ede Fix loaded apk string pool order
Loading in an APk changed the ordering of strings in the string pool.
When loading an apk, assign the strings to the same index as they
are in the ResStringPool.

Bug: 118831219
Test: "aapt2 dump strings left.apk" prints in the correct order,
"aapt2 convert left.apk --output-format binary -o left_binary.apk" has
entries in the correct order, and aapt2_tests

Change-Id: I00014c02195f39c1152a110e90083d9b14e9216e
2018-11-15 11:37:01 -08:00
Ryan Mitchell
e4e989ccba RRO: Added partition policies for overlays
<overlayable> tags can now have policy elements that indicate which
partition the overlay apk must reside on in order to be allowed to
overlay a resource. This change only adds parsing of <policy> and
encoding of policy in the proto ResourceTable. A later change will add
the encoding of policy and overlayable in the binary APK.

<overlayable>
  <policy type="system|vendor|product|product_services|public" >
    <item type="string" name="oof" />
  </policy>
</overlayable>

Bug: 110869880
Test: make aapt2_tests
Change-Id: I8d4ed7b0e01f981149c6e3190af1681073b79b03
2018-11-06 00:10:26 +00:00
Ryan Mitchell
5d2755129d AAPT2: Reformatted dump command invocations
Use with:
  aapt2 dump apc [apc]
  aapt2 dump configurations [apk]
  aapt2 dump strings [apk]
  aapt2 dump resources [apk]
  aapt2 dump xmlstrings [apk] --file [file]
  aapt2 dump xmltree [apk] --file [file]

Will add permissions and badging in a later commit.

Bug: 73351292
Test: Manual tests of the commands
Change-Id: I97eec01222af14053a98bd70255f1bfecd16b1c4
2018-08-14 15:56:29 -07:00
Adam Lesinski
bbf429795d AAPT2: Fix issue with deserializing binary XML
We assumed that a raw text value set for an attribute meant there
were no compiled values set either.

This would only really happen for attributes that did not belong to any
namespace (no prefix:), since we always kept their raw string values
in case some code relied on it.

Bug: 72700446
Test: make aapt2_tests
Change-Id: Icba40a1d4b181bfe7cad73131c4dbe5ba7f8b085
2018-02-14 16:11:23 -08:00
Adam Lesinski
da9eba300b Dump XML files in aapt2 dump
Test: manual
Change-Id: Id7f656c1385f6d1fec8eaa993c8e2b36bb1ae5b0
2018-02-14 10:49:32 -08:00
Adam Lesinski
71be70507d AAPT2: Propagate SPEC_OVERLAYABLE flag to final APK
Resources can be marked as overlayable, which means they can
be overlaid by runtime resource overlays.

This change propagates this state to the final resource table that
is installed on device.

Future work:
- Have the idmap tool respect the overlayable state and ignore
  entries that overlay anything else.

Bug: 64980941
Test: make aapt2_tests
Change-Id: Id45b1e141a281be2ee32a4ac3096fcf1114d523b
2017-12-18 14:16:02 -08:00
Adam Lesinski
5a217b052d AAPT2: Fix debugging output
Test: manual
Change-Id: I3a362caf7bd691df399db00c6280e0a208a406da
2017-11-16 16:58:02 -08:00
Adam Lesinski
93190b79d1 AAPT2: Better debugging output
Test: make aapt2_tests
Change-Id: I7778b773201381538dc1f2e376abee4eb33e44c0
2017-11-07 11:21:10 -08:00
Adam Lesinski
e59f0d80ec AAPT2: Enable building proto artifacts
- This will allow the bundle tool to ingest the outputs of the aapt2 link
  phase.

Bug: 64143208
Test: manual
Change-Id: I1a4b3ce5c2ffbbdc4bc642c3371a9ef2e7e9ad71
2017-10-19 13:22:48 -07:00
Adam Lesinski
d3ffa844f5 AAPT2: Cleanup Visitors for XML and Values
Test: make aapt2_tests
Change-Id: Ib61f64c155a380115610edeaf2d65e60258a2426
2017-09-28 14:08:50 -07:00
Shane Farmer
6ed4061abf AAPT2: Debug::DumpXml display compiled attribute values.
Test: manually dumped a compiled  AndroidManifest.xml file.

Change-Id: I14f022fd1f95693b21d07b7b8c1886f23738242a
2017-09-06 10:21:02 -07:00
Adam Lesinski
6b37299129 AAPT2: Change XmlDom to exclude Namespace as a node
In preparation for exporting an XML proto format for UAM to consume,
this change brings the XML DOM API more in line with other APIs that
do not make the Namespace a separate node.

Treating Namespace declarations as just properties of an Element
node makes the implementation of algorithms much simpler, as
the constraints that Namespace nodes have only one child
are now built in and traversing to find Element nodes
is much simpler.

Also made a bunch of quality of life improvements, like formatting and
comment style.

Test: make aapt2_tests
Change-Id: Ib97ff1c4252b7907e2cc1f13a448dc4ca3b809a4
2017-08-17 17:43:08 +00:00
Adam Lesinski
4ca56978a9 AAPT2: Add workaround for non-standard package IDs
The dynamic ref table used to map build-time IDs to runtime IDs
is mainly used for shared resource libraries and has a few built-in
mappings (app 0x7f and framework 0x01).

Using a non-standard package ID like 0x80 causes a failure in package ID
lookup. The solution is to ship the dynamic_ref_table with an identity mapping
with any resource table that uses a non-standard package ID.

Adds some tests to ensure this works correctly.

Bug: 37498913
Test: make libandroidfw_tests
Test: make aapt2_tests
Change-Id: Ic3f67942384d34e7fdcbc94ded360e940e3ebc8a
2017-04-26 21:55:31 -07:00
Adam Lesinski
ce5e56e243 AAPT2: Rename to match new style
Use Google3 naming style to match new
projects' and open source google projects' style.

Preferred to do this in a massive CL so as to avoid
style inconsistencies that plague legacy code bases.
This is a relatively NEW code base, may as well keep
it up to date.

Test: name/style refactor - existing tests pass
Change-Id: Ie80ecb78d46ec53efdfca2336bb57d96cbb7fb87
2016-10-26 19:30:23 -07:00
Adam Lesinski
cacb28f2d6 Use Google3 style guide with .clang-format
Test: style change only, builds ok
Change-Id: I885180e24cb2e7b58cfb4967c3bcb40058ce4078
2016-10-19 12:18:14 -07:00
Adam Lesinski
5eeaaddffd AAPT2: Add Inline Complex XML support
See: https://developer.android.com/guide/topics/resources/complex-xml-resources.html

Change-Id: I8274c85e25cabf90423141c228697e873167d136
2016-08-31 18:32:34 -07:00
Adam Lesinski
355f285ffd AAPT2: Implement density stripping and initial Split support
When a preferred density is supplied, the closest matching densities
will be selected, the rest stripped from the APK.

Split support will be enabled in a later CL. Command line support is still
needed, but the foundation is ready.

Bug:25958912
Change-Id: I56d599806b4ec4ffa24e17aad48d47130ca05c08
2016-02-17 18:17:25 -08:00
Adam Lesinski
e4bb9eb5af AAPT2: Introduce notion of 'product' to ResourceTable
This allows us to preserve the various product definitions during the compile
phase, and allows us to select the product in the link phase.

This allows compiled files to remain product-independent, so that they do not need
to be recompiled when switching targets.

Bug:25958912
Change-Id: Iaa7eed25c834b67a39cdc9be43613e8b5ab6cdd7
2016-02-12 22:21:48 -08:00
Adam Lesinski
52364f7ae3 AAPT2: Variety of small fixes to get the build working
- Add option to rename package in AndroidManifest.xml
- Support default versionName and versionCode
- Accept True and False as valid booleans

Change-Id: I400e350b9dcd0fd1c197d1929144299c7823617d
2016-01-13 18:48:24 -08:00
Adam Lesinski
24b8ff0faf AAPT2: Fix references to private parent
Change-Id: Id4697551b6c8cb6167f562de593006ae3c6158c0
2015-12-17 11:01:55 -08:00
Adam Lesinski
9e10ac7015 AAPT2: Process <java-symbols> and private symbol package
Need to introduce the idea of multiple levels of visibility to support <java-symbol>.

Public, Private, Undefined.

Public means it is accessible from outside and requires an ID assigned.
Private means that we explicitly want this to be a symbol (show up in R.java), but not visible
to other packages. No ID required.

Undefined is any normal resource. When --private-symbols is specified in the link phase,
these resources will not show up in R.java.

Change-Id: Icba89221e08e685dee7683786aa7112baf28c856
2015-10-22 12:52:49 -07:00
Adam Lesinski
1ab598f46c AAPT2: Separate out the various steps
An early refactor. Some ideas became clearer as
development continued. Now the various phases are much
clearer and more easily reusable.

Also added a ton of tests!

Change-Id: Ic8f0a70c8222370352e63533b329c40457c0903e
2015-10-16 12:13:08 -07:00
Adam Lesinski
d13fb24986 AAPT2: Debug: Dump only targetted style
Change-Id: Id7c5a4b5d0880520e1fea05e5a31d398946c5f05
2015-05-12 20:45:20 -07:00
Adam Lesinski
330edcdf13 AAPT2: Support static lib referencing static lib
When a static library A references static library B,
and app C references both A and B, we get the following symbol merging,
symbols from library B get imported twice.

We must only check that symbol references to library B are valid
when building library A. We should only merge all the symbols
when building final app C.

Change-Id: I23cba33b0901dcbb5328d9c9dfaa6a979c073c36
2015-05-12 20:41:31 -07:00