Commit Graph

105 Commits

Author SHA1 Message Date
Adam Lesinski
a9743826bd IDMAP: Warn when RROs overlay non-overlayable resources
First pass at getting overlays to respect which resources should be
overlaid. First step is to call it out but not enforce.

Bug: 64980941
Test: manual (inspect the warnings at boot)
Change-Id: I40baee1110d4bc1e54e7f2f9d3b73ffabb067f90
2017-12-18 17:25:27 -08:00
Adam Lesinski
498f6053da libandroidfw: Remove pre-verification
This added more up-front cost to loading an APK and didn't provide
a significant benefit to resource retrieval.

Test: make libandroidfw_tests
Change-Id: Idbf993abc433fa8c8950d106c66469b310b66f7f
2017-12-05 00:29:38 +00:00
Adam Lesinski
73f6f9daf6 libandroidfw: Add SparseEntry support for LoadedArsc
go/o-restable-sparse-entries

Test: make libandroidfw_tests
Change-Id: Ib1a7d1fc69008390eee53a1de04356dc50e05b45
2017-11-29 22:19:13 +00:00
Adam Lesinski
1c855a0bc8 libandroidfw: Do not clear last resource id in ResolveReference
If the value passed to AssetManager::ResolveReference is not a
reference, the caller may be expecting for the last reference to
not be cleared, as a more appropriate value should most likely be
retained.

This was causing an issue when a caller was manually resolving
references and expecting the last resource ID resolved to be propagated
across calls to ResolveReference.

Test: make libandroidfw_tests
Change-Id: I5b7f586e2cd541059023eaa9ba23e324a21a9a1e
2017-11-29 10:20:26 -08:00
Adam Lesinski
441500b23f libandroidfw: Support loading ApkAssets from a file descriptor
Test: make aapt2_tests
Change-Id: I041f9e9e3d3f6a10684cbd8baa49f4dda7d6dc40
2017-11-13 17:53:45 -08:00
Roozbeh Pournader
bb9965612c Merge "Parse BCP 47 locale names with Unicode extensions" 2017-10-31 18:28:32 +00:00
Igor Viarheichyk
e7bc60a954 Parse BCP 47 locale names with Unicode extensions
Locale parser is modified to handle BCP 47 locale extensions and
recognize numbering system specifications. This fixes incorrect
interpretation of an extension key as a region code.

Numbering system specification is stored in the configuration data,
however it's not used for best configuration selection yet.

Bug: 67508876
Test: build and run libandroidfw_tests

Change-Id: If4072b9f39e80f60d4567841afe9ce7a92979574
2017-10-26 18:04:43 -07:00
Adam Lesinski
1a1e9c2724 AssetManager2: Run ApkAssets that have failed verification
ApkAssets who have failed verification should still run for
compatibility. Not all resources are accessed, and therefore
errors in the APK are not necessarily fatal. However, this means
we must do bounds checks when retrieving resources, which is
slower.

Test: make libandroidfw_tests && $ANDROID_BUILD_TOP/out/host/<host>/nativetest64/libandroidfw_tests/libandroidfw_tests
Test: make libandroidfw_benchmarks && adb sync system && adb sync data && /data/benchmarktest64/libandroidfw_benchmarks/libandroidfw_benchmarks
Change-Id: I4cc926c064bca0491785d82cdac0419d74d7d9b0
2017-10-17 16:03:30 -07:00
Adam Lesinski
30080e2f28 AssetManager2: Improve Theme performance
This change brings Theme ApplyStyle down to 2x the original performance
and Theme attribute retrieval to less than the original performance.
Yay!

Benchmarks ran on marlin-eng
----------------------------------------------------------------------
Benchmark                               Time           CPU Iterations
----------------------------------------------------------------------
BM_ThemeApplyStyleFramework          8540 ns       8500 ns      82105
BM_ThemeApplyStyleFrameworkOld       5280 ns       5258 ns     148849
BM_ThemeGetAttribute                    8 ns          8 ns   88388549
BM_ThemeGetAttributeOld                11 ns         11 ns   63394463

ApplyStyle still takes some time, and the weird thing is that if I
switch the data structure of ThemeType to use an
std::vector<ThemeEntry>, the performance becomes better than the
original implementation! The issue is that std::vector<T> takes up 24
bytes, which would make Themes take up 8 more bytes per ThemeType, which
is unacceptable. Still trying to isolate where the performance gain is
coming from.

Test: make libandroidfw_tests && $ANDROID_BUILD_TOP/out/host/<host>/nativetest64/libandroidfw_tests/libandroidfw_tests
Test: make libandroidfw_benchmarks && adb sync system && adb sync data && adb shell /data/benchmarktest64/libandroidfw_benchmarks/libandroidfw_benchmarks
Change-Id: I0e7a756afd44b6aac1521e69c2b907258c262d3e
2017-10-17 11:37:49 -07:00
Adam Lesinski
970bd8d283 AssetManager2: Implement IDMAP support
This enables RRO (runtime resource overlays) with AssetManager2

Test: make libandroidfw_tests
Test: out/host/<platform>/nativetest64/libandroidfw_tests/libandroidfw_tests --testdata=frameworks/base/libs/androidfw/tests/data
Change-Id: Id8079104faefbfaa3f4017d8f7ee1a8968f151a2
2017-10-13 10:23:34 -07:00
Adam Lesinski
873ef0e230 libandroidfw_tests: package test data correctly
- Convert to use soong, which allows bundling test data with benchmarks.
- Also separate libandroidfw_benchmarks so that it doesn't depend
on gtest.

Test: manual
Change-Id: I45bd222fafa87172c77b7f5bb2a7a89ddca72361
2017-10-12 16:02:00 -07:00
Adam Lesinski
03ebac8c68 A few fixes to AssetManager2 for compat
Theme copying should behave the way it did with the old AssetManager
(copy only the framework attributes when copying from a Theme object
from a different AssetManager).

Cleanup the dependencies on libziparchive in ApkAssets.

Test: make libandroidfw_tests
Test: out/host/<platform>/nativetests64/libandroidfw_tests/libandroidfw_tests --testdata=frameworks/base/libs/androidfw/tests/data
Change-Id: I973f7e6eb14ce311306e2ec66a623a4790c8d233
2017-09-25 18:24:41 -07:00
Adam Lesinski
32e7501a27 Fix support for @empty in style resolution
If @empty is encountered in XML, do not fallback
to searching through the theme.

Bug: 36891052
Test: make aapt2_tests
Test: bit CtsContentTestCases:android.content.res.cts.TypedArrayTest
Change-Id: Ie3bf7b70af9c7913513a1092afd95d26bec5e635
2017-05-11 11:28:29 -07:00
Adam Lesinski
ed69ce84bd libandroidfw: Fix mass logspam of ResourceTypes warnings
An overlay was incorrectly leaking its own resources into the
framework resource package, which caused warnings for every app
that tried to access framework resources (all of them).

This change skips including any resources that are not overlaying
anything (not present in IDMAP).

Bug: 36256974
Test: make libandroidfw_tests
Change-Id: I8c710af6849bb848938825aacca02799ee96c003
2017-03-20 14:51:13 -07:00
Adam Lesinski
0e25d9ae77 libandroidfw: Search all packages for an identifier
In order to allow multiple packages with the same package name, but
different package ID, we need to keep searching packages until the
resource is found.

Bug: 30999713
Test: make libandroidfw_tests
Change-Id: If4540e12731ca18a18e9e550a9bf248606a586c5
2017-03-16 15:45:16 -07:00
Adam Lesinski
edb88a2b94 Merge "Add ResTable_sparseTypeEntry support" 2017-02-18 00:39:18 +00:00
Adam Lesinski
c8f71aa67e Add ResTable_sparseTypeEntry support
Benchmarks on bullhead-userdebug show that there is a negligent
performance impact when using sparse entries on a 30% loaded
sparse type of 1000 resources.

Benchmark                                             Time           CPU Iterations
-----------------------------------------------------------------------------------
BM_SparseEntryGetResourceSparseLarge                255 ns        254 ns    2751408
BM_SparseEntryGetResourceNotSparseLarge             254 ns        254 ns    2756534

Bug: 27381711
Test: make libandroidfw_tests aapt2_tests
Change-Id: I051ea22f2f6b2bc3696e446adc9e2a34be18009f
2017-02-15 12:40:02 -08:00
Adam Lesinski
d1ecd7af68 AssetManager2: Various fixes
- Use FileMaps to open Assets (prevents closing of ApkAssets underlying
zip)
- Implement OpenDir and List methods
- Fix issue where DynamicRefTable wasn't properly constructed

Test: make libandroidfw_tests
Change-Id: Ib21a84e1114d028120744aa3bc1c6eb9d9399fa8
2017-02-15 10:50:23 -08:00
Adam Lesinski
929d6517df AssetManager2: Add GetResourceId
Add ability to lookup a resource by name.

Test: make libandroidfw_tests
Change-Id: I262ba5ce4c9892458226fbdb44cf21f9877fb92d
2017-02-08 06:04:52 -08:00
Adam Lesinski
0c40524953 AssetManager2: Add other support methods
- Add GetResourceConfigurations()
- Add GetResourceLocales()
- Add ResolveReference()
- Add stub for GetResourceId()
- Change LoadedArsc and ApkAssets factory method to return const

Test: make libandroidfw_tests
Change-Id: Ia797dc9381a523b1a3e7029048a413e544730379
2017-01-31 16:20:29 -08:00
Adam Lesinski
c6aada9c57 LoadedArsc: Support feature splits.
Test: make libandroidfw_tests
Change-Id: I278273e688da597f4af86dd55f87750501ef8154
2017-01-27 16:52:53 -08:00
Adam Lesinski
90572a4ab8 Merge "libandroidfw: Add new support for shared libraries" 2017-01-27 22:30:24 +00:00
Romain Guy
4832745b84 Add color mode to activity/window
The color mode lets an application request a wide color gamut for
a specific window. This will also be used in the future to request
HDR. The color mode is currently either default (sRGB) or an undefined
wide gamut color space chosen by the platform. These attributes could
later be used to choose a specific color space if we deem this important
or useful.

This change also renames the various "colorimetry" attributes and
constants to "color mode" for consistency. These symbols were
added in O and can be safely renamed.

Test: CtsColorModeTestCases
Bug: 32984164
Change-Id: I4d4691dd12dbe3f3aa6a5cf893cff39aa16c739e
2017-01-24 15:55:09 -08:00
Romain Guy
c9ba559021 Add wide color gamut and HDR resource qualifiers
Bug: 32984164
Test: Config_test, AaptConfig_test and aapt2_tests
Change-Id: Ie9c82bfe2d36b1d6180ee223250ab5bb2ce90dd4
2017-01-19 18:44:00 -08:00
TreeHugger Robot
1f9f11007d Merge "Move StringPiece to libandroidfw" 2017-01-19 23:40:06 +00:00
Adam Lesinski
d5083f6f6b Move StringPiece to libandroidfw
libandroidfw needs to make use of StringPiece, so
move it to libandroidfw and update all code referencing
StringPiece in aapt2.

Test: make libandroidfw_tests libaapt2_tests
Change-Id: I68d7f0fc7c651b048d9d1f5e7971f10ef5349fa1
2017-01-17 18:55:51 -08:00
Adam Lesinski
fdaa9c58c9 Merge "New implementation of AssetManager/ResTable" 2017-01-17 23:58:36 +00:00
Roozbeh Pournader
383ac409df Merge "Treat Latin American locales specially" 2017-01-13 01:53:15 +00:00
Adam Lesinski
da431a22da libandroidfw: Add new support for shared libraries
This adds support for shared resource libraries in the new
ResTable/AssetManager implementation.

The dynamic package map encoded in resources.arsc is parsed
and stored with LoadedArsc, and combined to form a resolved table
in AssetManager2.

Benchmarks show that this implementation is an order of magnitude
faster on angler-userdebug (make libandroidfw_benchmarks).

Test: libandroidfw_tests
Change-Id: I57c80248728b63b162bf8269ac9495b53c3e7fa0
2017-01-11 17:20:36 -08:00
Roozbeh Pournader
a192a8ced6 Treat Latin American locales specially
Due to legacy reasons, Android translations of European Spanish were
kept under 'es', while Latin American Spanish translations were kept
under 'es-US'. The combination of this, and the new locale
preference rules in Nougat, resulted in 'es' winning over 'es-US' for
all Latin American locales, since 'es' was a direct ancestor, while
'es-US' was just a fallback.

The changes in Nougat had assumed that app developers would put Latin
American Spanish translations under 'es-419', but that could create a
backward-compatibility problem under older Android versions that did
not support three-digit region codes properly.

This CL keeps the Nougat logic and its locale parent tree, but
special-cases es-US and es-MX to be treated as equivalents of es-419
in cases where they are present and es-419 is not.

Bug: 31545805
Bug: 34126460
Test: unit tests are included
Change-Id: Iab26f41294587ee044685a5a6560520c7cbb06f7
2017-01-11 15:52:08 -08:00
Adam Lesinski
7ad1110ecd New implementation of AssetManager/ResTable
The multiwindow model and Resources-per-activity
model that came in N puts greater demands on AssetManagers.
They are created whenever window dimensions change, which
can be frequently. There is a need to be able to cheaply
create a new AssetManager for each Activity, which shares
a lot of underlying state.

In order to make the creation of AssetManagers cheap,
we need a new implementation of the native AssetManager
and ResTable to support immutable representations of
APKs. This new data structure/class is ApkAssets.

ApkAssets have the same functionality of an AssetManager, except
that they operate on a single APK, and they do not do any caching.
Once loaded, they are immutable.

ApkAssets will be exposed as a Java object, with its implementation in
native code. The existing Java StringBlock will be owned by ApkAssets,
which means that Strings can be shared across AssetManagers.

ApkAssets can be cached by the ResourcesManager. Creating an AssetManager
requires only a list of ApkAssets and a configuration.

AssetManager2 (named with the suffix '2' for now while transitioning
to the new implementation) caches bags that are accessed.

Since ApkAssets are expected to be kept around longer, they do more validation
of the resource table, which cause slower load times. Measured on an angler-userdebug,
loading the framework assets takes 11ms with ApkAssets, and 2ms with the old
AssetManager implementation.

The tradeoff is that there does not need to be any security checks once an ApkAssets
is loaded, and regular resource retrieval is faster. Measured on an angler-userdebug,
accessing resource (android:string/ok) with many locales takes 18us with AssetManager2,
and 19us with AssetManager (this is per resource, so these add up).

Test: make libandroidfw_tests
Change-Id: Id0e57ee828f17008891fe3741935a9be8830b01d
2017-01-11 13:30:57 -08:00
Adam Lesinski
06d3e8fec7 libandroidfw: Revert null check in ApplyStyle
The out parameter `out_indices` is expected to be non-null
and so the extra null check adds a cost to performance
and opens the door to misusing the API by not supplying
`out_indices`.

Test: make libandroidfw_tests
Change-Id: Ie66fd837c5e24ec2838156e7b67f54c15cd27933
2017-01-06 16:51:48 -08:00
Adam Lesinski
351471f928 libandroidfw: Make tests less flaky
Changing any of the test APKs could change resource IDs.
The resource IDs are all made public and assigned an ID
to avoid this issue.
Test: make libandroidfw_tests

Change-Id: Idd17c25f4ac86a0ad5b2b8da6f968e5d9b2346c1
2016-12-12 14:10:46 -08:00
Adam Lesinski
4c67a475a3 Make tests use APKs instead of exploded APKs
Tests would expect parts of the APK to be unzipped and
maintained. Instead, we now decompress the required files
from the test APKs on test setup. This simplifies
test maintenance substantially.

Test: make libandroidfw_tests && libandroidfw_tests --testdata=frameworks/base/libs/androidfw/tests/data
Change-Id: I3d2100af22df913e02401dedcf9842cdb32b2a3b
2016-12-05 19:03:47 -08:00
John Reck
f32adf4475 Clean up ApplyStyle JNI
Bug: 32573798

Mark input uint32_t[] as const. Use Read-only JNI
array access for input as it's faster than critical access.

Use non-movable arrays for TypedArray so that the address can
be resolved and stored, avoiding the need to do JNI array
access for the output.

Indicies is always non-null, so remove the optional checks.

Eliminate unused return value.

Benchmark results:
twelveKeyInflate 4963us -> 4713us
simpleViewInflate 73us -> 60us

Test: Device boots, benchmarks show faster

Change-Id: Ic3bde5aee31407d8903913f97f2218daf074499a
2016-12-02 12:49:20 -08:00
Adam Lesinski
ea7897954f Add --testdata flag for easier testing
Instead of hardcoding or assuming a path for testdata,
allow the testdata path to be specified via the command
line.
Test: make libandroidfw_tests

Change-Id: Ideae880b21c157b70a11bb5a90a94556771aead6
2016-11-29 15:01:15 -08:00
Adam Lesinski
7a37b74d37 Add tests for attribute resolution
- Adds unit tests for attribute resolution. These include
  some test data resource tables and compiled XML files.
- Convert touched files to Google style guide.

Test: make libandroidfw_tests
Change-Id: Ib3a36061dc874de5f6a266b4e82c0a12ef435f23
2016-10-17 18:33:42 -07:00
Adam Lesinski
0358efe4f7 Fix race with Asset destruction and printing allocation stats
A race could occur when printing the list of Asset allocations for
debugging purposes.

Each Asset object would insert themselves into a global linked list
on construction and remove themselves on destruction. Iterating the list
and the insertion/remove operations all acquire a global lock.

The race occurs after the Asset subclass destructor runs but before the Asset
base class destructor runs, which performs the actual removal from the list.

The vtable of the object being destroyed ends up pointing at the base Asset class'
vtable, and during the iteration of the global list, a pure virtual method is called
leading to an abort, since the wrong vtable is dereferenced.

This change moves the insertion/removal of the Asset object into the global list
to the concrete class, which adds some maintenance overhead but solves the problem.

Bug:31113965
Test: make libandroidfw_tests
Change-Id: I1a620897e5e04a8519ee247883bba0719b1fa6f3
2016-10-17 14:04:09 -07:00
Roozbeh Pournader
7e5f96f1a3 Move matching Tagalog and Filipino to ResourceTypes.cpp
Previously, if a mix of "fil" and "tl" resources existed in Resources
(from mixing resources from libraries for example), only resources
from one or the other would be chosen, resulting in default resources
getting surprisingly used. Now, we resolve the equivalent languages
at a per-resource levels (breaking ties for the identical code).

Also, previously if both "tl" and "fil" resources were present in
assets, getLocales() could return a list with duplicate locales.
This change removes Filipino duplicates in the return value of
AssetManager::getLocales().

Finally, there was a bug in the replacement of "tl" with "fil" that
considered any locale starting with the letter "tl" to be Tagalog.
This failed in case of various languages, including Klingon ("tlh")
and Tlingit ("tli"). It's now fixed.

Bug: 29073000
Change-Id: I0e8b9ae337ced2e640a2575897948c4c5ca307d3
2016-06-15 11:45:58 -07:00
Adam Lesinski
8ac51d14b6 Resource shared libraries: fix theme references
Theme values that would reference other theme values would not work
if they were declared in a shared library.

We now introduce a parallel resource type to TYPE_DYNAMIC_REFERENCE,
TYPE_DYNAMIC_ATTRIBUTE, which allows us to lookup and resolve
theme value references from shared libraries.

Bug:28687378
Change-Id: I4f2364e3e8b567679f90784fcaaea12b6b05e926
2016-05-10 10:45:52 -07:00
Shammi Khattar
7134ce3540 ZipUtils: Fix wrong timestamps when getEntryInfo
"tm_mon" format should align with "ZipEntry::setModWhen" in aapt.
"tm_isdst" should be initialized, or it will because random value
and cause error in function mktime().

BUG:28021145

(cherry picked from commit bb0eb3f4cb33c077b296a18555a9a1a966ff696b)

Change-Id: Ia39cf63fcd4bb39c24af080562960279106f526c
2016-04-28 15:07:34 +00:00
Adam Lesinski
b7e1ce0775 Optimize ResTable::getLocales() to improve bindApplication performance
Change from linear searching for uniqueness to binary search.

Bug:27198799
Change-Id: I1ccb6e93cc213810848f07d631d9d8de7c719803
2016-04-18 15:06:50 -07:00
Tim Murray
98e80076c6 Revert "Optimize ResTable::getLocales() to improve bindApplication performance"
This reverts commit 5520581b5f.

bug 28189634

Change-Id: I2d2b859f6d9bd44434fa901cce990583f514980c
2016-04-14 16:06:29 -07:00
Adam Lesinski
5520581b5f Optimize ResTable::getLocales() to improve bindApplication performance
Change from linear searching for uniqueness to binary search.

Bug:27198799
Change-Id: Ifa4672929df286c4693ab1f77716f08945941b0c
2016-04-13 13:25:09 -07:00
Adam Lesinski
ff5808d223 AssetManager: Cache a pre-filtered list of configurations
When we set the parameters for a ResTable, we can pre-filter which
resources match and only look at that smaller list when getting entries.
This helps A LOT with types that have many configurations, like strings
and all their various locales.

We must store the cached entries in a parallel data structure because parts
of the main Type object are shared with other ResTables, causing data races.

Bug:25499111
Change-Id: I63e37dcbd683fc9f1e7d0f3a6ed4c1c01e0fc575
2016-03-09 17:33:14 -08:00
Roozbeh Pournader
7960898f6f Fix script-related parts of locale resource matching
Previously, a bit was kept to find if the script of a locale was
explicitly "provided" in a resource. This was not backward
compatible, and failed in some edge cases when the package was
created with older versions of AAPT that did not set the bit.

The cases would happen when the old resource had an explicit script
specified in its locale, but since the "provided" bit was not set in
the package, we would assume that the script was computed by us.

This CL replaces the "provided" bit with a "computed" bit, so the
default value of the bit (set to "false" for old packages) would be
correct.

Bug: 27156990
Change-Id: I99e7f1ad8f70c90e25ab3640ed34cc1a6f8d1d64
2016-03-03 15:19:37 -08:00
Roozbeh Pournader
4de4596ac1 Fix locale matching algorithm for resources
We get ResTables two different ways: one is from AAPT, another from
settings-based requests from the Java side. In the settings-based
requests, localeScript will be autocomputed, but for AAPT-filled
tables (especially if they come from older versions of AAPT), we need
to compute the script.

Previously, locales that came from packages were incorrectly assumed
to have "undeterminable" scripts, rather than "undetermined" scripts.
This led to us mistakenly falling back to the old logic of requiring
the locales' countries to match, rather than just looking at computed
scripts.

Bug: 27157452

Change-Id: Id7e346d3ecfb17273ffb63de5bcb4849a6eafbbd
2016-02-12 05:18:38 +00:00
Roozbeh Pournader
27953c349f Make default resources a better match for en-US requests
When locale fallback landed, resources which specified an 'English'
locale started to be considered a better match for en-US, even though
traditionally, apps tend to ship US English resources under their
default locale.

This fixes that, and makes en-US requests match default locales.

Bug: 26756573
Bug: 26789680
Bug: 26803868
Change-Id: I460c276bfc6ddba0439dcdf87497a0aece0fa05d
2016-02-01 14:56:52 -08:00
Roozbeh Pournader
b927c559e1 Implement smarter locale resource selection
* Add support for determining script from language and region.
* Add support for determining special parents of locales.
* Add support for smart comparison of locales with only a difference
  in region, using the locale parentage tree.
* Fix LocaleData.matchScore() to not fallback to old locale matching
  behavior if we can't determine a script.
* Allow four-character variant codes. (Previously, only five- to
  eight-character variant codes were allowed.)

Bug: 7296673
Bug: 26589793
Change-Id: Ibde0a48c0564ff383b41068095a5cbacfe7b94bc
2016-01-21 13:47:22 -08:00
Tao Bai
1375e5f180 Add test to cover loading shared-lib with appAsLib as true.
This patch made AppAsLib_test use its own resource

Bug 22487604

Change-Id: Iac4cc949f1b25c326a287a49e0b031bf6831e9e9
2015-11-06 14:33:49 -08:00