Commit Graph

304 Commits

Author SHA1 Message Date
Xiaoyu Jin
d4ce296d90 Merge "Register callback to statsD for AppSearchStorageInfo" into sc-dev 2021-07-10 01:20:18 +00:00
Xiaoyu Jin
57bd208139 Register callback to statsD for AppSearchStorageInfo
Bug: b/173532925, b/193254235
Test: statsd_testdrive 10116
      atom {
        app_search_storage_info {
          user: 0
          total_storage_size_bytes: 3929088
          document_storage_info {
            num_alive_documents: 1116
            num_deleted_documents: 4933
            num_expired_documents: 0
            document_store_size_bytes: 3637760
            document_log_size_bytes: 2928640
            key_mapper_size_bytes: 49152
            document_id_mapper_size_bytes: 49152
            score_cache_size_bytes: 122880
            filter_cache_size_bytes: 73728
            corpus_mapper_size_bytes: 20480
            corpus_score_cache_size_bytes: 4096
            namespace_id_mapper_size_bytes: 20480
            num_namespaces: 21
          }
          schema_store_storage_info {
            schema_store_size_bytes: 39424
            num_schema_types: 4
            num_total_sections: 9
            num_schema_types_sections_exhausted: 0
          }
          index_storage_info {
            index_size_bytes: 248320
            lite_index_lexicon_size_bytes: 155648
            lite_index_hit_buffer_size_bytes: 262144
            main_index_lexicon_size_bytes: 0
            main_index_storage_size_bytes: 0
            main_index_block_size_bytes: 4096
            num_blocks: 1
            min_free_fraction: 0.99999905
          }
        }
      }
      appsearch cts tests

Change-Id: I009c0932e36afb467c93013215d1e67dfae5a28a
2021-07-09 17:05:23 -07:00
Terry Wang
878a0d7c8f Merge "Add check for optimize after each mutation operation." into sc-dev 2021-07-09 22:47:24 +00:00
Terry Wang
da0e9a9b37 Add check for optimize after each mutation operation.
Optimize is an important job for AppSearch to work properly.
It could collect garbage and release resources. Without it, the garbage
resource will last forever and the device's space will be fulled up by
zombie documents.

The algorithm to trigger optimize is:
1: Query garbage resource info after a batch of mutation operations.
2: Only trigger optimize if there are enough resource could be released.

The behavior exists in AppSearch Jetpack for a while and working properly
for our Jetpack dogfooders.

Bug: 175255572
Test: FrameworkOptimizeStrategyTest
Test: AppSearchConfigTest
Test: AppSearchImplTest
Change-Id: Ib7ae475cc035d1b69969df1e22ac409895e0e3fa
2021-07-09 12:09:10 -07:00
Tej Singh
7961d27429 Cleanup appsearch platform logger
Test: TH
Bug: 192684294
Change-Id: I8e6828b9cbcbd30d06370ad19f2d36290298da25
2021-07-08 14:45:36 -07:00
Tej Singh
6f5306d6e7 Merge "Cleanup AcitivityManagerConstants and AppSearch" into sc-dev 2021-07-07 17:04:41 +00:00
Tej Singh
77b3d5db92 Cleanup AcitivityManagerConstants and AppSearch
Test: TH
Bug: 192684294
Change-Id: Iaf4d7893214297408f609ce58573846f2b6adf5e
2021-07-02 15:01:11 -07:00
Alexander Dorokhine
733537f1a6 Merge "Add ability to enforce limits on docs per package and size of doc." into sc-dev 2021-07-02 01:46:27 +00:00
Alexander Dorokhine
5a26c6b39c Merge "Move libicing.so out of /usr/lib, into the appsearch apex." into sc-dev 2021-07-01 21:29:56 +00:00
Alexander Dorokhine
5e4a11cd10 Add ability to enforce limits on docs per package and size of doc.
This is needed to prevent abuse of our service and to share icing
docids in framework, which are the resource we are most likely to
run out of.

Without this change, an app could use the platform backend to index
a very high number of documents and exhaust the docid limit, thereby
preventing any other app from using AppSearch.

Bug: 170371356
Test: New testcases added to AppSearchImplTest
Change-Id: I03ade35072bc69b84f8fcefed72b3c3bc2b8ee68
2021-07-01 13:22:08 -07:00
Paul Duffin
56dc119776 Merge "Enable appsearch to perform its own hidden API processing" into sc-dev 2021-06-29 22:37:46 +00:00
Alexander Dorokhine
24da2e6b35 Move libicing.so out of /usr/lib, into the appsearch apex.
Bug: 169826971
Test: Presubmit
Change-Id: Ibe5dd5014c5a70cb5b0fcf3eb33db9911047bdc8
2021-06-28 19:44:58 -07:00
Paul Duffin
6a87d217c7 Enable appsearch to perform its own hidden API processing
Previously, the hidden API encoding of the appsearch boot dex jars,
i.e. those dex jars that appsearch contributes to the bootclasspath
were done as part of the monolithic hidden API processing. This change
causes the encoding to be done by the appsearch's
bootclasspath_fragment.

This change involves the following:
* Addition of the fragments property to the appsearch's
  bootclasspath_fragment module to list all the other
  bootclasspath_fragment modules on which this depends.
* Addition of the additional_stubs property to add stubs for APIs that
  are not provided by another bootclasspath_fragment.

The build automatically checks that the hidden API flags which are
computed by appsearch and encoded into its boot dex jars match those
that are generated by the monolithic processing so this is guaranteed
to be safe.

Bug: 179354495
Test: m com.android.appsearch
      - ensure that the generated APEX is byte-for-byte identical
        before and after these changes.
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - make sure that they are not changed by this.
Change-Id: I8e5a60d64d3e147535e11a4ad30404f9888f53d2
2021-06-29 02:17:11 +01:00
Alexander Dorokhine
e665d39fd4 Merge "Port appsearch-service to the system_server_current sdk." into sc-dev 2021-06-28 16:27:55 +00:00
Alexander Dorokhine
9da0fe5179 Port appsearch-service to the system_server_current sdk.
The hidden APIs have already been fixed, so this CL mostly changes
build rules.

Bug: 181787682
Bug: 146218515
Test: Presubmit
Change-Id: I5d68d68ce8f39753f62c5e3680216877b7d7f819
2021-06-24 22:58:46 -07:00
Cassie Wang
f98a5efba1 Merge "Prevent instant apps from using AppSearch." into sc-dev 2021-06-24 02:36:40 +00:00
Cassie Wang
8417a0f253 Prevent instant apps from using AppSearch.
Bug: 181875256
Test: atest -m -c --rebuild-module-info CtsAppSearchHostTestCases
CtsAppSearchTestCases FrameworksCoreTests:android.app.appsearch
FrameworksServicesTests:com.android.server.appsearch

Change-Id: I50fa222f51a9d10395bd6b5815580170561d96d0
2021-06-23 14:15:27 -07:00
Alexander Dorokhine
9a8625df25 Sync framework from jetpack.
Changes included:
* 131d852: Remove usages of hidden APIs in the CTS tests.
* c984c24: Change AppSearchSchema and PropertyConfig #toString()

Bug: 146218515
Bug: 179680545
Test: Presubmit
Change-Id: Ibcef982cfea95ede2ae8e6075a052a83b44298c4
2021-06-23 13:14:01 -07:00
Alexander Dorokhine
bb978dfe38 Update framework from jetpack.
Changes included:
* 79f2ffe: Fixes required for export.
* e1ca63a: Implement empty schema shouldn't have a version number.
* 8b76be4: Refactor VisibilityStore from a no-op implementation into an interface.
* 47ba533: Remove the max repeatable length limit for GenericDocument
* 52ca287: Add GetSchemaResponse cts test.
* 56585f5: Change PutDocumentsRequestTest to be a cts test.

Bug: 183050495
Bug: 180058203
Bug: 191592792
Bug: 189161227
Bug: 183239766
Test: Presubmit
Change-Id: I30f51a18d697d3a5e43d2c63549ab19a36bbe99e
2021-06-22 12:32:15 -07:00
Alexander Dorokhine
f9d86a4aeb Update framework from jetpack.
Included changes:
* a8c984: Add more tests for logging
* 501a51: Remove some deprecated TODOs.
* 639f25: Add SchemaMigrationStats.
* 0b8885: Use consistent terminology for 3p and system access.
* 064c36: Add a CTS test for snippeting window sizes and max match counts.

Bug: 173532925
Bug: 187879464
Bug: 180058203
Test: Presubmit
Change-Id: I878fd6c7a42cccb237898392aac8c615830eb564
2021-06-18 14:28:34 -07:00
Alexander Dorokhine
9b26c3c9be Guard system usage reporting by checking the caller has system access.
Adding this check in a testable way requires refactoring VisibilityStore
to pull it out of management by the AppSearchImpl object. It is now
initialized as a sibling of AppSearchImpl, managed by
AppSearchUserInstanceManager.

This has several benefits:
* Breaks the complicated initialization inter-dependency between
  AppSearchImpl and VisibilityStore
* Reduces duplicative singleton managers
* Allows AppSearchImpl to be tested more easily by accepting a
  "hasSystemSurfaceable" boolean that can be set in tests
* Reduces the number of times we have to call into VisStore; we can
  determine whether the caller has system access in advance instead of
  repeating the check for every schema type

Bug: 180058203
Bug: 183031844
Test: GlobalSearchSessionCtsTest#testReportSystemUsage_ForbiddenFromNonSystem
Test: GlobalSearchSessionPlatformCtsTest#testReportSystemUsage
Test: VisibilityStoreTest

Change-Id: I84c84819f287628ccf8af369f5481a8e90255f62
2021-06-18 10:48:58 -07:00
Terry Wang
5c86c0c10e Merge "Clear the uninstalled package data in AppSearch." into sc-dev 2021-06-18 06:59:18 +00:00
Nikita Ioffe
a2d7f96154 Merge "Explicitly set generate_hashtree: false" into sc-dev 2021-06-16 10:30:35 +00:00
Terry Wang
c5d1afa418 Clear the uninstalled package data in AppSearch.
Since we cannot access locked user's data and clear it when a package
got removed. When a user is unlocking, we should prune their data and
remove all data that doesn't belong to any of installed packages.

This will also help in handling the case where device gets
rebooted before the data is cleared for an uninstalled package.

Bug: 186151459
Test: atest --rebuild-module-info CtsAppSearchHostTestCases
Change-Id: Ic6d151621a88df7806901781d3edfec65700c35d
2021-06-16 01:52:26 -07:00
Alexander Dorokhine
66e09afd54 Switch to a non-hidden way to compute data path and switch to subdir.
We need to store other things in the appsearch data dir other than what
Icing stores, such as the storage info cache.

Bug: 177685938
Bug: 181787682
Test: Presubmit
Change-Id: I903c34a2c891d617217d44f5126d0d31c8c6bf39
2021-06-15 16:43:28 -07:00
Alexander Dorokhine
ad37babc9f Merge "Update framework from jetpack." into sc-dev 2021-06-15 23:20:43 +00:00
Terry Wang
aca6fae8b2 Merge "Close AppSearchImpl in handling user removed broadcast." into sc-dev 2021-06-15 00:42:21 +00:00
Nikita Ioffe
c452ed3df1 Explicitly set generate_hashtree: false
Non-updatable APEXes are going to be only present on a dm-verity
protected read-only partition. For such APEXes there is no need in
embedding a hashtree.

This is a no-op change in preparation of switching default value of
generate_hashtree to true.

Bug: 190621617
Test: m
Change-Id: I0283fb03908a87490e5e29b803553977b8b6729f
2021-06-14 22:05:39 +01:00
Alexander Dorokhine
c85a2775c8 Fix some incorrect annotations in AppSearch.
Bug: 180763772
Test: Builds
Change-Id: I9f5dd9af8e3f3715be58004bb70bee5c5692b721
2021-06-10 09:00:45 -07:00
Terry Wang
a3d8ce430c Close AppSearchImpl in handling user removed broadcast.
Flushing takes a few ms, it seems like a pretty good trade off compared
to keeping an icing instance in memory forever.

Bug: 179390369
Test: presubmit
Change-Id: Ic0493137dc8c33d44bd074bbd9ca4ae442049513
2021-06-09 18:51:54 -07:00
Alexander Dorokhine
7cdd56994f Update framework from jetpack.
Changes included:
* 9b55e76: Remove GeneralStats
* b467803: Upstream fixes for UserHandle and VisibilityStore.
* 47d8dca: Allow FrameworkOptimizeStrategy to be export to Framework.

Bug: 173532925
Bug: 181787682
Bug: 175255572
Test: Presubmit
Change-Id: I1aec9cb824f8ef64df98f252ed5a6338abc9e152
2021-06-09 16:35:22 -07:00
Xiaoyu Jin
e298f682ca Merge "Use AppSearchConfig in PlatformLogger" into sc-dev 2021-06-09 20:26:47 +00:00
Xiaoyu Jin
0c5d987407 Use AppSearchConfig in PlatformLogger
bug: 173532925
Test: atest CtsAppSearchTestCases FrameworksCoreTests:android.app.appsearch
FrameworksServicesTests:AppSearchImplTest
FrameworksServicesTests:com.android.server.appsearch.stats.PlatformLoggerTest
FrameworksMockingServicesTests:com.android.server.appsearch.AppSearchConfigTest

Change-Id: I3589f0071d456e2167cd207e83acaf1d884f9992
2021-06-08 13:06:35 -07:00
Xiaoyu Jin
a1d4f99dbe Log two new fields for initializeStats
Test: it builds
bug: b/173532925

Change-Id: Ia31fc1fb54827efa140e16eb654e0ddfeeb28a07
2021-06-08 19:41:59 +00:00
Alexander Dorokhine
8943b271ce Move to non-hidden APIs for handleUser and broadcast registration.
The implementation of handleIncomingUser is copied from
ag/13200660 (1d1547b).

Bug: 181787682
Test: Presubmit
Change-Id: I946199a8844e7fd6ea5574b873da56d8b8630f62
2021-06-04 09:05:00 -07:00
Cassie Wang
bf90bb447e Merge "Update VisibilityStore to handle role-based system visibility." into sc-dev 2021-06-03 22:16:06 +00:00
Alexander Dorokhine
98014567af Update VisibilityStore to handle role-based system visibility.
Bug: 187113133
Test: VisibilityStoreTest
Change-Id: Ic28110fba4a3639b087869d6b3af90e4f3f6c207
2021-06-03 11:29:05 -07:00
Xiaoyu Jin
b7d75ad788 Merge "Add AppSearchConfig in AppSearch" into sc-dev 2021-06-02 22:13:37 +00:00
Xiaoyu Jin
951f75637f Add AppSearchConfig in AppSearch
It manages the P/H flags from DeviceConfig.

bug: 173532925
Test: atest FrameworksMockingServicesTests:AppSearchConfigManagerTest

Change-Id: I668ecd8190bbe7a21ab3a23b52d08f4dce400491
2021-05-26 01:08:50 +00:00
Terry Wang
08c51052a5 Annotate AppSearchManager APIs with @UserHandleAware
This helps readers understand that the AppSearch APIs can operate
across users.

Bug: 179822197
Test: presubmit
Change-Id: I0be942f16d0211ea1ea68d72ac82c254b9dd6d02
2021-05-24 14:47:26 -07:00
Alexander Dorokhine
0f7f36d29a Update framework from jetpack.
Included changes:
* d953a83b: Minor Migration update to apply API Review feedback
* b5b73db0: Expand Javadoc around Tokenization
* 45ac5ac2: Make GenericDocument.Builder reusable.
* 6d55d6d6: Logging stats for remove (2)

Bug: 187419874
Bug: 184174886
Bug: 184575874
Bug: 173532925
Test: Presubmit
Change-Id: Id6509e939e9038e5ec56bde610e4af902888b9d6
2021-05-20 14:24:50 -07:00
Alexander Dorokhine
c9233287e7 Stop depending on internal Preconditions.java from service-appsearch.
Bug: 181787682
Test: Presubmit
Change-Id: I2698daafca553ee2463e6f7dd1badda272287ba3
2021-05-20 13:45:05 -07:00
Alexander Dorokhine
83fbf50685 Merge "Update statsd config to be mainline compatible." into sc-dev 2021-05-20 20:36:23 +00:00
Alexander Dorokhine
c9429f43e0 Merge "Remove PackageManagerInternal from AppSearchService." into sc-dev 2021-05-20 20:36:04 +00:00
Alexander Dorokhine
6737deacc2 Merge "Fix HiddenApiTest#testSignatureMethodsThroughReflection." into sc-dev 2021-05-20 17:47:00 +00:00
Artur Satayev
98d5c6b32f Merge "Add systemserverclasspath_fragment." into sc-dev 2021-05-20 14:27:23 +00:00
Anton Hansson
eaf26d5a41 Update API txt following a metalava fix
The txt was previously referencing VcnGatewayConnectionConfig.MIN_MTU_V6
which isn't part of the API. The API txt should be complete on its own,
so inline the value instead.

Similar case for appsearch annotations referencing non-API values.

Bug: 188527836
Test: m checkapi
Change-Id: Ideb34ab78c9a8fc4d589e58e810316b6f36b7a43
2021-05-20 11:14:05 +01:00
Alexander Dorokhine
a4354f989e Update statsd config to be mainline compatible.
Instead of using the "framework" module, we use our own module for
generating the atoms.

Bug: 181787682
Test: Builds
Change-Id: I626e2ddcad60d7a15c33b7a143b9e5d0ed0bb289
2021-05-19 21:52:06 +00:00
Alexander Dorokhine
ad8a40a56f Remove PackageManagerInternal from AppSearchService.
Most public and system APIs use UserHandle instead of UserIdInt, so this
CL ports AppSearch to do so as well, so that there are less conversions
required.

Bug: 181787682
Test: Presubmit
Change-Id: I29e4ba6bc73b994f1ece077ce515e1465ce54b12
2021-05-19 13:59:45 -07:00
Alexander Dorokhine
cadaa8b892 Fix HiddenApiTest#testSignatureMethodsThroughReflection.
It could not parse the annotation.

Bug: 188503851
Test: atest HiddenApiTest#testSignatureMethodsThroughReflection
Test: flash image, remove jetpack @SdkSuppress, and run jetpack tests
Change-Id: I05093b61b8658ad95db4f7e49c2f64bc314aa4b1
2021-05-18 16:54:25 -07:00