Protect Cursors with CloseGuard and report implicit cleanup.
By default, the leak detection code will print a one-line warning
message about failing to call the cleanup function. To report more
details like the original stacktrace and SQLite query statement, enable
the correspoinding VmPolicy flags as the example below, please.
/*
* public void onCreate() {
* if (DEVELOPER_MODE) {
* StrictMode.setVmPolicy(new VmPolicy.Builder()
* .detectLeakedSqlLiteObjects() // for SqlLiteCursor
* .detectLeakedClosableObjects() // for any Cursor
* .penaltyLog()
* .build());
* }
* super.onCreate();
* }
*/
By enabling detectLeakedSqlLiteObjecs, the original SQLiteCursor query
statement is reported when close() or its equivalent cleanup function
is not called before finalize().
By enabling detectLeakedClosableObjects, the new CloseGuard report will
be provided, along with the original stack trace captured in open() or
its equivalent.
The former has better performance as it doesn't capture an original
stack trace during open() stage. Only enable the latter if performance
impact is not an issue. Both of them can be enabled at the same time.
Bug: 168639120
Test: manually test with an example SQLite Android application, with and
without calling cursor.close() after a db.rawQuery().
Change-Id: Ibe9fcdc8119c2e4651df1983e7ccd793f29e8e9d
Signed-off-by: Li Li <dualli@google.com>
These are APIs that have @UnsupportedAppUsage but for which we don't
have any evidence of them currently being used, so should be safe to
remove from the unsupported list.
This is a resubmit of ag/12929664 with some APIs excluded that caused
test failures; see bugs 171886397, 171888296, 171864568.
APIs excluded:
Landroid/bluetooth/le/ScanRecord;->parseFromBytes([B)Landroid/bluetooth/le/ScanRecord;
Landroid/os/Process;->myPpid()I
Landroid/os/SharedMemory;->getFd()I
Landroid/hardware/input/InputManager;->INJECT_INPUT_EVENT_MODE_WAIT_FOR_FINISH:I
Bug: 170729553
Test: Treehugger
Change-Id: I8285daa8530260251ecad6f3f38f98e263629ca7
These are APIs that have @UnsupportedAppUsage but for which we don't
have any evidence of them currently being used, so should be safe to
remove from the unsupported list.
Bug: 170729553
Test: Treehugger
Change-Id: I4c8fd0006f950de9955242e93968fb0996ceb372
Refactoring to avoid paying the cost of extra StringBuilder that are
quickly disposed.
Bug: 170978902
Test: none
Exempt-From-Owner-Approval: trivial refactoring
Change-Id: Icd914a63cdadf8123c1e5a5073f85245f0791f0b
The recently-built Error Prone checker has found many instances where
we're always paying the cost of StringBuilder concatenation, even in
the typical cases where preconditions are successfully met.
Benchmarks have shown that even when replacing these with varargs
formatter strings, the default case is 20x faster.
Bug: 170978902
Test: none
Exempt-From-Owner-Approval: trivial refactoring
Change-Id: If8c00bc73467bfb91ec16c162969c9d26ca53646
In SQLiteQueryBuilder, allow empty tokens when checking for invalid
tokens during the strict grammar check.
Fixes: 151151800
Fixes: 161769478
Test: atest SQLiteQueryBuilderTest#testStrictQueryEmptyToken
Change-Id: Iac1cdd643253fd186a164b863d65d6e92698fd38
(cherry picked from commit e409ec2492)
A few months ago MediaProvider forked SQLiteQueryBuilder and we
fixed bugs in how the new insert(), update(), and delete() methods
were executed, but those fixes never made their way back upstream.
This CL is a clean cherry-pick of existing logic.
Bug: 155149941
Test: atest android.database.sqlite.cts.SQLiteQueryBuilderTest
Change-Id: I00d2ee88539b12d27a2ed99fff0546d1e3543b0e
Otherwise we'd risk applying different arguments to active versus
future connections.
Bug: 152005629
Test: atest CtsDatabaseTestCases:android.database.sqlite.cts.SQLiteDatabaseTest
Change-Id: I6e83a47d9874825fd14940245412372ef9227e79
Developers have been able to register custom collators using syntax
like "SELECT icu_load_collation()", but collators are registered per
database connection.
Since we don't expose any details APIs for interacting with connection
pools directly, developers can end up with flaky behavior as their
queries rotate through the pool of connections, as only a subset of
connections will have their collation registered.
This solve this, we add a new execPerConnectionSQL() method to
ensure that a given statement is executed on all current and future
database connections.
Bug: 152005629
Test: atest CtsDatabaseTestCases:android.database.sqlite.cts.SQLiteDatabaseTest
Change-Id: I459fb7b18660d2a04eec92d1e9cc410d769e361d
Starting in R, there is a new public API overload that delivers a
flags argument. Some apps may be relying on a previous hidden API
that delivered a userId argument, and this change is used to control
delivery of the new flags argument in its place.
There are dozens of these hidden API users are in the system UID,
either in the system process or the Settings app, so we hard-code
giving them the legacy behavior, since refactoring would be messy
between internal and AOSP branches.
Also adjust incoming and outgoing method signatures to use slightly
more flexible Collection<Uri>, which has handy methods like
contains() and isEmpty().
Bug: 150939131
Test: atest --test-mapping packages/providers/MediaProvider
Test: atest FrameworksServicesTests:com.android.server.devicepolicy.DevicePolicyManagerTest
Exempt-From-Owner-Approval: trivial refactoring
Change-Id: If6a77449e19215cf1c60d4217e62fc04b0959bfc
As part of MediaProvider becoming a Mainline module, some partners
need to understand more details about what triggered a Uri change
notification. This helps listening apps avoid making expensive and
race-condition-prone calls back into MediaProvider; typically when
they're only interested in insert and delete operations. This change
uses the existing "flags" argument to communicate the reason.
This change adds overloads to ContentObserver for listening apps to
receive these flags. In addition, we add overloads that deliver a
clustered set of multiple Uris together in a single Binder transaction
to improve overall efficiency. (This matches well with the existing
CR.notifyChange() API that we added earlier this year, since they
both work with Iterable<Uri>.)
Tests to verify that we only collapse Uris together when all other
method arguments are identical.
Bug: 147778404, 144464323
Test: atest CtsDatabaseTestCases
Test: atest CtsContentTestCases:android.content.cts.ContentResolverTest
Test: atest FrameworksServicesTests:com.android.server.content.ObserverNodeTest
Test: atest --test-mapping packages/providers/MediaProvider
Change-Id: I0bbd8a8b4a898ab6f891d085de0ecb4d68cbe302
SQLite ships with a handful of basic functions, such as UPPER() as
a scalar function and MAX() as a aggregate function. We now have
several use-cases for adding custom functions, where it's otherwise
prohibitively expensive to perform post-processing on the returned
Cursor, as that requires copying processed data to yet another
MatrixCursor before returning to apps.
This change adds the ability for developers to register custom
scalar and aggregate functions on databases that they've opened;
some contrived examples are scalar functions like REVERSE() for
reversing a string, or aggregate functions like STDDEV().
To give developers the most flexibility, we use the Java functional
interfaces for defining these operations, as developers may already
be familiar with the contracts of those methods. This also opens
the door to quickly adapting existing code through utility methods
like BinaryOperator.minBy(Comparator).
Bug: 142564473
Test: atest CtsDatabaseTestCases:android.database.sqlite.cts.SQLiteDatabaseTest
Change-Id: I9fa0e60ec77bab676396729cc9cb8ba8aaf56224
Adjust Exception to more general Throwable, and add docs for
new ContentResolver and ContentProvider overloads. Also add docs
for default state of strict query options.
Bug: 131598520, 141227540, 147287177
Test: atest --test-mapping packages/providers/MediaProvider
Change-Id: Icb0153b8c8a352db39de301074e948bfaa676ddb
The bulk of the work needed to get MediaProvider building against
the "system_current" SDK surface has been slowly merged over the
last few months, and this change makes the last few adjustments.
This adds a new StorageVolumeCallback which is simpler version of
StorageEventListener that simply delivers the changed StorageVolume.
Move DownloadManager logic into a onMediaStoreDownloadsDeleted()
method which hides the implementation details of how the OS connects
with that implementation.
Make local copies of some ExifInterface parsing logic; they could
be added to the androidx version in an unbundled release. Make a
local copy of RedactingFileDescriptor, since it's only needed for
the next few weeks until FUSE is globally enabled.
Bug: 137890034
Test: atest --test-mapping packages/providers/MediaProvider
Change-Id: Ib416eb8724781bdd234c8b7d728dee8b695ad6ac
Existing annotations in libcore/ and frameworks/ will deleted after the migration. This also means that any java library that compiles @UnsupportedAppUsage requires a direct dependency on "unsupportedappusage" java_library.
Bug: 145132366
Test: m && diff unsupportedappusage_index.csv
Change-Id: I0c336de56bc4a041dc97ff9b7927f62f0b44b457
Merged-In: I0c336de56bc4a041dc97ff9b7927f62f0b44b457
Existing annotations in libcore/ and frameworks/ will deleted after the migration. This also means that any java library that compiles @UnsupportedAppUsage requires a direct dependency on "unsupportedappusage" java_library.
Bug: 145132366
Test: m && diff unsupportedappusage_index.csv
Change-Id: I0c336de56bc4a041dc97ff9b7927f62f0b44b457
We had hoped that SQLiteQueryBuilder could enforce some pretty strict
grammar rules, but there are apps depending on the full quirkiness
of what SQLite will allow, which is pretty much "expr" everywhere.
So we shift our strategy from a whitelist-style approach to a
blacklist-style one, where we're willing to let any valid keyword
be used _except_ for ones we know are rooted in abuse, such as
performing SELECT subqueries or relying on SQL injection to blur
the lines between well-defined clauses.
Bug: 146478800, 146482076
Test: atest --test-mapping packages/providers/MediaProvider
Change-Id: Ib57ceab0e229e01177cabd8277f3c989c817b10c
These custom collators are added by ./android/sqlite3_android.cpp,
so let callers use them.
Bug: 140666363
Test: atest android.database.sqlite.cts.SQLiteQueryBuilderTest
Change-Id: I6674b04afb5bbb5f9870329431549eca8e4f5bc6
go/cleanup-greylist-txt
These have already been greylisted, however due to bugs/omissions in the tooling have been kept in go/greylist-txt instead of being annotated in the code.
Bug: 137350495
Test: m
Change-Id: Id6c1f5e403a0e66edb1102ee45f3bf19f244fb09
Merged-In: Id6c1f5e403a0e66edb1102ee45f3bf19f244fb09
Malicious callers can leak side-channel information by using
subqueries in any untrusted inputs where SQLite allows "expr" values.
This change offers setStrictGrammar() to prevent this by outright
blocking subqueries in WHERE and HAVING clauses, and by requiring
that GROUP BY and ORDER BY clauses be composed only of valid columns.
This change also offers setStrictColumns() to require that all
untrusted column names are valid, such as those in ContentValues.
Relaxes to always allow aggregation operators on returned columns,
since untrusted callers can always calculate these manually.
Bug: 135270103, 135269143
Test: atest android.database.sqlite.cts.SQLiteQueryBuilderTest
Test: atest FrameworksCoreTests:android.database.sqlite.SQLiteTokenizerTest
Change-Id: I0dacb53170ce573a2fe103cbff455782bfdb5d41
This is an automatically generated CL. See go/UnsupportedAppUsage
for more details.
Exempted-From-Owner-Approval: Mechanical changes to the codebase
which have been approved by Android API council and announced on
android-eng@
Bug: 137350495
Test: m
Change-Id: I55c7845cb7c22808a85dd5185eccfe93c2064b76
Malicious callers can leak side-channel information by using
subqueries in any untrusted inputs where SQLite allows "expr" values.
This change offers setStrictGrammar() to prevent this by outright
blocking subqueries in WHERE and HAVING clauses, and by requiring
that GROUP BY and ORDER BY clauses be composed only of valid columns.
This change also offers setStrictColumns() to require that all
untrusted column names are valid, such as those in ContentValues.
Relaxes to always allow aggregation operators on returned columns,
since untrusted callers can always calculate these manually.
Bug: 135270103, 135269143
Test: atest android.database.sqlite.cts.SQLiteQueryBuilderTest
Test: atest FrameworksCoreTests:android.database.sqlite.SQLiteTokenizerTest
Change-Id: I109fbb51e4bd2ed3c3b23a7c5f0cb795f7d8b984
SQLiteDebug.Const isn't supposed to be preloaded.
Fix: 134176355
Test: "setprop db.log.slow_query_threshold.10045 0" and make sure ...
it works for UID 10045
Change-Id: I06ca1531e968faa2378d0cb7f627e2c6a87cf502
A content observer is registered by default when setting a notification
uri for a Cursor, in order to make the Cursor correctly notify listeners
of all changes to its URI, not just the ones made locally.
This is not required for DocumentCursor, because it already has a
separate mechanism for watching for all changes made to the data backed
by the cursor.
This avoids DocumentProviders having to call into system_server to
answer queries about directory trees, which can otherwise add up to
significant amounts of time for large directory trees. In my tests,
this improves the performance of iterating through a directory by
roughly 20%. This number is likely to be higher on non-test devices,
that probably see more binder contention, and will also depend on the
structure of the file tree.
Bug: 130276310
Test: SAF test app
Change-Id: I386363b0608c420e9847caf6fbf6686641c955e2
When traversing a file tree using SAF, this method is called for
every file in the tree, so wasted cycles add up to quite a lot
of total time for large trees.
Optimizations:
- don't look up info not necessary by the cursor, by inspecting
the projection columns
- remove a few redundant double-lookups, of e.g. the File path
- mark variables final where possible
On my device/file tree, this reduces the total time spent in includeFile
from ~33 seconds to ~22 seconds. This is where the majority of cycles
are spent when traversing a dir with SAF.
Bug: 130276310
Test: SAF test app
Change-Id: Ibdfc335253a90eb87795a7a4eecb8b7b89601f09
As long as the column being aggregated is valid with respect to
all existing projection checks, then we're willing to apply an
aggregation function on it.
Bug: 129220616
Test: atest --test-mapping packages/providers/MediaProvider
Change-Id: I8e4b665f5abd6553939a35d9f46247cc63c4c25f
TranslatingCursor previously assumed each column name appeared
only once in the base cursor and used getColumIndex(String) to
find the index of it. It would then translate only that single
index.
Instead, maintain a set of column indices that need translating.
Also add a unit test.
Bug: 125467331
Test: atest FrameworksCoreTests:TranslatingCursorTest
Test: verified broken app in bug report is working correctly
Change-Id: I33a24bf6474338210ec7b9c6b54912ed49f23cb3
We stopped using WAL, so let's just disable it
Change-Id: I14f4f3ca6521db12a2fbde8c20122bb4b2cc8186
Fix: 118613196
Test: Manual test with removing DB files
If they were null, then the Parcelable would fail to work.
Bug: 126726802
Test: manual
Change-Id: I7929ffa2f20e5de1c8e68e8263cca99496e9d014
Exempt-From-Owner-Approval: Trivial API annotations
