Commit Graph

543 Commits

Author SHA1 Message Date
Jeff Sharkey
34e3e68375 Merge "Mark deprecated APIs, support column greylisting." 2018-09-14 18:38:15 +00:00
Mathew Inwood
db0657a4f4 Merge "Move some members to the "Q blacklist"." 2018-09-14 13:54:28 +00:00
Mathew Inwood
8c854f86a4 Move some members to the "Q blacklist".
Based on some analysis, these fields/methods are likely false positives.
Set maxTargetSdk=P so that any apps using them are required to migrate off
them in future. See the bug for more details.

Exempted-From-Owner-Approval: Automatic changes to the codebase
affecting only @UnsupportedAppUsage annotations, themselves added
without requiring owners approval earlier.

Bug: 115609023
Test: m
Change-Id: I719b5c94e5b1f4fa562dd5d655953422958ad37e
2018-09-14 13:18:34 +01:00
Jeff Sharkey
7049e651fa Mark deprecated APIs, support column greylisting.
Update MediaStore APIs that are effectively no-ops, including
MiniThumbFile which is no longer used.

Add support for "greylisted" columns to SQLiteQueryBuilder, so that
we can support abusive columns that match specific regex patterns
while in "strict" mode, such as COUNT(*).

Bug: 111268862
Test: atest MediaProviderTests
Test: atest cts/tests/tests/provider/src/android/provider/cts/MediaStore*
Change-Id: I364ef5bfa338ef8492b0b29c2a5c13c4189467ee
2018-09-13 17:14:27 -06:00
Jeff Sharkey
ec5f7d1625 Support more createPackageContextAsUser() users.
We created this API to make it easy to pass a given UserHandle into
all Managers obtained from a given Context, which works great for
"normal" users, but we should also support special users like ALL
and CURRENT.

Also add an AutoCloseable marker to make try-with-resources easier.

Bug: 112153259
Test: atest android.content.cts.ContextTest
Change-Id: I261dfcc5cfdfc76bda5d70181785e11c2715a558
2018-09-12 14:02:11 -06:00
Makoto Onuki
eb932592ad Add test API for sqlite CTS
Test: CtsDatabaseTestCases
Bug: 112777941
Bug: 111939259
Change-Id: Ibd394aa801ddfbb383acb9ae73c2029b47965f83
2018-08-27 12:14:05 -07:00
Makoto Onuki
efa01c37b7 Merge "Fix OWNERS (change google.com to android.com)" am: 7a8e363c0b am: 906a31f49f
am: 8fd96af2b1

Change-Id: Ibe5b3d6bf9d11fce9d091eb3e3872b6cd9e66afc
2018-08-25 23:33:02 -07:00
Makoto Onuki
8fd96af2b1 Merge "Fix OWNERS (change google.com to android.com)" am: 7a8e363c0b
am: 906a31f49f

Change-Id: Ie5089669974c9e9e9c99525c50eddaef6f479874
2018-08-25 23:23:37 -07:00
Makoto Onuki
d52f363924 Merge "Actually make OWNERS in sync with the internal version" am: 7e54042f9c
am: 83c37379da

Change-Id: I1bbd7ccc7b610d9cbdcf9f7f8f52b734679766cd
2018-08-24 15:17:22 -07:00
Makoto Onuki
96e06002ed Truncate(*1) it if WAL file is too big when opening DB
*1 -- not truncate(2) but "PRAGMA wal_checkpoint(TRUNCATE)"

Otherwise, depending on how an app operate on a DB, SQLite may not
have a chance to "shrink" the WAL file.

Fixes: 112777941
Bug: 111939259
Test: atest /android/master/frameworks/base/core/tests/coretests/src/android/database/sqlite/SQLiteCompatibilityWalFlagsTest.java
Test: Manual test with google dailer:
1. With normalized_spam.db-wal > 100MB and receive a phone call
-> WAL file gets truncated to 0 bytes.

2. Restart the dialer process and receive a phone call again
-> WAL file is already 0 bytes; won't be truncated.

3. Restart with the WAL file deleted
Same as #2. WAL file will be created before the added logic, but is 0 bytes,
so it won't be truncated.

4. Test with settings put global sqlite_compatibility_wal_flags truncate_size=1024
-> make sure the threshold is overridden

Change-Id: I2b193603e5dfa493ccccb8123db592f0e9c0e7ae
2018-08-24 15:07:21 -07:00
Makoto Onuki
1fda56bbda Fix OWNERS (change google.com to android.com)
Bug: N/A
Test: N/A
Change-Id: I9a9eecfff1a72d7da55be0df73f29985b19c1e38
2018-08-24 14:58:42 -07:00
Makoto Onuki
83c1a68491 Actually make OWNERS in sync with the internal version
Bug: N/A
Test: N/A
Change-Id: Idc1dcc7753a5cf731475ae0f8f01ccd54ac0e3b3
2018-08-24 12:47:20 -07:00
Makoto Onuki
49a46611ef Merge "Update OWNERS for database code" am: aea74ff62a
am: 92732c2a25

Change-Id: I43a02b809a275df2ddbc2987f811a5f40c2a8461
2018-08-24 11:29:37 -07:00
Makoto Onuki
5b3c10cb28 Update OWNERS for database code
Test: N/A

Change-Id: If657f1ac87d9a9e77162816c7ac6abb719b834d4
Merged-in: If657f1ac87d9a9e77162816c7ac6abb719b834d4
2018-08-22 15:43:13 -04:00
Makoto Onuki
2132e0814c Merge "Default to FULL synchronous mode for sqlite DBs in system server" into pi-dev
am: 30fdcbad94

Change-Id: I0f4b5f91cf0a877bc518b555abb8380487c68d31
2018-08-17 16:48:09 -07:00
Makoto Onuki
962f78649f Default to FULL synchronous mode for sqlite DBs in system server
Test: change pin and immediately crash the kernel with
adb shell 'su root sh -c "echo c >/proc/sysrq-trigger"' and boot
Bug: 112175067

Change-Id: Ia5f43f3118e2297fbea43c805ef2f4577bf8a9bf
(cherry picked from commit 50e00c8dc4)

Merged-in: Ia5f43f3118e2297fbea43c805ef2f4577bf8a9bf
2018-08-17 16:35:46 -07:00
Makoto Onuki
3729e9ce4e Default to FULL synchronous mode for sqlite DBs in system server
Test: change pin and immediately crash the kernel with
adb shell 'su root sh -c "echo c >/proc/sysrq-trigger"' and boot
Bug: 112175067

Change-Id: Ia5f43f3118e2297fbea43c805ef2f4577bf8a9bf
(cherry picked from commit 50e00c8dc4)
2018-08-17 21:09:03 +00:00
Nan Zhang
871e38a975 Merge "Make SQLiteDebug @TestApi" am: eb973c97da
am: c8255eea82

Change-Id: I77908f72147ee0682a307f23d708ab640ffdf4f0
2018-08-17 12:36:23 -07:00
Nan Zhang
eb973c97da Merge "Make SQLiteDebug @TestApi" 2018-08-17 16:53:11 +00:00
Fyodor Kupolov
6a82990b5f Make SQLiteDebug @TestApi
Otherwise, Metalava based android_test_current.jar doesn't contain
SQLiteDebug.class which caused build failure on:
out/target/common/obj/APPS/CtsDatabaseTestCases_intermediates/classes-full-debug.jar

Test: CtsDatabaseTestCases
Bug: 77543575
Change-Id: Ied9948e962fb7a81e56be0360cb94b0c318ad14d
Merged-In: Ied9948e962fb7a81e56be0360cb94b0c318ad14d
(cherry-picked from 6fe565e441)
Exempt-From-Owner-Approval: Cherry-picked a CL from git_master, and owner just agreed to use Exempt-From tag.
2018-08-17 16:32:17 +00:00
Mathew Inwood
9a5aa0c251 resolve merge conflicts of b41f50dfd0 to pi-dev-plus-aosp
Bug: None
Test: I solemnly swear I tested this conflict resolution.
Merged-In: I2bcc7f0eee31a1775dd7a5dd0f291d0d37bc1292
Change-Id: I227ce12b7419ce8541a362f98adc50965b779c3b
2018-08-16 10:25:51 +01:00
TreeHugger Robot
6bcd8150a4 Merge "Add slow query log for "open"." 2018-08-14 20:39:59 +00:00
TreeHugger Robot
064da5ff3d Merge "Add @UnsupportedAppUsage annotations" 2018-08-14 18:40:23 +00:00
Makoto Onuki
66813285ab Add slow query log for "open".
Bug: 111939259
Test: executed apps with slow query log enabled and check logcat output.
Change-Id: I803822cb06ed03fce10a45cc1498eaa951990688
2018-08-14 10:48:21 -07:00
Makoto Onuki
bb040710f2 Merge "Default to FULL synchronous mode for sqlite DBs in system server" 2018-08-10 15:30:17 +00:00
Mathew Inwood
f86bea9b42 Add @UnsupportedAppUsage annotations
For packages:
  android.database.sqlite
  android.database

This is an automatically generated CL. See go/UnsupportedAppUsage
for more details.

Exempted-From-Owner-Approval: Mechanical changes to the codebase
which have been approved by Android API council and announced on
android-eng@

Bug: 110868826
Test: m
Change-Id: I0da613d00611e89e49bf73c9ffe35700818efc6b
Merged-In: I2bcc7f0eee31a1775dd7a5dd0f291d0d37bc1292
2018-08-10 16:10:20 +01:00
Mathew Inwood
41b3194f6d Add @UnsupportedAppUsage annotations
For packages:
  android.database.sqlite
  android.database

This is an automatically generated CL. See go/UnsupportedAppUsage
for more details.

Exempted-From-Owner-Approval: Mechanical changes to the codebase
which have been approved by Android API council and announced on
android-eng@

Bug: 110868826
Test: m
Change-Id: I2bcc7f0eee31a1775dd7a5dd0f291d0d37bc1292
2018-08-10 16:00:53 +01:00
Makoto Onuki
50e00c8dc4 Default to FULL synchronous mode for sqlite DBs in system server
Test: change pin and immediately crash the kernel with
adb shell 'su root sh -c "echo c >/proc/sysrq-trigger"' and boot
Bug: 112175067

Change-Id: Ia5f43f3118e2297fbea43c805ef2f4577bf8a9bf
2018-08-09 11:00:54 -07:00
Jeff Sharkey
a5d0bf1701 RedactingCursor that redacts specific columns.
For new storage changes in Q, we need to redact the visibility of
the "_data" column on any returned cursors.  We could cook this up
using SQLiteQueryBuilder projection re-mappings, but that's a pretty
involved change due to the number of tables and teasing apart
public-vs-hidden API columns.

Instead, this RedactingCursor class offers to "redact" requested
columns by replacing with specific values, such as "/dev/null".

Test: atest frameworks/base/core/tests/coretests/src/android/database/RedactingCursorTest.java
Bug: 111960973
Change-Id: Iad36986fc0b7a0cae10ff216755e284320991e18
2018-08-09 11:50:24 -06:00
Makoto Onuki
6cb203357c Enhance slow-query log output
- Show DB filename
- Print result for execute-for-string and for-long to debug PRAGMA
related issues.

Bug: 112175067
Test: booted with "setprop db.log.slow_query_threshold 0" and
"setprop db.log.detailed 1"

Change-Id: I2a4093755b3fe7a6ae2c0fad93f8224da834062e
2018-08-08 13:54:08 -07:00
Jeff Sharkey
506994268b DO NOT MERGE. Extend SQLiteQueryBuilder for update and delete.
Developers often accept selection clauses from untrusted code, and
SQLiteQueryBuilder already supports a "strict" mode to help catch
SQL injection attacks.  This change extends the builder to support
update() and delete() calls, so that we can help secure those
selection clauses too.

Bug: 111085900
Test: atest packages/providers/DownloadProvider/tests/
Test: atest cts/tests/app/src/android/app/cts/DownloadManagerTest.java
Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Change-Id: Ib4fc8400f184755ee7e971ab5f2095186341730c
Merged-In: Ib4fc8400f184755ee7e971ab5f2095186341730c
2018-08-03 13:43:44 -06:00
Jeff Sharkey
57b04a8680 DO NOT MERGE. Execute "strict" queries with extra parentheses.
SQLiteQueryBuilder has a setStrict() mode which can be used to
detect SQL attacks from untrusted sources, which it does by running
each query twice: once with an extra set of parentheses, and if that
succeeds, it runs the original query verbatim.

This sadly doesn't catch inputs of the type "1=1) OR (1=1", which
creates valid statements for both tests above, but the final executed
query ends up leaking data due to SQLite operator precedence.

Instead, we need to continue compiling both variants, but we need
to execute the query with the additional parentheses to ensure
data won't be leaked.

Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Bug: 111085900
Change-Id: I6e8746fa48f9de13adae37d2990de11c9c585381
Merged-In: I6e8746fa48f9de13adae37d2990de11c9c585381
2018-08-03 13:43:40 -06:00
Makoto Onuki
a761d2b354 Rework slow-query log
- Allow per-uid slow query log with "db.log.slow_query_threshold.UID"
- Add db.log.bindargs to enable bind args log in slow query log
 (debuggable build only)

Bug: 111210683
Test: setprop db.log.slow_query_threshold 0 -> slow query log enabled for all processes
Test: setprop db.log.slow_query_threshold.ACORE-UID 0 -> slow query log enabled for acore
Test: setprop db.log.bindargs 1  + setprop db.log.slow_query_threshold.ACORE-UID 0 + kill acore ->
  Bind args also logged

Change-Id: I3e85479e0c946308837b827f856016bc4fbe1ea3
2018-08-02 20:33:19 +00:00
Jeff Sharkey
30b77bce53 Rewrite MediaStore thumbnail internals.
Keeps existing public APIs intact, but rewrite the internal to pave
the way for storage changes coming in Q.  All thumbnail generation
and reading now happens in MediaProvider, since apps won't have
direct access to MiniThumbFile files in Q.

Instead, InternalThumbnails now uses openTypedAssetFileDescriptor()
to open the original Uri, but with an EXTRA_SIZE indicating that
it'd like a thumbnail of a specific size.  This uses best-practices
which allow apps obtaining a Uri permission grant to a specific
media Uri to request a thumbnail of that media without additional
permissions.  This also lets us use CancellationSignal to cancel
thumbnail requests that are no longer needed, and we start checking
this signal at each query() point internally.

Move internals to use ExecutorService pattern, which will let us
expand the thread pool in a future CL based on device capabilities.

Previous logic that asynchronously generated thumbnails after certain
actions is now replaced with simpler invalidation.  This paves the
way for us to move proactive thumbnail generation to be driven by an
idle maintenance service, to be better stewards of battery life.  We
continue to generate thumbnails on-demand when not yet cached.

ThumbnailUtils already handles extracting EXIF thumbnails, so we don't
need to duplicate that work locally.

Flag that enables dumping of all raw SQL for debugging purposes on
a per-process basis.

Bug: 111268862, 111890672
Test: atest cts/tests/tests/provider/src/android/provider/cts/MediaStore*
Change-Id: Ibcdbc564c8fee09ddd33ebccc470fa5cbdff97b4
2018-07-28 17:40:04 -06:00
Jeff Sharkey
b89df9eac8 Support for appending "standalone" WHERE chunks.
The existing appendWhere() methods aren't very friendly for
developers, since they require manual tracking of state to decide if
subsequent standalone chunks should be prefixed with "AND".

While it's tempting to offer direct argument binding on the builder
class, we can't really deliver on that API in a secure way, so instead
add separate bindSelection() method which explicitly burns arguments
into a standalone selection string, which can then be appended to
the builder.

This was the last piece of new functionality being used by
SQLiteStatementBuilder, so we can delete that class and migrate
users back to SQLiteQueryBuilder.

Bug: 111268862
Test: atest frameworks/base/core/tests/coretests/src/android/database/DatabaseUtilsTest.java
Test: atest frameworks/base/core/tests/utiltests/src/com/android/internal/util/ArrayUtilsTest.java
Test: atest cts/tests/tests/provider/src/android/provider/cts/MediaStore*
Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Change-Id: I418f24338c90bae8a9dad473fa76329cea00a8c5
2018-07-27 13:45:51 -06:00
Jeff Sharkey
42122bfecf Bind update() args as Object[] for performance.
It's wasteful to convert them to String when SQLite already knows
how to bind specific data types, including funky types like byte[].

Also promote to public API, since they're generally useful.

Bug: 111085900
Test: atest packages/providers/DownloadProvider/tests/
Test: atest cts/tests/app/src/android/app/cts/DownloadManagerTest.java
Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Change-Id: I5b418bca1204773fd2795156a2f47906ca1e1a6b
2018-07-26 10:49:53 -06:00
Jeff Sharkey
b13ea30447 Extend SQLiteQueryBuilder for update and delete.
Developers often accept selection clauses from untrusted code, and
SQLiteQueryBuilder already supports a "strict" mode to help catch
SQL injection attacks.  This change extends the builder to support
update() and delete() calls, so that we can help secure those
selection clauses too.

Bug: 111085900
Test: atest packages/providers/DownloadProvider/tests/
Test: atest cts/tests/app/src/android/app/cts/DownloadManagerTest.java
Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Change-Id: Ib4fc8400f184755ee7e971ab5f2095186341730c
2018-07-25 15:25:56 -06:00
Jeff Sharkey
be8e0d00c2 Execute "strict" queries with extra parentheses.
SQLiteQueryBuilder has a setStrict() mode which can be used to
detect SQL attacks from untrusted sources, which it does by running
each query twice: once with an extra set of parentheses, and if that
succeeds, it runs the original query verbatim.

This sadly doesn't catch inputs of the type "1=1) OR (1=1", which
creates valid statements for both tests above, but the final executed
query ends up leaking data due to SQLite operator precedence.

Instead, we need to continue compiling both variants, but we need
to execute the query with the additional parentheses to ensure
data won't be leaked.

Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Bug: 111085900
Change-Id: I6e8746fa48f9de13adae37d2990de11c9c585381
2018-07-25 14:02:02 -06:00
Jeff Sharkey
e849fffd96 Merge "Revert SQLiteQueryBuilder for now." 2018-07-19 20:12:20 +00:00
Jeff Sharkey
91be9263e3 Revert SQLiteQueryBuilder for now.
We've encountered subtle bugs in how apps are using this public
API, so revert it back to exactly what shipped in the last
release, and move functionality to new SQLiteStatementBuilder
class, since we already have several customers using it.

Test: atest cts/tests/tests/provider/src/android/provider/cts/MediaStore*
Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Bug: 111486645
Change-Id: Ief059e987f2421e19f6f57a94320c313946a26d7
2018-07-19 09:52:20 -06:00
TreeHugger Robot
8877e2ef53 Merge "Update OWNERS for database code" 2018-07-19 00:11:15 +00:00
Makoto Onuki
b247a686e2 Update OWNERS for database code
Test: N/A

Change-Id: If657f1ac87d9a9e77162816c7ac6abb719b834d4
2018-07-18 15:56:36 -07:00
Jeff Sharkey
a58d6e3405 Execute "strict" queries with extra parentheses.
SQLiteQueryBuilder has a setStrict() mode which can be used to
detect SQL attacks from untrusted sources, which it does by running
each query twice: once with an extra set of parentheses, and if that
succeeds, it runs the original query verbatim.

This sadly doesn't catch inputs of the type "1=1) OR (1=1", which
creates valid statements for both tests above, but the final executed
query ends up leaking data due to SQLite operator precedence.

Instead, we need to continue compiling both variants, but we need
to execute the query with the additional parentheses to ensure
data won't be leaked.

Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Bug: 111085900
Change-Id: Ie85a95003ae134eef2fdfbf074c2f82d0a6a9f26
2018-07-16 17:21:50 -06:00
Jeff Sharkey
99cc118423 Add support for appending standalone phrases.
When users are building queries, they often need to append several
standalone SQL clauses, and it's tedious to track their first clause
so they can manually append " AND " to each subsequent clause.

So add new appendWherePhrase() API which appends a standalone phrase
which is AND'ed together with any existing WHERE query.

Also fix bug in update() which would turn null values into the
string literal "null" instead of passing them through as SQL NULL.

Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Bug: 111085900
Change-Id: Ia280dd864895654239503e080eaef925f5620d37
2018-07-16 14:56:03 -06:00
Jeff Sharkey
bba22006d6 GROUP BY and HAVING aren't ready to be strict.
Despite SQLite documentation saying these clauses follow the "expr"
syntax, queries will fail with obscure "row value misused" errors,
so remove the strict enforcement for now.

Test: manual
Bug: 111480817
Change-Id: Ib8cdcb4e6456110589203189fcfa14e75cd68718
2018-07-16 10:44:15 -06:00
Jeff Sharkey
6adc98c09c Extend SQLiteQueryBuilder for update and delete.
Developers often accept selection clauses from untrusted code, and
SQLiteQueryBuilder already supports a "strict" mode to help catch
SQL injection attacks.  This change extends the builder to support
update() and delete() calls, so that we can help secure those
selection clauses too.

Extend it to support selection arguments being provided when
appending appendWhere() clauses, meaning developers no longer need
to manually track their local selection arguments along with
remote arguments.

Extend it to support newer ContentProvider.query() variant that
accepts "Bundle queryArgs", and have all query() callers flow
through that common code path.  (This paves the way for a future
CL that will offer to gracefully extract non-WHERE clauses that
callers have tried smashing into their selections.)

Updates ContentValues to internally use more efficient ArrayMap.

Bug: 111268862
Test: atest frameworks/base/core/tests/utiltests/src/com/android/internal/util/ArrayUtilsTest.java
Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Change-Id: I60b6f69045766bb28d2f21a32c120ec8c383b917
2018-07-13 18:11:37 -06:00
Fyodor Kupolov
34c59f96dc Merge "New behavior of setSynchronousMode" into pi-dev am: 492457e94e
am: e953a1c5ab

Change-Id: I9bb84e2bafb1161fe85c7af05cf9fb4ae865e212
2018-06-04 16:51:53 -07:00
Fyodor Kupolov
8ba2089165 New behavior of setSynchronousMode
setSynchronousMode is now applicable to any journal mode.
This can be useful for apps that want to use stricter durability
settings for their database.

Test: android.database.sqlite.cts.SQLiteDatabaseTest
Bug: 86953240
Change-Id: I7dbaa6d0bc0975da8067e87b402f63f51b4add10
2018-06-01 12:11:42 -07:00
Fyodor Kupolov
6fe565e441 Make SQLiteDebug @TestApi
Test: CtsDatabaseTestCases
Bug: 77543575
Change-Id: Ied9948e962fb7a81e56be0360cb94b0c318ad14d
2018-05-07 16:13:08 -07:00
Joshua Baxter
b3c20232f1 docs: fixed typo
am: 3639e2f929

Change-Id: I6f7381045bb25fdcc9f86e3e2dd0d7b8c4eda853
2018-03-27 17:34:53 +00:00