Commit Graph

519 Commits

Author SHA1 Message Date
TreeHugger Robot
6bcd8150a4 Merge "Add slow query log for "open"." 2018-08-14 20:39:59 +00:00
TreeHugger Robot
064da5ff3d Merge "Add @UnsupportedAppUsage annotations" 2018-08-14 18:40:23 +00:00
Makoto Onuki
66813285ab Add slow query log for "open".
Bug: 111939259
Test: executed apps with slow query log enabled and check logcat output.
Change-Id: I803822cb06ed03fce10a45cc1498eaa951990688
2018-08-14 10:48:21 -07:00
Makoto Onuki
bb040710f2 Merge "Default to FULL synchronous mode for sqlite DBs in system server" 2018-08-10 15:30:17 +00:00
Mathew Inwood
41b3194f6d Add @UnsupportedAppUsage annotations
For packages:
  android.database.sqlite
  android.database

This is an automatically generated CL. See go/UnsupportedAppUsage
for more details.

Exempted-From-Owner-Approval: Mechanical changes to the codebase
which have been approved by Android API council and announced on
android-eng@

Bug: 110868826
Test: m
Change-Id: I2bcc7f0eee31a1775dd7a5dd0f291d0d37bc1292
2018-08-10 16:00:53 +01:00
Makoto Onuki
50e00c8dc4 Default to FULL synchronous mode for sqlite DBs in system server
Test: change pin and immediately crash the kernel with
adb shell 'su root sh -c "echo c >/proc/sysrq-trigger"' and boot
Bug: 112175067

Change-Id: Ia5f43f3118e2297fbea43c805ef2f4577bf8a9bf
2018-08-09 11:00:54 -07:00
Jeff Sharkey
a5d0bf1701 RedactingCursor that redacts specific columns.
For new storage changes in Q, we need to redact the visibility of
the "_data" column on any returned cursors.  We could cook this up
using SQLiteQueryBuilder projection re-mappings, but that's a pretty
involved change due to the number of tables and teasing apart
public-vs-hidden API columns.

Instead, this RedactingCursor class offers to "redact" requested
columns by replacing with specific values, such as "/dev/null".

Test: atest frameworks/base/core/tests/coretests/src/android/database/RedactingCursorTest.java
Bug: 111960973
Change-Id: Iad36986fc0b7a0cae10ff216755e284320991e18
2018-08-09 11:50:24 -06:00
Makoto Onuki
6cb203357c Enhance slow-query log output
- Show DB filename
- Print result for execute-for-string and for-long to debug PRAGMA
related issues.

Bug: 112175067
Test: booted with "setprop db.log.slow_query_threshold 0" and
"setprop db.log.detailed 1"

Change-Id: I2a4093755b3fe7a6ae2c0fad93f8224da834062e
2018-08-08 13:54:08 -07:00
Makoto Onuki
a761d2b354 Rework slow-query log
- Allow per-uid slow query log with "db.log.slow_query_threshold.UID"
- Add db.log.bindargs to enable bind args log in slow query log
 (debuggable build only)

Bug: 111210683
Test: setprop db.log.slow_query_threshold 0 -> slow query log enabled for all processes
Test: setprop db.log.slow_query_threshold.ACORE-UID 0 -> slow query log enabled for acore
Test: setprop db.log.bindargs 1  + setprop db.log.slow_query_threshold.ACORE-UID 0 + kill acore ->
  Bind args also logged

Change-Id: I3e85479e0c946308837b827f856016bc4fbe1ea3
2018-08-02 20:33:19 +00:00
Jeff Sharkey
30b77bce53 Rewrite MediaStore thumbnail internals.
Keeps existing public APIs intact, but rewrite the internal to pave
the way for storage changes coming in Q.  All thumbnail generation
and reading now happens in MediaProvider, since apps won't have
direct access to MiniThumbFile files in Q.

Instead, InternalThumbnails now uses openTypedAssetFileDescriptor()
to open the original Uri, but with an EXTRA_SIZE indicating that
it'd like a thumbnail of a specific size.  This uses best-practices
which allow apps obtaining a Uri permission grant to a specific
media Uri to request a thumbnail of that media without additional
permissions.  This also lets us use CancellationSignal to cancel
thumbnail requests that are no longer needed, and we start checking
this signal at each query() point internally.

Move internals to use ExecutorService pattern, which will let us
expand the thread pool in a future CL based on device capabilities.

Previous logic that asynchronously generated thumbnails after certain
actions is now replaced with simpler invalidation.  This paves the
way for us to move proactive thumbnail generation to be driven by an
idle maintenance service, to be better stewards of battery life.  We
continue to generate thumbnails on-demand when not yet cached.

ThumbnailUtils already handles extracting EXIF thumbnails, so we don't
need to duplicate that work locally.

Flag that enables dumping of all raw SQL for debugging purposes on
a per-process basis.

Bug: 111268862, 111890672
Test: atest cts/tests/tests/provider/src/android/provider/cts/MediaStore*
Change-Id: Ibcdbc564c8fee09ddd33ebccc470fa5cbdff97b4
2018-07-28 17:40:04 -06:00
Jeff Sharkey
b89df9eac8 Support for appending "standalone" WHERE chunks.
The existing appendWhere() methods aren't very friendly for
developers, since they require manual tracking of state to decide if
subsequent standalone chunks should be prefixed with "AND".

While it's tempting to offer direct argument binding on the builder
class, we can't really deliver on that API in a secure way, so instead
add separate bindSelection() method which explicitly burns arguments
into a standalone selection string, which can then be appended to
the builder.

This was the last piece of new functionality being used by
SQLiteStatementBuilder, so we can delete that class and migrate
users back to SQLiteQueryBuilder.

Bug: 111268862
Test: atest frameworks/base/core/tests/coretests/src/android/database/DatabaseUtilsTest.java
Test: atest frameworks/base/core/tests/utiltests/src/com/android/internal/util/ArrayUtilsTest.java
Test: atest cts/tests/tests/provider/src/android/provider/cts/MediaStore*
Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Change-Id: I418f24338c90bae8a9dad473fa76329cea00a8c5
2018-07-27 13:45:51 -06:00
Jeff Sharkey
42122bfecf Bind update() args as Object[] for performance.
It's wasteful to convert them to String when SQLite already knows
how to bind specific data types, including funky types like byte[].

Also promote to public API, since they're generally useful.

Bug: 111085900
Test: atest packages/providers/DownloadProvider/tests/
Test: atest cts/tests/app/src/android/app/cts/DownloadManagerTest.java
Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Change-Id: I5b418bca1204773fd2795156a2f47906ca1e1a6b
2018-07-26 10:49:53 -06:00
Jeff Sharkey
b13ea30447 Extend SQLiteQueryBuilder for update and delete.
Developers often accept selection clauses from untrusted code, and
SQLiteQueryBuilder already supports a "strict" mode to help catch
SQL injection attacks.  This change extends the builder to support
update() and delete() calls, so that we can help secure those
selection clauses too.

Bug: 111085900
Test: atest packages/providers/DownloadProvider/tests/
Test: atest cts/tests/app/src/android/app/cts/DownloadManagerTest.java
Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Change-Id: Ib4fc8400f184755ee7e971ab5f2095186341730c
2018-07-25 15:25:56 -06:00
Jeff Sharkey
be8e0d00c2 Execute "strict" queries with extra parentheses.
SQLiteQueryBuilder has a setStrict() mode which can be used to
detect SQL attacks from untrusted sources, which it does by running
each query twice: once with an extra set of parentheses, and if that
succeeds, it runs the original query verbatim.

This sadly doesn't catch inputs of the type "1=1) OR (1=1", which
creates valid statements for both tests above, but the final executed
query ends up leaking data due to SQLite operator precedence.

Instead, we need to continue compiling both variants, but we need
to execute the query with the additional parentheses to ensure
data won't be leaked.

Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Bug: 111085900
Change-Id: I6e8746fa48f9de13adae37d2990de11c9c585381
2018-07-25 14:02:02 -06:00
Jeff Sharkey
e849fffd96 Merge "Revert SQLiteQueryBuilder for now." 2018-07-19 20:12:20 +00:00
Jeff Sharkey
91be9263e3 Revert SQLiteQueryBuilder for now.
We've encountered subtle bugs in how apps are using this public
API, so revert it back to exactly what shipped in the last
release, and move functionality to new SQLiteStatementBuilder
class, since we already have several customers using it.

Test: atest cts/tests/tests/provider/src/android/provider/cts/MediaStore*
Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Bug: 111486645
Change-Id: Ief059e987f2421e19f6f57a94320c313946a26d7
2018-07-19 09:52:20 -06:00
TreeHugger Robot
8877e2ef53 Merge "Update OWNERS for database code" 2018-07-19 00:11:15 +00:00
Makoto Onuki
b247a686e2 Update OWNERS for database code
Test: N/A

Change-Id: If657f1ac87d9a9e77162816c7ac6abb719b834d4
2018-07-18 15:56:36 -07:00
Jeff Sharkey
a58d6e3405 Execute "strict" queries with extra parentheses.
SQLiteQueryBuilder has a setStrict() mode which can be used to
detect SQL attacks from untrusted sources, which it does by running
each query twice: once with an extra set of parentheses, and if that
succeeds, it runs the original query verbatim.

This sadly doesn't catch inputs of the type "1=1) OR (1=1", which
creates valid statements for both tests above, but the final executed
query ends up leaking data due to SQLite operator precedence.

Instead, we need to continue compiling both variants, but we need
to execute the query with the additional parentheses to ensure
data won't be leaked.

Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Bug: 111085900
Change-Id: Ie85a95003ae134eef2fdfbf074c2f82d0a6a9f26
2018-07-16 17:21:50 -06:00
Jeff Sharkey
99cc118423 Add support for appending standalone phrases.
When users are building queries, they often need to append several
standalone SQL clauses, and it's tedious to track their first clause
so they can manually append " AND " to each subsequent clause.

So add new appendWherePhrase() API which appends a standalone phrase
which is AND'ed together with any existing WHERE query.

Also fix bug in update() which would turn null values into the
string literal "null" instead of passing them through as SQL NULL.

Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Bug: 111085900
Change-Id: Ia280dd864895654239503e080eaef925f5620d37
2018-07-16 14:56:03 -06:00
Jeff Sharkey
bba22006d6 GROUP BY and HAVING aren't ready to be strict.
Despite SQLite documentation saying these clauses follow the "expr"
syntax, queries will fail with obscure "row value misused" errors,
so remove the strict enforcement for now.

Test: manual
Bug: 111480817
Change-Id: Ib8cdcb4e6456110589203189fcfa14e75cd68718
2018-07-16 10:44:15 -06:00
Jeff Sharkey
6adc98c09c Extend SQLiteQueryBuilder for update and delete.
Developers often accept selection clauses from untrusted code, and
SQLiteQueryBuilder already supports a "strict" mode to help catch
SQL injection attacks.  This change extends the builder to support
update() and delete() calls, so that we can help secure those
selection clauses too.

Extend it to support selection arguments being provided when
appending appendWhere() clauses, meaning developers no longer need
to manually track their local selection arguments along with
remote arguments.

Extend it to support newer ContentProvider.query() variant that
accepts "Bundle queryArgs", and have all query() callers flow
through that common code path.  (This paves the way for a future
CL that will offer to gracefully extract non-WHERE clauses that
callers have tried smashing into their selections.)

Updates ContentValues to internally use more efficient ArrayMap.

Bug: 111268862
Test: atest frameworks/base/core/tests/utiltests/src/com/android/internal/util/ArrayUtilsTest.java
Test: atest cts/tests/tests/database/src/android/database/sqlite/cts/SQLiteQueryBuilderTest.java
Change-Id: I60b6f69045766bb28d2f21a32c120ec8c383b917
2018-07-13 18:11:37 -06:00
Fyodor Kupolov
34c59f96dc Merge "New behavior of setSynchronousMode" into pi-dev am: 492457e94e
am: e953a1c5ab

Change-Id: I9bb84e2bafb1161fe85c7af05cf9fb4ae865e212
2018-06-04 16:51:53 -07:00
Fyodor Kupolov
8ba2089165 New behavior of setSynchronousMode
setSynchronousMode is now applicable to any journal mode.
This can be useful for apps that want to use stricter durability
settings for their database.

Test: android.database.sqlite.cts.SQLiteDatabaseTest
Bug: 86953240
Change-Id: I7dbaa6d0bc0975da8067e87b402f63f51b4add10
2018-06-01 12:11:42 -07:00
Fyodor Kupolov
6fe565e441 Make SQLiteDebug @TestApi
Test: CtsDatabaseTestCases
Bug: 77543575
Change-Id: Ied9948e962fb7a81e56be0360cb94b0c318ad14d
2018-05-07 16:13:08 -07:00
Joshua Baxter
b3c20232f1 docs: fixed typo
am: 3639e2f929

Change-Id: I6f7381045bb25fdcc9f86e3e2dd0d7b8c4eda853
2018-03-27 17:34:53 +00:00
Joshua Baxter
3639e2f929 docs: fixed typo
Test: make ds-docs

Bug: 36941741

Change-Id: I451d8095bcc5d30dbefe3503441d718a69d821ca
2018-03-26 18:17:08 -07:00
Fyodor Kupolov
681ec3128e Include additional information in the dumpsys
Test: adb shell dumpsys dbinfo system
Bug: 64262688
Change-Id: I43d76e505e4d5598994c9af3e615c572017f120c
2018-03-20 18:50:07 -07:00
Fyodor Kupolov
692573b1b2 Introduced DISABLE_COMPATIBILITY_WAL flag
If set, Compatibility WAL will not be used.
Currently this flag is set if SQLiteOpenHelper.setWriteAheadLogging(false)
is called before opening the database. Previously this call was ignored
since ENABLE_WRITE_AHEAD_LOGGING flag is only set when setWriteAheadLogging(true)

Test: CtsDatabaseTestCases
Bug: 74116447
Change-Id: Ic486e178b9da5d747840be739303a2685b91f35b
2018-03-06 13:31:30 -08:00
Fyodor Kupolov
ab05b143ba Added SQLiteOpenHelper.setOpenParams
Using the new constructor can be difficult as it requires an app to
make a decision which version of constructor to use before instantiating
the object. By adding setOpenParams method, apps can keep the old
structure of the code and only add API level check after instantiating
a helper object.

Test: SQLiteOpenHelperTest
Bug: 70863722
Change-Id: I1705dd790e3549d0d8e75eb33b1d9b5bdadcb3c9
2018-02-12 15:33:13 -08:00
Andreas Gampe
3f24e69dbe Frameworks: Annotate trivial @GuardedBy in core/java
Add @GuardedBy for simple functions that require a single lock
and are named XYZLocked.

Bug: 73000847
Test: m
Test: m javac-check-framework RUN_ERROR_PRONE=true
Change-Id: Icb5114fea2ff2385e1cc7511121026099e05c0ee
2018-02-08 02:19:42 -08:00
Fyodor Kupolov
a17858d072 Merge "Fix JavaDoc for enableWriteAheadLogging" 2018-02-07 01:51:57 +00:00
Jeff Sharkey
ad357d1839 Pass in the user defined by Context.
The majority of Manager-style classes already use Context.getUserId()
when making calls into the OS, so clean up the remaining callers to
unify behind this strategy.

This gives @SystemApi developers a nice clean interface to interact
across user boundaries, instead of manually adding "AsUser" or
"ForUser" method variants, which would quickly become unsustainable.

Test: builds, boots
Bug: 72863821
Exempt-From-Owner-Approval: trivial changes
Change-Id: Ib772ec4438e57a2ad4950821b9432f9842998451
2018-02-03 02:11:45 +00:00
Fyodor Kupolov
4b5c35984c Fix JavaDoc for enableWriteAheadLogging
Test: build
Bug: 65206410
Bug: 64262688
Change-Id: I92e27b048ae8717c5007ac39d6a6de7b59dc7072
2018-02-02 14:57:50 -08:00
Chris Craik
0834e3561e @FastNative for CursorWindow
Test: adb install -r $OUT/data/app/CorePerfTests/CorePerfTests.apk && adb shell cmd package compile -m speed -f com.android.perftests.core && adb shell am instrument -w -e class android.database.CursorWindowPerfTest com.android.perftests.core/android.support.test.runner.AndroidJUnitRunner

Change-Id: I4d2f8846592426b993ca44a187ac0e2de7313170
2018-01-26 15:03:49 -08:00
Fyodor Kupolov
2abd2a4976 Clarify usage of context in the JavaDoc
The context is only used for resolving database paths.

Test: build
Bug: 70863722
Change-Id: I675dcd35da9d73b57dbd0bf3e357bfffdf485168
2018-01-18 01:39:10 +00:00
Fyodor Kupolov
7fcd659705 Enabled SQLiteCompatibilityWalFlags class
In system_server, it should be deactivated until InstallSystemProviders
is finished.

Test: SQLiteCompatibilityWalFlagsTest
Test: setting put global ... + verify that dumpsys dbinfo has the new flag
Bug: 70226732
Bug: 70517616
Change-Id: Iec6a5e61a5d7e4cc0ac602f2b630357c54bb4456
2017-12-15 17:34:52 -08:00
Fyodor Kupolov
ee90c03fe7 Added compatibility WAL flags for Global.Settings
Added Settings.Global.SQLITE_COMPATIBILITY_WAL_FLAGS -
configuration flags for SQLite Compatibility WAL. Encoded as a key-value
list, separated by commas. E.g.:
compatibility_wal_supported=true, wal_syncmode=OFF

SQLiteCompatibilityWalFlags caches the value of
SQLITE_COMPATIBILITY_WAL_FLAGS on first access and keeps it through
the lifetime of the process for consistent behavior across all
connections.

Test: SQLiteCompatibilityWalFlagsTest
Test: setting put global ... + verify that dumpsys dbinfo has the new flag
Bug: 70226732
Bug: 70517616
Change-Id: Ifacbf5908c83351ebe5dea676eeb716af039fb14
2017-12-14 12:21:27 -08:00
Fyodor Kupolov
f0d4810026 Added database package OWNERS
Test: N/A
Change-Id: I30993e7e20265b88d2cda246155fde50081866cb
2017-11-28 14:14:37 -08:00
Fyodor Kupolov
25095c0802 Close available secondary connections if schema changes
If DDL statement is executed on one connection, the schema change is
not always visible to other connections due to sqlite3_column_count()
API limitation. We have to close secondary connections to prevent the
issue.

Test: cts/SQLiteOpenHelperTest
Bug: 33695159
Change-Id: I862da71ecec5b1edc880dbfadf22efb2274ca10a
2017-11-17 15:22:20 -08:00
TreeHugger Robot
3c31b2f32a Merge "Allow null context in the constructor" 2017-11-16 02:52:31 +00:00
Fyodor Kupolov
9d4c3d9a50 Allow null context in the constructor
Legacy apps can use it for in-memory databases.

Test: manual
Bug:69379861
Change-Id: I989524c38d6e2ea3b2637bbe6534f2906c5b7bb6
2017-11-15 16:04:24 -08:00
TreeHugger Robot
b0ce0a8209 Merge "Fixed typo in MergeCursor" 2017-11-15 19:45:37 +00:00
Fyodor Kupolov
5c4b623d54 Fixed typo in MergeCursor
Test: n/a
Bug: 68247902
Change-Id: I484d3aa23415769b205635908f7d73d8113d866b
2017-11-15 00:05:04 +00:00
Fyodor Kupolov
d3fe8f09c8 Merge "Apps can now specify journal/synchronous mode" 2017-11-14 17:53:06 +00:00
Kevin Hufnagle
0dd496b455 Merge "Merge "docs: Added SQLite versions used on API levels 26 and 27." into oc-mr1-dev am: 9e628b60ba" into oc-mr1-dev-plus-aosp
am: be8f36d844

Change-Id: I1f64688b30ce70b66a2f07bc47fd199dc1e3dcb0
2017-11-10 17:32:10 +00:00
Kevin Hufnagle
5dfdfa0a14 docs: Added SQLite versions used on API levels 26 and 27.
The android.database.sqlite package summary page now mentions the
versions of SQLite that are available on Android devices running Oreo
(API level 26) and API level 27.

Test: make ds-docs -j16

Bug: 69106585
Change-Id: I77a323c58343fa978ade5397d613566ba25f00c2
2017-11-09 10:12:50 -08:00
Fyodor Kupolov
13a4b37e87 Apps can now specify journal/synchronous mode
Added setJournalMode/setSynchronousMode. These methods control
journal/syncrhonous mode when ENABLE_WRITE_AHEAD_LOGGING flag is not set

Bug: 33044236
Test: manual + CtsDatabaseTestCases
Change-Id: Iffef75f6941030aae5ab3e239cd65550515f5fe0
2017-11-08 16:13:40 -08:00
Fyodor Kupolov
fd22a196a6 Update wal mode of the connection when disabling WAL
Previously calling disableWriteAheadLogging wasn't updating
journal mode of the primary connection.

Test: manual + SQLiteDatabaseTest cts
Bug: 33044236
Change-Id: Ifd105dacdd10ef59b10a023aba0cc43fbb1bba97
2017-11-06 15:08:59 -08:00
TreeHugger Robot
01f5db7240 Merge "Added setFillWindowForwardOnly" 2017-11-02 02:10:46 +00:00