The main theme of this change is encapsulation. This change
preserves all existing functionality but the implementation
is now much cleaner.
Instead of a "database lock", access to the database is treated
as a resource acquisition problem. If a thread's owns a database
connection, then it can access the database; otherwise, it must
acquire a database connection first, and potentially wait for other
threads to give up theirs. The SQLiteConnectionPool encapsulates
the details of how connections are created, configured, acquired,
released and disposed.
One new feature is that SQLiteConnectionPool can make scheduling
decisions about which thread should next acquire a database
connection when there is contention among threads. The factors
considered include wait queue ordering (fairness among peers),
whether the connection is needed for an interactive operation
(unfairness on behalf of the UI), and whether the primary connection
is needed or if any old connection will do. Thus one goal of the
new SQLiteConnectionPool is to improve the utilization of
database connections.
To emulate some quirks of the old "database lock," we introduce
the concept of the primary database connection. The primary
database connection is the one that is typically used to perform
write operations to the database. When a thread holds the primary
database connection, it effectively prevents other threads from
modifying the database (although they can still read). What's
more, those threads will block when they try to acquire the primary
connection, which provides the same kind of mutual exclusion
features that the old "database lock" had. (In truth, we
probably don't need to be requiring use of the primary database
connection in as many places as we do now, but we can seek to refine
that behavior in future patches.)
Another significant change is that native sqlite3_stmt objects
(prepared statements) are fully encapsulated by the SQLiteConnection
object that owns them. This ensures that the connection can
finalize (destroy) all extant statements that belong to a database
connection when the connection is closed. (In the original code,
this was very complicated because the sqlite3_stmt objects were
managed by SQLiteCompiledSql objects which had different lifetime
from the original SQLiteDatabase that created them. Worse, the
SQLiteCompiledSql finalizer method couldn't actually destroy the
sqlite3_stmt objects because it ran on the finalizer thread and
therefore could not guarantee that it could acquire the database
lock in order to do the work. This resulted in some rather
tortured logic involving a list of pending finalizable statements
and a high change of deadlocks or leaks.)
Because sqlite3_stmt objects never escape the confines of the
SQLiteConnection that owns them, we can also greatly simplify
the design of the SQLiteProgram, SQLiteQuery and SQLiteStatement
objects. They no longer have to wrangle a native sqlite3_stmt
object pointer and manage its lifecycle. So now all they do
is hold bind arguments and provide a fancy API.
All of the JNI glue related to managing database connections
and performing transactions is now bound to SQLiteConnection
(rather than being scattered everywhere). This makes sense because
SQLiteConnection owns the native sqlite3 object, so it is the
only class in the system that can interact with the native
SQLite database directly. Encapsulation for the win.
One particularly tricky part of this change is managing the
ownership of SQLiteConnection objects. At any given time,
a SQLiteConnection is either owned by a SQLiteConnectionPool
or by a SQLiteSession. SQLiteConnections should never be leaked,
but we handle that case too (and yell about it with CloseGuard).
A SQLiteSession object is responsible for acquiring and releasing
a SQLiteConnection object on behalf of a single thread as needed.
For example, the session acquires a connection when a transaction
begins and releases it when finished. If the session cannot
acquire a connection immediately, then the requested operation
blocks until a connection becomes available.
SQLiteSessions are thread-local. A SQLiteDatabase assigns a
distinct session to each thread that performs database operations.
This is very very important. First, it prevents two threads
from trying to use the same SQLiteConnection at the same time
(because two threads can't share the same session).
Second, it prevents a single thread from trying to acquire two
SQLiteConnections simultaneously from the same database (because
a single thread can't have two sessions for the same database which,
in addition to being greedy, could result in a deadlock).
There is strict layering between the various database objects,
objects at lower layers are not aware of objects at higher layers.
Moreover, objects at higher layers generally own objects at lower
layers and are responsible for ensuring they are properly disposed
when no longer needed (good for the environment).
API layer: SQLiteDatabase, SQLiteProgram, SQLiteQuery, SQLiteStatement.
Session layer: SQLiteSession.
Connection layer: SQLiteConnectionPool, SQLiteConnection.
Native layer: JNI glue.
By avoiding cyclic dependencies between layers, we make the
architecture much more intelligible, maintainable and robust.
Finally, this change adds a great deal of new debugging information.
It is now possible to view a list of the most recent database
operations including how long they took to run using
"adb shell dumpsys dbinfo". (Because most of the interesting
work happens in SQLiteConnection, it is easy to add debugging
instrumentation to track all database operations in one place.)
Change-Id: Iffb4ce72d8bcf20b4e087d911da6aa84d2f15297
For non-localized strings like the keywords in SQL statements, we shouldn't use
locale-specific case transformations.
(Cherry pick of 03f8f84c345e7c861a6de889d434b9d82cf04c8d.)
Bug: 4284951
Change-Id: Ib5180daf7892af7b645c906fdfeacf9da87bd667
We can do this because the classes already cannot be subclassed
by applications due to the fact they only have package private
constructors.
One very nice consequence of this observation is that we can hide or
delete several @deprecated protected members which are effectively
inaccessible because applications cannot create subclasses!
Change-Id: I2d3a0d2ad72b9289ebcdf907e4e4e6caf27f9076
On user-debug and eng builds, you can set the
"db.log.slow_query_threshold" system property to queries that
take longer than the specified number of milliseconds.
Set it to 0 to log all queries.
This property has been around for a while but it was implemented
poorly. In particular, it *changed* the behavior of the query
by calling getCount() while holding the Db connection.
In normal operation, the query will not actually run until later.
By putting the timing logic into fillWindow() instead, we ensure
that we only measure queries that actually ran. We also capture
cases where the cursor window gets filled multiple times.
Bug: 5520301
Change-Id: I174f5e1ea15831a1d22a36e9a804d7755f230b38
Bug: 5520301
When an application requests a row from a SQLiteCursor that
is not in the window, instead of filling from the requested
row position onwards, fill from a little bit ahead of the
requested row position.
This fixes a problem with applications that seek backwards
in large cursor windows. Previously the application could
end up refilling the window every time it moved back
one position.
We try to fill about 1/3 before the requested position and
2/3 after which substantially improves scrolling responsiveness
when the list is bound to a data set that does not fit
entirely within one cursor window.
Change-Id: I168ff1d3aed1a41ac96267be34a026c108590e52
When the StrictMode VM flag is changed mid-lifecycle, pass along at
least a basic trace.
Bug: 5516561
Change-Id: I6c922ba20f34349c35624cfc04d1c71ee56ad6ef
Bug: 5220669
The CrossProcessCursorWrapper is a useful building block
for wrapping cursors that will be sent to processes remotely.
It can also transform normal Cursors into CrossProcessCursors
transparently.
The new class fixes common performance problems and bugs
that applications have encountered when implementing
ContentProviders.
Change-Id: Icc59fec10add3f7d8cfbd0495447860c1872c752
Use the CrossProcessCursor interface in the way it was intended
without introducing special cases for AbstractWindowedCursor.
This is possible now that we do not distinguish between local-only
and remotable CursorWindows so we don't need to provide a window
to the AbstractWindowedCursor for it to fill; it can provide one
for itself if it wants one.
This logic makes it possible to create CrossProcessCursor
implementations that perform just as well as AbstractWindowedCursors.
Change-Id: I764b25ee6311d28c50d1930705346b265faec86a
Bug: 5218310
This change fixes problems calling the default fillWindow
on Cursors that contain BLOBs. It should also be more efficient
by avoiding redundant string conversions for numeric datatypes.
Change-Id: Ied515bf6299bc8d3c14e76055d85fd35e7c05952
There is no difference and has never really been a difference
between local-only and remotable CursorWindows. By removing the
distinction officially in the API, we will make it easier to
implement CrossProcessCursor correctly. CrossProcessCursor
is problematic currently because it's not clear whether a call
to getWindow() will return a local-only window or a remotable window.
As a result, the bulk cursor adaptor has special case handling
for AbstractWindowedCursors vs. ordinary CrossProcessCursors
so that it can set a remotable window before the cursor fills it.
All these problems go away if we just forget about local-only
windows being special in any way.
Change-Id: Ie59f517968e33d0ecb239c3c4f60206495e8f376
Bug: 5332296
The memory dealer introduces additional delays for reclaiming
the memory owned by CursorWindows because the Binder object must
be finalized. Using ashmem instead gives CursorWindow more
direct control over the lifetime of the shared memory region.
The provider now allocates the CursorWindows and returns them
to clients with a read-only protection bit set on the ashmem
region.
Improved the encapsulation of CursorWindow. Callers shouldn't
need to care about details like how string fields are allocated.
Removed the compile-time configuration of string and numeric
storage modes to remove some dead weight.
Change-Id: I07c2bc2a9c573d7e435dcaecd269d25ea9807acd
Bug: 5332296
Ensure that there is always an owner for each CursorWindow
and that references to each window are acquired/released
appropriately at all times.
Added synchronization to CursorToBulkCursorAdaptor to
prevent the underlying Cursor and CursorWindow from being
remotely accessed in ways that might violate invariants,
resulting in leaks or other problems.
Ensured that CursorToBulkCursorAdaptor promptly releases
its references to the Cursor and CursorWindow when closed
so they don't stick around longer than they should, even
if the remote end hangs onto the IBulkCursor for some reason.
CursorWindow respects Parcelable.FLAG_WRITE_RETURN_VALUE
as an indication that one reference to the CursorWindow is
being released. Correspondingly, CursorToBulkCursorAdaptor
acquires a reference to the CursorWindow before returning
it to the caller. This change also prevents races from
resulting in the transfer of an invalid CursorWindow over
the wire.
Ensured that BulkCursorToCursorAdaptor promptly releases
its reference to the IBulkCursor when closed and throws
on attempts to access the cursor while closed.
Modified ContentProviderNative to handle both parts of
the wrapping and unwrapping of Cursors into IBulkCursors.
This makes it a lot easier to ensure that the right
things happen on both ends. Also, it turns out that
the only caller of IContentProvider.bulkQuery was
ContentProviderNative itself so there was no need
to support bulkQuery on ContentProviderProxy and it was
just getting in the way.
Implement CloseGuard on CursorWindow.
Change-Id: Ib3c8305d3cc62322f38a06698d404a2989bb6ef9
Bug: 5332296
We can't replace these with AIDL generated proxies just yet, but
at least we can make them a little more conformant.
Change-Id: I1814f76d0f9c5e44a7fd85a12b2e3c2b7e3c9daa
Bug: 5332296
Removed dead code in SQLiteCursor related to the use of a background
query thread. This code could result in CursorWindows being modified
concurrently or used after free. This code is broken, unused and
is just in the way.
Added comments to explain how CursorWindow ownership is
supposed to work for AbstractWindowedCursors. (There are still cases
where cursor windows get dropped on the floor without being closed.
Those will be taken care of in a subsequent patch.)
Cleaned up SQLiteQuery.fillWindow to eliminate duplicate code and
remove bits that were only needed for background loading, like
returning -1.
Change-Id: I03e8e2e73ff0c11df76d63f57df4c5ada06ae1cb
Bug: 5332296
The code is functionally equivalent, but a little more efficient
and much easier to maintain.
Change-Id: I90670a13799df05831843a5137ab234929281b7c
db_sample logging is thrashing the event log hard, and is currently
neither maintained nor heeded by anybody on the current release. It
can be re-enabled simply by throwing the appropriate static boolean
to 'true', but for now this should greatly improve the utility of our
event logs. (We were seeing a rollover period of 20 minutes or less;
ideally we want to see the event log run at least half a day before
rolling.)
Bug 5419627
Bug 5104300
Change-Id: I2125544130aae142974102dbad3b557e49fcd494
This allows providers to return SQLiteCursor with a bundle, without wrapping
it in a CursorWrapper, which has performance penalty on cross-process
queries.
Bug 5220669
Change-Id: I46b9dfb3d8e01e9caf080bb24acfd97551441e25
This introduces a new facility for code during the boot process
to display messages to the user through a progress dialog. This
is only for use when performing longer-than-usual post-upgrade
operations such as running dexopt on applications or upgrading
databases.
Change-Id: I0e78439ccec3850fb67872c22f235bf12a158dae
log time in the following 2 situations
1. all transactions. time measured = wall time between begin-commit
2. queries (which are not in tranactions)
Change-Id: I67be9487a96072695aff3529ba4a257f4c8ec596
Dynamically changing a max cache size is clumsy; almost everyone
should set this when they create the cache.
Fix SQLiteDatabase to copy entries into a new cache when the size
is changed. In pratice this will always be immediately after the
SQLiteDatabase is created. Since the cache field is no longer
final, change the guard on the cache field to the SQLiteDatabase
instance itself.
Change-Id: I4e325f06edc551636723568a52770c0982e2d945
they are becoming pretty disruptive bugs.
many more instances of gmail crashing with locking protocol error.
and a positive identification of WAL as the problem in Music app
AND one instance of gmail crash that I know of.
who knows how many folks are NOT reporting the gmail crash.
too bad we relied on pre-released version sqlite feature
without seeing it complettly tested in the field.
not a safe feature to turn on at this point in time.
maybe more testing and debugging in Ice timeframe.
Change-Id: I283ad26ba7e1793772a372aa8e24df0cb96ce2ef
should help a little bit in debugging deadlocks in the process.
especially when the an application does this:
thread1: begin transaction
thread2: synchronized void blahMethod { do some code; begintransaction;}
thread1: with lock ion the database, it now tries to get into the above
synchronized method
thread2 has synchronized monitor and thred1 has db lock.
each wants the other lock
deadlock.
and our stacktraces will not tell us that thread1 has the database lock.
with this CL, we will know WHO owns the database lock and which thread
dies waiting for the lock.
Change-Id: If51578e7fa0a70ee3d762eb2233d1fddd98a2e86
