Commit Graph

5647 Commits

Author SHA1 Message Date
Tej Singh
affc08a340 Merge "Split buckets on boot complete" into rvc-dev 2020-04-22 01:16:28 +00:00
Jeffrey Huang
64c55e7e8a Merge "Remove IStatsd reference from StatsLog.java" into rvc-dev 2020-04-22 00:08:44 +00:00
Tej Singh
e678cb753a Split buckets on boot complete
Also clean up a bit of code on splitting on app upgrades

Piggy-backed off the app upgrade tests, adding parameterized tests to
also test boot complete event.

Refactored some value metric test code to increase code reuse and
assertions.

Fixed a broken value metric test that had assertions commented out.

Refactored NamedLatch into MultiConditionTrigger to avoid creating a
thread before necessary.

Test: atest statsd_test
Test: push a simple test config, reboot, wait, get data. Made sure
the bucket was split
Bug: 144099206
Bug: 154511974

Change-Id: I73858b5db08e8cda762bd8091b30da8738d1fd88
2020-04-21 15:31:14 -07:00
Jeffrey Huang
d8a7f947ba Remove IStatsd reference from StatsLog.java
Also remove sendAppBreadcrumb binder api because it's no longer used.

Bug: 154264326
Test: atest com.google.android.statsd.gts.StatsdHostTestCases
Change-Id: Ic51a057bb01a89a24337521a49c54a52e2073cd1
2020-04-21 15:04:28 -07:00
Ruchir Rastogi
3360c54cd6 Merge "Handle errors within LogEvent" into rvc-dev 2020-04-21 17:07:18 +00:00
Ruchir Rastogi
b639065e0a Merge "Move resetState from FieldValue to LogEvent" into rvc-dev 2020-04-21 16:53:13 +00:00
Ruchir Rastogi
48dbf835ce Move resetState from FieldValue to LogEvent
This helps save memory because FieldValues are copied around much more
than LogEvents.

+ removed unused variables from LogEvent

Test: bit statsd_test:*
Bug: 154267590
Change-Id: If444ef79c85a45526fce183b67d9d8a2e74fa10d
2020-04-20 17:43:56 -07:00
TreeHugger Robot
386cdc2c44 Merge "Stop socket listener when terminating by SIGTERM" into rvc-dev 2020-04-20 23:50:50 +00:00
TreeHugger Robot
9e48995431 Merge "Fixed a typo" into rvc-dev 2020-04-20 18:37:32 +00:00
Jiyong Park
28665792a2 Merge "Mark some aidl_interface modules as unstable" into rvc-dev 2020-04-20 03:23:06 +00:00
Ruchir Rastogi
07f7adb47e Handle errors within LogEvent
Errors are recorded within StatsdStats. The associated CL to change
stats_log.proto within google3 is at cr/306795778.

Test: adb shell dumpsys stats --metadata
Test: bit statsd_test:*
Bug: 144373276
Change-Id: Id80cace9350f77d64a2d401f7fac7b12501e82ee
2020-04-19 14:42:02 -07:00
Muhammad Qureshi
e3ef95cf13 Merge "Remove kMaxPushedAtomId from atoms_info." into rvc-dev 2020-04-18 00:13:52 +00:00
TreeHugger Robot
5aef02fd4a Merge "Fix statsd_test mts build target" into rvc-dev 2020-04-18 00:11:18 +00:00
Eric Jeong
0963dbd07d Stop socket listener when terminating by SIGTERM
- During the termination, StatsService is stopped while
StatsSocketListener is still running.
- This could cause invalid thread access.

Bug: 147316537
Test: Run a test app which keeps sending StatsLog and repeatedly execute
adb shell su root kill `pidof statsd`. statsd should not crash.

Change-Id: Iedd2dcb892f704bdba9adc864afa696de64b8e4c
2020-04-17 16:50:05 -07:00
Jeffrey Huang
9c30a95f5a Fix statsd_test mts build target
The test tried to execute statsd_test when the binary was named
statsd_test32. This fixes the naming issue.

Bug: 153035538
Test: make mts and check the binary name
Change-Id: I13485a95801f9ef71211cd1d36e8ea1fd3bbc0b6
2020-04-17 15:35:33 -07:00
Dichen Zhang
d9aaa5bac4 Fixed a typo
"am broadcast -a" rather than "am broadcast am"
Bug: 123230379
Test: commands #4 and #5 in bug report

Change-Id: I5c23cd86b4da1e03b65c9a68a3a9044466012e89
2020-04-17 17:22:28 +00:00
Muhammad Qureshi
cdf291d297 Remove kMaxPushedAtomId from atoms_info.
Hardcode kMaxPushedAtomId in StatsdStats.

Bug: 154164020
Test: bit statsd_test:*
Change-Id: Ibca731d9783686a9e3caf0ad580e1d62038d68e2
2020-04-17 10:10:17 -07:00
Muhammad Qureshi
7fee54bb27 Remove kAtomsWithAttributionChain from atoms_info
Use LogEvent::getAttributionChainIndex() to check presence of
attribution chains

Bug: 150414252
Test: bit statsd_test:*
Change-Id: Ic798d3b182c2b5aaf73444f2796d539cd3f7bf40
2020-04-16 21:51:44 -07:00
Ruchir Rastogi
9d411d7d8e Merge "ShellSubscriber: install SIGPIPE handler" into rvc-dev 2020-04-16 17:34:16 +00:00
TreeHugger Robot
702537c7ab Merge "Add statsd device tests as mts target" into rvc-dev 2020-04-16 16:59:32 +00:00
TreeHugger Robot
cbd9f03e82 Merge "Remove kTruncatingTimestampAtomBlackList" into rvc-dev 2020-04-16 13:07:00 +00:00
Muhammad Qureshi
1f976bf634 Merge "Remove kStateAtomsInfo from atoms_info." into rvc-dev 2020-04-16 13:01:52 +00:00
Jeffrey Huang
c6b2830e81 Add statsd device tests as mts target
Added multilib to cc_test as a workaround to the build system being
unable to separate 32 and 64bit architectures.

Bug: 153035538
Test: m -j
Change-Id: Ib7f3b01816d7cbd69b3dbf4dbf63ccb22b0c5b6a
2020-04-15 16:06:40 -07:00
Muhammad Qureshi
a7de00022b Remove kTruncatingTimestampAtomBlackList
Use shouldTruncateTimestamp flag in LogEvent instead of the blacklist
to determine whether timestamp should be truncated.

Bug: 150414373
Test: bit statsd_test:*
Change-Id: I4e21ae4909484dd898b9fcd3ce7a018f5e9d982f
2020-04-15 11:34:35 -07:00
TreeHugger Robot
58df220212 Merge "Add new atoms to track user journeys, such as user switches." into rvc-dev 2020-04-15 03:31:00 +00:00
TreeHugger Robot
259e20acec Merge "Add NamedLatch to statsd" into rvc-dev 2020-04-15 02:15:26 +00:00
Muhammad Qureshi
bfc4bdb594 Remove kStateAtomsInfo from atoms_info.
Use the annotation information in LogEvent/FieldValue instead.

- Initialize nested to true in FieldValue.mAnnotations

- Add filterPrimaryKey function to HashableDimensionKey for populating a
HashableDimensionKey with all FieldValues that have the primary key
annotation set.

- Create StateTrackers without checking if atom is a state atom. (We
  can't do this check anymore)

- Remove mAtomId, mStateField, mNested, mPrimaryFields, mDefaultState,
    mResetState members from StateTracker. Use the information passed in
    LogEvent in onLogEvent instead.

- Update tests to log annotations when logging events.

- Remote kStateAtomsFieldOptions from atoms_info.

- Make MetricProducer::mStateGroupMap const

- Rename handlePartialReset to clearStateForPrimaryKey

- Store "default" states in mStateMap. An entry in mStateMap means the
state is not kStateUnknown. Before, an entry in mStateMap meant the
state is not the "default" state.

- Consolidate all state change logic in updateStateForPrimaryKey()
    - remote StateTracker::updateState
    - handleReset and clearStateForPrimaryKey call
    updateStateForPrimaryKey for resetting and clearing of states
    respectively.

- Create a helper method for notifying StateTracker listeners

- Make StateManager::registerListener void

Bug: 151110842
Test: bit statsd_test:*
Change-Id: Ifb8371b213a178fcccaa484086fbdd283dbaec49
2020-04-14 17:22:15 -07:00
Tej Singh
769f35fc0f Add NamedLatch to statsd
This is a sychronizing primitive that is similar to a latch, but a
thread must count down with an identifier. It will be used to make sure
the boot complete, uid map, and all pullers signals are received before
triggering the bucket split.

The latch's countDown operation takes in a string identifier, so that if
the same operation happens twice, it is only counted once.

Bug: 144099206
Test: atest statsd_test

Change-Id: I261a3e50eabbc4998ca30ddf2d67a9a1e788911e
2020-04-14 16:50:12 -07:00
Varun Shah
acb5634dc4 Add new atoms to track user journeys, such as user switches.
Define new atoms to better log and understand various user journeys,
such as user switches, user starts, and user creation.

The UserLifecycleJourneyReported atom defines a user's journey and
holds a user's information such as their user id, user type, and flags
associated with the user. This atom includes a session-id which is used
to link to the UserLifecycleEventOccurred atom which keeps track of
lifycycle events that occur throughout a user's journey. As more user
journeys are defined, there could be more Events added in the future.

Note: the JourneyReported atom can currently be logged more than once
per user journey since there is no record of the user's ongoing journey.
This will be updated in a future CL.

Bug: 146505521
Bug: 150788910
Test: statsd_testdrive 264 265
Change-Id: Iff3847be64d718fb2ec17e58c33d47f7fa4b627a
2020-04-14 11:25:07 -07:00
Ruchir Rastogi
04d75c37ba ShellSubscriber: install SIGPIPE handler
Install a handler that ignores SIGPIPE. Before this, SIGPIPE was not
being handled, which caused a SIGKILL to be sent to statsd. Note that
even if SIGPIPE is handled, writes to closed pipes will return EPIPE.

Bug: 153595161
Test: procedure specified in b/153595161
    1. cat statsd config | adb shell cmd stats data-subscribe
    2. Terminate the shell command
    3. cat statsd config | adb shell cmd stats data-subscribe
Change-Id: Ib5e679ccd7cdf7182452466b316488439871db99
2020-04-13 17:46:09 -07:00
Susi Kharraz-Post
3ddadd8f3c Rename sharesheet started atom proto fields.
When copying over the fields into Google3 the linter failed on two field
names that are camel case rather than snake case. Changing here so the
two match. In addition, one of the enum fields didn't have a default
value.

Bug: 147508495
Test: compiles + ChooserActivityTest runs
Change-Id: Iacec9654350763e691a4a10087f693839fd5ea83
2020-04-13 10:05:29 -04:00
Jiyong Park
db589dded4 Mark some aidl_interface modules as unstable
With b/152655547, all aidl_interface modules are considered as stable
unless it is explicitly with "unstable: true". This change marks the
aidl_interface that are not used across updatable module bounraries
as unstable, so that the build system does not run the API
dumping/checking on them.

Bug: 152655547
Test: m
Change-Id: I1257c66de6dd42b2d32d47ed74cb2878f79d14fb
2020-04-13 13:00:47 +09:00
Tej Singh
e715d2f0e7 Merge "Statsd test mapping" into rvc-dev 2020-04-11 03:49:17 +00:00
Jeffrey Huang
b010fc5534 Merge changes I3f912418,Idd2465d5 into rvc-dev
* changes:
  Send boot completed to statsd
  Send puller registration signal to statsd
2020-04-10 18:38:00 +00:00
Ryan Mitchell
6cb79cdfb7 Merge changes from topic "invalidate_idmap" into rvc-dev
* changes:
  Fix InstallOverlayTests fail to install overlay
  Test that upgrading target apk invalidates idmap
  Invalidate idmap when target updates
2020-04-10 15:50:15 +00:00
TreeHugger Robot
926a6d7a33 Merge "Whitelist MediaProvider for statsd metrics" into rvc-dev 2020-04-10 06:19:52 +00:00
TreeHugger Robot
89d5f51c7e Merge "Improve MediaProvider WW atoms" into rvc-dev 2020-04-10 06:19:46 +00:00
Tej Singh
5d823b30fa Statsd test mapping
Makes a test mapping for statsd so that unit tests run on presubmit.

Changes to make the tests pass:
1. Require root. This is needed to write to disk, since the tests don't
run as statsd's uid

2. Remove AndroidTest.xml file in favor of the autogenerated one.

3. Remove a check in StatsService.test for getUidFromArgs. The test
checked a failure case where we passed a number bigger than INT32_MAX.
However, on a 32 bit device, strtol will return INT32_MAX when an
overflow happens, since it returns a 32 bit number on a 32 bit device.

4. Refactor a lot of e2e tests to sort dimensions, ensuring that the
dimensions are always in order, instead of relying on implicit ordering
of hashing, which can change.

5. Change a long to an int64 in TestActivationsPersistAcrossSystemServerRestart

Test: statsd_test
Bug: 129613474
Change-Id: I80dfa3bfd50ebe6d2c8c0c3ba201f3ad06b68910
2020-04-09 22:20:59 -07:00
Jeffrey Huang
d8f533060d Send boot completed to statsd
Bug: 153384066
Test: m -j
Change-Id: I3f91241851a93d0869dad210bc278b3a1ba6b762
2020-04-09 17:29:19 -07:00
TreeHugger Robot
d11054d31f Merge "[stats] add metrics for package installer v2" into rvc-dev 2020-04-09 21:22:09 +00:00
Nikita Ioffe
e9ed63aa92 Merge "Introduce static @hide PowerManager.isRebootingUserspaceSupportedImpl()" into rvc-dev 2020-04-09 16:39:54 +00:00
Songchun Fan
c296cf746a [stats] add metrics for package installer v2
// Successful installation:
$ out/host/linux-x86/bin/statsd_testdrive -s 98141FFBA005QD 263
...
event_metrics {
  data {
    elapsed_timestamp_nanos: 147291968369
    atom {
      package_installer_v2_reported {
        is_incremental: true
        package_name: "com.unity.megacity"
        duration_millis: 1379
        return_code: 1
      }
    }
  }
}
...

// Failed installation:
$ out/host/linux-x86/bin/statsd_testdrive -s 98141FFBA005QD 263
...
event_metrics {
  data {
    elapsed_timestamp_nanos: 94418812060
    atom {
      package_installer_v2_reported {
        is_incremental: true
        package_name: ""
        duration_millis: 37
        return_code: -20
      }
    }
  }
}
...

Test: manual
BUG: 152913040
Change-Id: Ibb02122e7007f8e3afd65cbb98236bc37c0d24db
2020-04-09 09:33:34 -07:00
Nikita Ioffe
57300d9057 Introduce static @hide PowerManager.isRebootingUserspaceSupportedImpl()
This method consolidates logic of testing whenever rebooting userspace
is supported in a place that can be used both from
PowerMager.isRebootingUserspaceSupported() API and from
PowerManagerService.reboot() implementation.

Consequently, this makes `adb shell svc power reboot userspace` also
benefit from that check, and fail userspace reboot on devices that don't
support it.

Also tweaked logic of ignoring RemoteException to take into account
userspace reboot.

Test: adb root
Test: adb shell setprop init.userspace_reboot.is_supported 0
Test: adb shell svc power reboot userspace & verified error message
Test: adb shell setprop init.userspace_reboot.is_supported 1
Test: adb shell svc power reboot userspace & verified no error message
Test: atest PowerManagerTest
Test: atest CtsUserspaceRebootHostSideTestCases
Bug: 152803929
Change-Id: I2d3a8e0ae1320c408a838c5c5fdf4cd977b167b3
2020-04-09 14:07:22 +01:00
Hyunyoung Song
b3a298d4af Merge "Flatten LauncherUIChanged / Migrate Launcher Workspace snapshot data" into rvc-dev 2020-04-09 08:03:36 +00:00
Jeffrey Huang
d7fda53855 Send puller registration signal to statsd
Bug: 153384066
Test: m -j
Change-Id: Idd2465d52e6720d7bce26ecc7998721b9d9790e3
2020-04-08 22:20:41 -07:00
Hyunyoung Song
0ff10a4a4a Flatten LauncherUIChanged / Migrate Launcher Workspace snapshot data
Bug: 137777105
Bug: 144953948

Test: builds

Change-Id: I55b5e952cdd2579cae1ec85b6f90949c8ccd40f8
2020-04-08 22:03:08 -07:00
Ruchir Rastogi
ffa34f05cf Remove kAtomsWithUidField from atoms_info
Instead of using the map from atoms_info, we now receive uid information
in the form of StatsEvent annotations. The isUid annotation is exposed
from both LogEvent and FieldValue.

Test: bit statsd_test:*
Test: atest GtsStatsdHostTestCases
Bug: 150414601
Change-Id: Iebbe4ce5668de1ab91485daa1be9197cde6e8309
2020-04-08 19:09:33 -07:00
Christine Tsai
5205e14500 Merge "statsd unit test and benchmark refactor" into rvc-dev 2020-04-08 23:09:17 +00:00
TreeHugger Robot
7284bf15da Merge "Fix AIBinder_linkToDeath cookies" into rvc-dev 2020-04-08 18:52:00 +00:00
tsaichristine
8dca82ed94 statsd unit test and benchmark refactor
Abstract test utilities into two new functions: writeAttribution and
parseStatsEventToLogEvent

Bug: 149590301
Test: bit statsd_test:*
&& bit statsd_benchmark:*

Change-Id: I5f77646b6d2d828344b8b6de8777a60d98f96d58
2020-04-08 11:48:09 -07:00