Statsd dumpsys wasn't handling multiple configs properly. Also, doing a
dump would delete the data that had been saved to disk. This cl fixes
both those issues.
Test: run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.atom.HostAtomTests#testDumpsysStats
Change-Id: I5f2f86c70fbcbc1407aae1b1ffeb7fbdb9f67d0e
Previously, most StatsService calls only allow the caller to specify the
uid if it is Userdebug/EngBuild. This applied even if the caller was
just specifying its own UID; this cl allows such a case. It also allows
ROOT to impersonate SHELL.
Test: Manual testing on userdebug and userbuild devices.
Test: make statsd_test && adb sync data && adb shell data/nativetest64/statsd_test/statsd_test
Change-Id: I2685d13a8dc24d40b5dab8be9842f53be6888ba3
Adds a tool for local usage of statsd. The tool can:
-upload a config from a file
-get the report data from statsd
Both the config and the report can be either in binary or human-readable
format, as specified.
Usage:
make statsd_localdrive
./out/host/linux-x86/bin/statsd_localdrive
Also, adds the ability to specify whether dump-report should also erase
the data when it returns it. A test for this is added.
Test: make -j8 statsd_test && adb sync data && adb shell data/nativetest64/statsd_test/statsd_test
Test: make statsd_localdrive && ./out/host/linux-x86/bin/statsd_localdrive <commands>
Bug: 77909781
Change-Id: I9a38964988e90c4158a555f41879534267aadd32
* Creates an incident section for statsd data.
* Allows dump to output statsd data, in proto format.
* Hooks up two statsd outputs to bugreports:
-statsd report data in proto format
-statsd metadata (statsdstats) in text format
The incident section does not import stats_log.proto because that turns
out to be extremely difficult: stats_log.proto imports atoms.proto,
which imports more things and is enormous and causes all sorts of
problems. atoms.proto was purposefully never compiled in AOSP, so to
retain that feature, the incident section uses 'bytes' instead of an
actual message. Since this isn't ever read in AOSP (other than testing),
this should be fine.
Bug: 115678461
Test: take a bug report and confirm valid proto
Test: cts-tradefed run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.atom.HostAtomTests#testDumpsysStats
Change-Id: I1c370af7678d1dc7440ce299ea5ea4da6d33832b
Each config can choose to include version strings and installer with
each metrics report. This data may be useful in the cloud to filter
the app-specific data.
BUG: 115626330
Change-Id: I3972ff2a94e7f0347ac0cc8a443cf328c1731e13
Test: Modified unit-tests, verified on marlin-eng
+ Changed the output format from Atom to ShellData, which is a wrapper for repeated Atom
This is useful because pulled atoms are usually a list of atoms.
Test: statsd_test added
Bug: 110536553
Change-Id: I0e2f55bdd9015c9bc95b87a630297c6f13e39636
This change implements server-side of Stats HAL in statsd to be able to
receive loggings and parse them into LogEvent event before passing them to
StatsLogProcessor.
Bug: 114316519
Test: verified that statshal exists by calling 'adb shell lshal'
Change-Id: I43e7415a2d28e20af872049100cd6f2627cf3c02
+ This feature is for supporting perfd.
+ Perfd is built outside android and doesnot depend on any android libraries.
So the communication between perfd and statsd can only be via non-android IPCs
(such as socket, pipe, file descriptors etc.)
+ Perfd runs as a shell user, so using the existing shell cmd is a natural choice.
+ The input is a simple config, and output is a stream of atoms proto binary data.
+ Also cleaned up the code so that we use file descriptor directly instead of creating another
FILE*.
TODO: pulled atom subscription.
Bug: 110536553
Test: statsd_test and manually tested
Change-Id: I64b0061cc66b5f7648147885a2ac1af531c19917
+ Remove dead code
+ Add a simple log loss detection as a starter to see if there is any log loss
detected at all.
TODO: If we do see log loss, we can add more sophisticated logging and reset mechanism.
Bug: 80538532
Test: statsd_test
Change-Id: Iff150c9d8f9f936dbd4586161a3484bef7035c28
atom and logging for statsd to mimic procstats process duration metrics.
ActivityManagerSleepStateChanged
MemoryFactorStateChanged
ExcessiveCpuUsageReported
CachedKillReported
ProcessStateChanged
ProcessMemoryStatReported
Some of the enums will need to use frameworks proto. It is blocked by
other cls that are cleaning namespace, etc.
Bug: 110784286
Bug: 110538804
Bug: 110539297
Bug: 111561536
Bug: 111685402
Bug: 111128767
Test: cts test
Change-Id: I7502eb0bb36b90f43493b147a35bee48bad1b000
+ Created bugs for those TODOs that are still relevant.
+ Remove obsolete TODOs.
Test: no code change.
Change-Id: I41c2a89a882f087817ee6cbc3f095e1d80e1928e
This is to be consistent with other patterns such as UidMap.
This also makes unit test simpler.
Change-Id: I1558cd609e470481f269ecf2ae616277a95cfbf0
Bug: 72722120
Test: unit test
Reports written to disk don't contain the strings used, which will
make this report unusable if there are strings that don't show up
again. We should always include the strings, so this option is
removed entirely.
Also, we hard-coded the wrong number of fields when pulling
ModemActivityInfo. There are actually 10 fields, not 6.
Bug: 79601503
Test: Tested unit-tests pass on marlin-eng.
Change-Id: I6834b096ced77418a9cc2ddd79b08d1c9c447fae
We observe a single ConfigMetricsReportList can be greater than the
safe size for the binder transaction buffer since we only check the
size of the current metrics in progress, but we also return the
previous reports stored on disk.
This change will attempt to send another ConfigMetricsReportList
as soon as possible if there's already a report on disk.
Also fixes a bug when trying to trigger data fetch before the client
has registered the corresponding dataFetchOperation.
Bug: 79201869
Test: Tested manually on marlin-eng
Change-Id: I2d3677162804a27e7a7a95d482d80c46bd994a67
It only works on eng build. And all code is behind a build flag, so the
code will be stripped out in production builds.
Bug: 78239479
Test: manual
Change-Id: I20ee51822d18e6c77ca324a5327712cbed09593e
1. Hash the strings in metric dimensions.
2. Optimize the timestamp encoding in bucket.
Use bucket num for full bucket and millis for
partial bucket.
3. Encode the dimension path per metric and avoid
deduping it across dimensons.
Test: statsd test
Change-Id: I18f69654de85edb21a9c835c73edead756295e05
BUG: b/77813755
Enforces the dump permission in stats service and fixes a typo in an
enum name.
Bug: b/77723108
Test: adb shell dumpsys stats as root and not root
Test: build success on marlin (and cts build success)
Change-Id: I131f97fe4fd370d50bc4ad369377f6adcca5f6ae
APIs that return package usage data (such as the new StatsManager)
must ensure that callers hold both the PACKAGE_USAGE_STATS permission
and the OP_GET_USAGE_STATS app-op.
Add noteOp() method that can be called from native code.
Also add missing security checks on command interface.
Bug: 77662908, 78121728
Test: builds, boots
Change-Id: Ie0d51e4baaacd9d7d36ba0c587ec91a870b9df17
If the data receiver is experiencing delays, there may be a queue of
multiple intents to collect the same data. This timestamp makes it
easy in the receiver to de-dupe these requests to call getData.
Also, we update how StatsCompanionService gets the snapshot by
requesting data for all known apps. I notice that Keep seems to have
a uid active even when it appears uninstalled.
Bug: 77981668
Test: Flashed marlin-eng and manually verified.
Change-Id: I509e19383ec4a5da8746dd0c76ac71a948c6877d
When statsd reconnects to logd, statsd will read all logs from buffer again. To prevent us from
reprocessing old events, we do the following:
1. At any given moment, record the largest timestamp(T_max) and last timestamp (check point) that
we've seen before.
2. When reconnection happens, we look for the check point until we see a new log with a timestamp
larger than T_max.
-> If we found the CP, resume after the CP. Success
-> If we can't find CP, there is definitely log loss. We reset all configs.
Note:
1. Logd has an API to read logs after a certain timestamp. But this api is vulnerable to
time changes from Settings. So we cannot rely on it.
2. If logd inserts a new log (with older timestamp) before CP, we cannot detect it. It's not
possible to detect it without record all timestamps we have seen.
Test: statsd_test
Bug: 77813113
Change-Id: Ic3fdb47230807606ab11dc994cb162194adb8448
When StatsManager fails to connect to statsd, it now throws an exception
for the caller to catch. It also throws an exception of the config being
added is of an unreadable format.
Due to backwards compatibility issues, the old APIs could not be
changed, so new ones were made to replace the old ones. The old ones are
now temporary and will be removed when the compatibility issue is
resolved.
Bug: 77648233
Test: gts-tradefed run gts-dev --module GtsStatsdHostTestCases
Change-Id: Ibea05883a29b9b3ef9927d2f8fe295eb99832ab7
The previous scheme captured periodic snapshots for each config with
complex logic that's unnecessary and wasted memory. We actually don't
need to store any snapshots since we just convert the current state
into a snapshot and also include the deltas (change events) since the
previous report until now.
To make the system more robust, we also include up to 100 of the
deleted apps in the uid map.
Also, fix the wiring of the partial buckets so the metric producers
form partial buckets on both app upgrade and removal, but not on
installation of a new app.
Also, we update StatsCompanionService to also include disabled apps.
Bug: 77607583
Test: Verified unit-tests pass and added new e2e tests.
Change-Id: I98e1f544d6e6571545ae1581c4cebab807596f51
Test: statsd unit tests
Test: visually verified output of adb shell cmd stats print-stats was
sane
Test: making a CTS test that uses proto output
Bug: b/77548658
Change-Id: Ie60d66b9883f109446745dfcef97b7fb6194a1ac
Roughly check the config every hour to see whether the ttl expired.
If so, read the config from disk and recreate the metric manager.
Test: statsd test
BUG: b/77274363
Change-Id: I16838afe5bbe966c3a0f638869751f9b59a5a259
+ change StatsPullerManager internal time units to be consistent
+ use series of alarms for pullers, instead of use setRepeating
Bug: 76223345
Bug: 75970648
Test: cts test
Change-Id: I9e6ac0ce06541f5ceabd2a8fa444e13d40e36983
Statsd clients may want to set an empty config temporarily, so it's
more convenient to allow them to set an empty config instead of
having to use the removeConfig and then having to remember to call
StatsManager#setDataFetchOperation.
Test: Added unit-tests and check they pass on marlin-eng.
Bug: 74997752
Change-Id: I2e762e5ec01e5a2c9a3469fb330b53fefbd734d6
1. StatsdStats does not use the proto object in memory anymore.
2. lite_static -> lite
3. don't use sstream
Bug: 72129300
Test: statsd_test
Change-Id: I8a5adaf222d4d5034e8bf115215fb6dd5f042cac
