+ predefined "AID_X" will be provided as string type to statsd, and we will translate
to integer uid using the static map.
Test: statsd_test
Change-Id: Ie47d8481e0c456457e6881ebb9cb4ce008e772b8
2/ Handle Subscription for alert.
3/ Support no_report_metric
Bug: 69522276
Test: all statsd unit tests passed.
Change-Id: I851b235f2d149b8602b0cad632d5bf541962f40a
Splits out DurationAnomalyTracker-specific functions into their own
subclass.
Test: the unit tests and CTS tests
Change-Id: Id6eb74d232b4a9c3a932d805d1ba3f0ba43a88b1
+ Add log source whitelist in StatsdConfig
+ Some changes in UidMap API. Listener needs to be wp instead of sp.
+ Update dogfood app config to have log source
+ Increase the stats service thread pool size to 10 (9+1).
TODO: add unit tests(b/70805664). This unit test takes some time to write.
Test: statsd_test & manual
Change-Id: I129b1cc13db5114db7417580962bd7cc4438519d
We use one alarm clock for all pulled atoms.
If metrics from different configs are not aligned,
the clock will be set to repeat at higher and higher
frequency, and consume a lot of battery.
Current implementation assumes a 5min minimum bucket
size. New metric start time is set to be aligned to
the start time of statsd in the next 5min.
So it will ignore events up to 5min.
align puller alarm to minute bundary
Test: unit test
Change-Id: I77ffa3c13de363c780b1000181b9a9b780dd0846
The exception is EventMetricProducer. Each EventMetricProducer will still have a ProtoOutputStream
Because LogEvent comes as a fixed 4K, it's more memory efficient to have an 8k ProtoOutputStream for
storing the events.
Also removed finish() api in MetricProducer, which was intended to use with Dropbox.
Test: statsd_test & dogfood app
Bug: 70393808
Change-Id: I2efe4ecc76a88060a9aa5eb49d1fa6ea60bc5da8
Since there is a separate guardrail for memory used by uid map, we
no longer add the memory from uid map with the memory per each
config's metrics. We also prevent the byte size check from happening
too frequently. In order to mock the MetricsManager, we refactor
some of the existing methods.
Test: Added unit-tests and verified they all pass on marlin.
Change-Id: I15cf105f7d95f4016fdb0443b0a33eebe862cafb
Previously, we always sent a broadcast, even after we have exceeded
the memory limit for this config key. We switch the order so that
we drop the data if the limit is exceeded. If greater than 90% of the
way to the limit, we send the broadcast.
We need to find a way to unit-test this behavior.
Test: N/A.
Change-Id: I6ea40b9e34dceb19805d9af24495d72878f787e0
Checks if current memory usage of uid map is above a configured limit
and if so, we start deleting snapshots. If there are no more
snapshots, we begin deleting two of the deltas. Also records stats
in the guardrail StatsdStats. Also fixes an edge case where a config
is added after the snapshots are added. We request a snapshot of all
installed uid's at that moment. Finally, adds the uid map memory size
when determining if we should send a broadcast to trigger collection.
Test: Added unit-tests and check they pass on marlin.
Change-Id: Id5d86378bd1efe12a06b409164c777c0c6f4e3ab
Fixes a few items in AnomalyTracker, especially to do with what happens
when an anomaly alarm fires.
Test: unit tests still pass
Change-Id: Ia89bd617442e952e587336b890c3ca67430b5e21
+ StatsdStats is the global class that tracks the stats about statsd.
+ Added guardrail for classes that have a map which could potentially grow
unboundedly with the number of logs.
TODO: add unit tests & CTS for StatsdStats, and guardrail
add stats for pulled atoms.
Test: statsd_test
Change-Id: I0ea562de4dd3f6162f7923a9c193420b482c1d51
1. Create intent receiver in StatsCompanionService to listen to shutdown
events.
2. Create StatsWriter class to handle disk writes and deleting files.
3. Update StatsLogProcessor, ConfigManager, and StatsService to handle
files on disk using StatsWriter.
4. Add a wrapper for ConfigMetricsReport.
Still TODO is to be able to add a guardrail to prevent accumulating
excessive amount files on disk, which will be followed up by another
change.
Test: statsd, statsd_test
Change-Id: Ia0b3af315af545daa8b0078b3700c600aa7c285f
2/ Init anomaly from config based on the public language.
3/ Unit tests for anomaly detection in count/gauge producer.
4/ Revisit the duration tracker logic.
Test: unit test passed.
Change-Id: I2423c0e0f05b1e37626954de9e749303423963f2
StatsD will send a broadcast when we're 90% of the way to our
allocated memory limit for the configuration. If the memory usage
goes over the limit, we just lose all the data for this config.
Also modifies the adb shell commands to facilitate debugging of the
broadcasts.
Test: Manually tested on marlin-eng with custom gmscore code.
Change-Id: I517a15bd4c959aa221802f84a51f13141a725102
Also make atom WakelockStateChanged use the "Type" enum instead of int32.
Test: Unittests are passing. Statsd is working.
Change-Id: I0909e2d97297f78996a81366d66aae62d5bf5ce1
2. Resolve segfault when serializing proto into vector of uint8_t
3. Add a local byte size variable for EventMetric data as we cannot call
size() of ProtoOutputStream before we finish writing.
4. Replace hardcoded conversion of second to nano second with NS_PER_SEC
Test: statsd, statsd_test
Change-Id: I47a2ce2b05e6191c18596489682118edcb41e945
We push events from BatteryStatsImpl if an isolated uid is added or
removed and we have a custom rule in statsd to use these events to
update our uid map. In the future, we need to use this map to
convert all incoming uid's to their host uid.
Test: Added unit-test to UidMap_test.
Change-Id: I33c0451eb2c886161f22dd12e479d216fad0940d
outputs of each MetricProducer's and merge the results into
ConfigMetricsReport which holds ConfigKey, repeated field of
StatsLogReport, and UidMap. The data will be represented as
vector<uint8_t> which can be passed down to binder call for clients to
pick up. Also, all unnecessary dependencies to stats_log proto have been
removed.
Test: statsd, statsd_test
Change-Id: Ia69137cbc8613644a892e6be1e87b4858bd39fe3
And other miscellaneous fixes.
+ clang-format
+ 2 bug fixes, one in dump-report command, one in ResourcePowerManagerPuller
Test: statsd_test
Change-Id: Ibd164d948ad62adcc529d813df1210781e38be47
We send a snapshot of all installed apps with their uids every time
a user is added or removed and when statsd is started.
We keep track of the latest timestamp when a config key has retrieved
the UID map data. This allows us to remove older data when we're
guaranteed that all config sources have retrieved the old data.
Test: Added more unit tests to UidMap_test and passed on marlin-eng.
Change-Id: I34a3d61e75eedec44b98d896d7f6db0bc383f46a
size of each MetricsProducer's. The implementation of byteSize() method
is still TBD as it depends on migration to ProtoOutputStream.
Test: statsd, statsd_test
Change-Id: I966606044d7cb814dabe94192bacecad91f28177
statsd entries to clients. This change only includes changes on statds
side and does not include java library for clients to import. Java
library will be a separate change as it requires system api review.
Test: statsd, statsd_test
Change-Id: I306c6e9687801668cc0145b12d38406bfe634775
Now we support following metrics:
<Duration> of [app holding a wake lock], while [*this app*] is [in background] [AND] [screen is off]
[Slice] the output by [app name, wake lock name], with bucket size [30sec]
+ Also added onDumpReport() api in MetricsManager, it can be called from client to fetch the data from
statsd
+ Also added command line tool to dump the StatsLogReport from all metrics for debugging.
+ Synced proto from google3. with a pending cl (cr/172359050)
TODO: We need to add tons of tests to test the Metrics. I will work on it after this CL so people
can be unblocked.
I locally test the duration metric with wake lock with an app that generates StatsLog events.
Test: statsd_test
and manual test, and run:
adb shell cmd stats dump-report
We have a default config, which contains a metrics to count PROCESS_START event sliced by
package name.
Change-Id: I4838cc6cf025c143b7e84f43040703a78121fd25
It stores all of the parsed fields in a single vector, and
provides accessor methods to get at fields by index with
their correct type.
Test: statsd_test
Change-Id: I4fa94e4ce52db3ac87f19b62f9c85398de6e8145
- Add a ConfigManager class that tracks the configurations
that have been passed to us. Configurations are now
tracked by tuples of (uid,tag), where the tag is an
app-defined string, in case a single uid has multiple
configurations.
- Move all of the initialization into StatsService.
- Get rid of the ability to have multiple LogListeners. Raw
events are now pushed directly into StatsService, which
can distribute them to the interested parties (and will
eventually be able to do the proper locking).
- Add Log.h, which sets our LOG_TAG correctly.
- Move some of the related files that I expect will grow some
into their own subdirectories.
Test: statsd_test
Test: adb shell cmd stats config ...
Test: adb shell dumpsys stats
Change-Id: I79487603003d8a842d5bd319741f1ecbf72063d1
The UID map is updated by StatsCompanionService, which listens to broadcast
updates indicating that an app was updated/installed or removed. Also,
the entire map is updated when statsd first connects to the companion
service. Also, there is a way for metrics producers to subscribe to
updates, so that they can know when an app was upgraded.
Test: Created new unit-test for mapping and manually tested for install
and remove. Did not manually test the app upgrade.
Change-Id: I6676ae5c93b75c72d9badabb36aa9c40006db07d
+ also synced proto from google3 to fix the LogEntryMatcher proto
+ MetricsManager represents StatsdConfig, it's responsible for initializing and managing all
LogEntryMatcher, Condition, and Metrics. Start review from here.
+ Added more complete StatsdConfig initialization, including building the map for:
LogEntryMatcher -> Metrics
LogEntryMatcher -> Condition
Condition -> Metrics.
All the maps use index(int). The extra amount of memory for storing mappings help us
quickly process log events.
The StatsdConfig initialization process detects malformed config
- Circle dependency
- Missing definition
etc.
And once we detect ANY error, statsd will reject the config. And the resources related to this
config will be released.
Test: Added unit tests
Change-Id: I2c4aefdbf3e2aa1701eacbb2fb5e653819ec1fbb
This cl is to let statsd understand statsd_config, and compute metrics
defined in the config.
+ StatsLogProcessor is given a StatsdConfig (hard coded right now).
We construct a MetricProducer for each of the metric, and the metrics
share Condition and LogEntryMatchers
+ Added the CountMetricProducer type for CountMetric.
We can now count times of SCREEN_ON events given a config.
TODO: 1) conditions are not implemented.
2) slicings are not implemented in CountMetric
3) move the interaction to dropbox to a separate thread
4) decide how the in memory metrics would be used by anomaly detection
Test: manual test.
$ adb shell /system/bin/statsd
$ cat config_file.dat | adb shell cmd stats config
Change-Id: I38f4059c0dc5a827c338131d4a6fa7d4cbe865db
Added .clang-format, because there isn't an official .clang-format file for Android framework code.
before we upload changes, do:
clang-format -style=file -i [file list]
to format the files that you touched.
Test: formatting only. NO code changes.
Change-Id: I90e87f1ee6618da8ea9bc2221c609c415a4046a8
Statsd code now lives in android::os::statsd namespace. Existing files
are largely modified to follow this convention.
Exception: parse_util, since it seems a bit different.
Test: code compiles and existing statsd_tests still pass
Change-Id: Idf92a071b9ed172d01eb3087a4fa3609d67a038c
must be in serialized binary format. Also fixes small issue
in build rule related to proto lib.
Test: Manually tested that ADB command works without crashing.
Change-Id: Iba2e677561ff500adb601a598f73e8a7b32540e5
+ The DropboxWriter keeps data in cache, and flush to files once the
size exceeds the maximum value.
+ Different components should create their owner DropboxWriter with
different tags, e.g., anomly detection, experiment metrics, etc.
+ Copied stats_log related protos from g3
Test: run statsd, and adb shell dumpsys dropbox
Will add unit tests.
Change-Id: If06e9a9953be32082252b340a97124d732656b40
