Commit Graph

22 Commits

Author SHA1 Message Date
Yangster-mac
468ff04fd4 Proto change for dimension from condition.
Test: all statsd unit tests and cts tests passed.

Change-Id: I955143aca7c76e2e22a10d7ba42eb797bd32e147
2018-01-17 21:19:45 -08:00
Yangster-mac
b814481ad1 Use TimeUnit enum to specify the bucket size.
Test: all statsd unit test passed

Change-Id: I4f6b80ba2f8c984b06e46e6de6df3e546e99a968
2018-01-04 13:01:14 -08:00
Yangster-mac
a7fb12d2d2 1/ Use FieldMatcher to specify the value fields in value metric.
2/ rename number_of_buckets as num_buckets
3/ use double for the Alert's threshold

Test: statsd unit tests passed.
Change-Id: Id1f55f14d3712eddee561681e3cd77343f086c7a
2018-01-03 23:49:36 -08:00
Yangster-mac
94e197cceb 1/ Change all "name" to id in statsD.
2/ Handle Subscription for alert.
3/ Support no_report_metric

Bug: 69522276
Test: all statsd unit tests passed.
Change-Id: I851b235f2d149b8602b0cad632d5bf541962f40a
2018-01-03 15:34:00 -08:00
Yangster-mac
2087716f2b 1/ Support nested message and repeated fields in statsd.
2/ Filter gauge fields by FieldMatcher.
3/ Wire up wakelock attribution chain.
4/ e2e test: wakelock duration metric with aggregated predicate dimensions.
5/ e2e test: count metric with multiple metric condition links for 2 predicates and 1 non-sliced predicate.

Test: statsd unit test passed.

Change-Id: I89db31cb068184a54e0a892fad710966d3127bc9
2018-01-01 10:01:36 -08:00
Chenjie Yu
85ed838713 align metrics to 5min bundary
We use one alarm clock for all pulled atoms.
If metrics from different configs are not aligned,
the clock will be set to repeat at higher and higher
frequency, and consume a lot of battery.
Current implementation assumes a 5min minimum bucket
size. New metric start time is set to be aligned to
the start time of statsd in the next 5min.
So it will ignore events up to 5min.

align puller alarm to minute bundary

Test: unit test
Change-Id: I77ffa3c13de363c780b1000181b9a9b780dd0846
2017-12-16 17:08:46 -08:00
Stefan Lafon
12d01fa7c1 Rename more things in statsd_config.proto.
SimpleCondition -> SimplePredicate
simple_condition -> simple_predicate
Condition -> Predicate
condition -> predicate

Test: Ran statsd and statsd_test.

Change-Id: I3115fb51a4dc3cbe5f23535dfc448d5755b9ed7f
2017-12-05 09:20:28 -08:00
Stefan Lafon
b8c9aa8c9c Rename a few things in statsd_config.proto.
This for frameworks/base/

SimpleLogEntryMatcher -> SimpleAtomMatcher.
simple_log_entry_matcher -> simple_atom_matcher
LogEntryMatcher -> AtomMatcher
log_entry_matcher -> atom_matcher

Test: Ran statsd_test.

Change-Id: I7398758e3fab86235897c162884f5ceab3002510
2017-12-04 10:59:23 -08:00
Yao Chen
10535b9ad0 Reject the config if condition config has errors. And add log tag.
Test: added unit test.
Change-Id: I7108350978f7c1d0e8e87f22082e19fc76a75631
2017-11-27 14:15:36 -08:00
Yao Chen
c8862caf4f Merge "Revert "Reject the config if condition config has errors. And add log tag."" 2017-11-27 22:02:29 +00:00
Yao Chen
f94a51876d Revert "Reject the config if condition config has errors. And add log tag."
This reverts commit c9fa2f6d4e.

Change-Id: I91de13f8a102c58bbd3dfc37b09c977e78119854
2017-11-27 21:52:34 +00:00
TreeHugger Robot
1b5be51fd9 Merge "Reject the config if condition config has errors. And add log tag." 2017-11-27 21:29:12 +00:00
Yao Chen
c9fa2f6d4e Reject the config if condition config has errors. And add log tag.
Test: added unit test.
Change-Id: I5a9d6de2492b94bc5f1c88524f743607e60226c1
2017-11-27 11:35:59 -08:00
Yao Chen
b356151e63 Add StatsdStats and guardrail.
+ StatsdStats is the global class that tracks the stats about statsd.

+ Added guardrail for classes that have a map which could potentially grow
  unboundedly with the number of logs.

TODO: add unit tests & CTS for StatsdStats, and guardrail
      add stats for pulled atoms.

Test: statsd_test

Change-Id: I0ea562de4dd3f6162f7923a9c193420b482c1d51
2017-11-27 10:52:54 -08:00
Yangster-mac
e2cd6d509b 1/ Duration anomaly tracker with alarm.
2/ Init anomaly from config based on the public language.
3/ Unit tests for anomaly detection in count/gauge producer.
4/ Revisit the duration tracker logic.

Test: unit test passed.
Change-Id: I2423c0e0f05b1e37626954de9e749303423963f2
2017-11-20 15:37:24 -08:00
Yangster-mac
d1815dc7b8 Sync statsd to the public proto changes:
1/ use string id/name for all metrics/configs/matchers.
2/ move alerts out of metric proto.

Test: unit test passed.
Change-Id: I08269f47be30f83f6dbadf873e7d62213449932b
2017-11-13 22:09:08 -08:00
Yangster
1d4d686001 Gauge metric producer.
Test: manual tests passed for pushed device temperature.
Change-Id: I7592a4c04666606b745cdb41db8f9d8a96a966da
2017-11-03 16:42:43 -07:00
Chenjie Yu
b3dda41a5a reretry ValueMetric implementation and pulling mechanism
Note:
This is for value metric. The default operations is sum the diffs.
The test uses kernel wake lock, which also needs dimension by kernel
wake lock name.

The test is a bit cumbersome as it needs StatsCompanionService to do
the alarm, which is not exact alarm.

The internal state of a slice of bucket would look something like this:

4:ipc0000005e_727_android.hardwar
0      0
4:SensorService_wakelock
40      64
4:ipc0000005c_727_android.hardwar
...

Test: manual test on device.
Change-Id: I2ed0ac7d3c5fcba8b7611d46f38a38ffd8bdc92a
2017-11-01 15:58:45 -07:00
Yao Chen
5154a37930 Adding multi layer aggregation in DurationMetric
Newly supported metrics examples:

1) Compute [Total|Max] duration of [an app] holding [ANY] wake lock while [this app] is in
   [background] and [screen off], bucket size 30seconds, and slice output by uid.

2) Compute [Total|Max] duration of [ANY app] holding [ANY] wake lock while [this app] is in
   [background] and [screen off], bucket size 30 seconds.

+ DurationMetric proto has a "what" which is a SimpleCondition. It defines the atom level start
  and stop of the duration timer, and it has its atom dimension. e.g., for wake locks, the atom
  dimensions wil be uid and wl name.

+ Now dimension is explicitly specified in SimpleCondition proto instead of inferred from the "link"

+ Added support for "Or" and "Max" through 2 layers of aggregation.

TODO: (1) The way we track slicedCondition in duration metric is not efficient. optimize!
      (2) The output dimension should all use int32 instead of KeyMatcher. Fix in a future cl.

Test: Added some unit tests using gmock. Will add more unit tests.

Change-Id: I58a827624f01f9a54fcb80709c4de4ff94a8bc67
2017-11-01 10:07:21 -07:00
Yao Chen
729093df0d Add support for dimension, and link with condition and added DurationMetric
Now we support following metrics:
  <Duration> of [app holding a wake lock], while [*this app*] is [in background] [AND] [screen is off]
  [Slice] the output by [app name, wake lock name], with bucket size [30sec]

+ Also added onDumpReport() api in MetricsManager, it can be called from client to fetch the data from
  statsd

+ Also added command line tool to dump the StatsLogReport from all metrics for debugging.

+ Synced proto from google3. with a pending cl (cr/172359050)

TODO: We need to add tons of tests to test the Metrics. I will work on it after this CL so people
can be unblocked.

I locally test the duration metric with wake lock with an app that generates StatsLog events.

Test: statsd_test

      and manual test, and run:

      adb shell cmd stats dump-report
      We have a default config, which contains a metrics to count PROCESS_START event sliced by
      package name.

Change-Id: I4838cc6cf025c143b7e84f43040703a78121fd25
2017-10-19 15:40:55 -07:00
Joe Onorato
9fc9edf95a Clean up how we handle configurations, and other assorted cleanup
- Add a ConfigManager class that tracks the configurations
  that have been passed to us.  Configurations are now
  tracked by tuples of (uid,tag), where the tag is an
  app-defined string, in case a single uid has multiple
  configurations.
- Move all of the initialization into StatsService.
- Get rid of the ability to have multiple LogListeners. Raw
  events are now pushed directly into StatsService, which
  can distribute them to the interested parties (and will
  eventually be able to do the proper locking).
- Add Log.h, which sets our LOG_TAG correctly.
- Move some of the related files that I expect will grow some
  into their own subdirectories.

Test: statsd_test
Test: adb shell cmd stats config ...
Test: adb shell dumpsys stats
Change-Id: I79487603003d8a842d5bd319741f1ecbf72063d1
2017-10-18 04:18:58 -07:00
Yao Chen
caf339d004 More complete implementation for condition and log matchers in statsd.
+ also synced proto from google3 to fix the LogEntryMatcher proto

+ MetricsManager represents StatsdConfig, it's responsible for initializing and managing all
LogEntryMatcher, Condition, and Metrics. Start review from here.

+ Added more complete StatsdConfig initialization, including building the map for:
    LogEntryMatcher -> Metrics
    LogEntryMatcher -> Condition
    Condition       -> Metrics.

    All the maps use index(int). The extra amount of memory for storing mappings help us
    quickly process log events.

  The StatsdConfig initialization process detects malformed config
  - Circle dependency
  - Missing definition
  etc.

 And once we detect ANY error, statsd will reject the config. And the resources related to this
 config will be released.

Test: Added unit tests
Change-Id: I2c4aefdbf3e2aa1701eacbb2fb5e653819ec1fbb
2017-10-10 10:03:46 -07:00