Commit Graph

15 Commits

Author SHA1 Message Date
Yao Chen
9c1debe330 Add annotation to atoms that represent a state change in atoms.proto
+ A state change atom can have one exclusive state field, and any
  number of primary key fields.

  When there is primary key in the atom, it means the state belongs to the primary key.
  For example,
  message UidProcessStateChanged {
    optional int32 uid = 1 [(stateFieldOption).option = PRIMARY];
    optional android.app.ProcessStateEnum state = 2 [(stateFieldOption).option = EXCLUSIVE];
  }

  When there is no primary key fields in the atom, the state is global.
  For example,
  message ScreenStateChanged {
     optional android.view.DisplayStateEnum state = 1 [(stateFieldOption).option = EXCLUSIVE];
  }

+ The annotation is consumed by stats_log_api_gen to generate a static map from the state
  atoms to its primary fields, and exclusive fields

+ stats_log.proto is splitted into 2 proto files, because statsd needs proto lite, and c++
  lite proto library cannot properly ignore the field options which requires full proto.

 This CL doesn't change any logic in the statsd yet. A separate CL will use the field option
 information to correctly track the state.

Test: added unit tests in stats_log_api_gen_test. and statsd_test pases.
Change-Id: I9e8a979fe81ba60efd4d854bb7087ce4b2b147ec
2018-02-21 16:46:56 -08:00
Yao Chen
8a8d16ceea Statsd CPU optimization.
The key change is to revamp how we parse/store/match a log event, especially how we match repeated
field and attribution nodes, and how we construct dimensions and compare them.

+ We use a integer to encode the field of a log element. And also encode the FieldMatcher into an
integer and a bit mask. The log matching becomes 2 integer operations.

+ Dimension is stored as encoded field and value pair. Checking if 2 dimensions are equal is then
  becoming checking if the underlying integers are equal. The integers are stored contiguously
  in memory, so it's much faster than previous tree structure.

Start review from FieldValue.h

Test: statsd_test + new unit tests

Bug: 72659059

Change-Id: Iec8daeacdd3f39ab297c10ab9cd7b710a9c42e86
2018-02-12 10:38:45 -08:00
Yangster-mac
9369446f0b Support dimension in condition in metric producers.
Test: added e2e tests for count/duration metrics sliced by fields in condition and with/without links.

Change-Id: Ie34deba68e6780abdde458be3f0ce5284e76a1a2
2018-01-30 13:33:57 -08:00
Yangster-mac
94e197cceb 1/ Change all "name" to id in statsD.
2/ Handle Subscription for alert.
3/ Support no_report_metric

Bug: 69522276
Test: all statsd unit tests passed.
Change-Id: I851b235f2d149b8602b0cad632d5bf541962f40a
2018-01-03 15:34:00 -08:00
Yangster-mac
2087716f2b 1/ Support nested message and repeated fields in statsd.
2/ Filter gauge fields by FieldMatcher.
3/ Wire up wakelock attribution chain.
4/ e2e test: wakelock duration metric with aggregated predicate dimensions.
5/ e2e test: count metric with multiple metric condition links for 2 predicates and 1 non-sliced predicate.

Test: statsd unit test passed.

Change-Id: I89db31cb068184a54e0a892fad710966d3127bc9
2018-01-01 10:01:36 -08:00
Yao Chen
d5aa01b371 Remove the hacky HashableDimensionKey.
+ Add a real HashableDimensionKey as a wrapper of the dimension.
So we can get rid of the maps that we kept.

Pay down technical debt and reduce memory usage.

Test: statsd_test & manual
Change-Id: I233280cf1e2ce93da6a8cd4e8514abb066f4016d
2017-12-20 18:45:43 -08:00
Stefan Lafon
12d01fa7c1 Rename more things in statsd_config.proto.
SimpleCondition -> SimplePredicate
simple_condition -> simple_predicate
Condition -> Predicate
condition -> predicate

Test: Ran statsd and statsd_test.

Change-Id: I3115fb51a4dc3cbe5f23535dfc448d5755b9ed7f
2017-12-05 09:20:28 -08:00
TreeHugger Robot
97d43d525c Merge "Fix a bug in test that cause the flakyness in SimpleConditionTracker_test." 2017-11-27 22:13:20 +00:00
Yao Chen
9975205ded Fix a bug in test that cause the flakyness in SimpleConditionTracker_test.
Test: statsd_test
Change-Id: I9f785d26c857023fbe44d71eb3490a50fcc6139a
2017-11-27 11:40:45 -08:00
Yao Chen
b356151e63 Add StatsdStats and guardrail.
+ StatsdStats is the global class that tracks the stats about statsd.

+ Added guardrail for classes that have a map which could potentially grow
  unboundedly with the number of logs.

TODO: add unit tests & CTS for StatsdStats, and guardrail
      add stats for pulled atoms.

Test: statsd_test

Change-Id: I0ea562de4dd3f6162f7923a9c193420b482c1d51
2017-11-27 10:52:54 -08:00
TreeHugger Robot
a82d460baf Merge "Sync proto from google3" 2017-11-21 19:16:21 +00:00
Yao Chen
80235403d2 statsd: parse the new format of stats log
+ Changed how we construct LogEvent, now it's based on the context from log_msg
  without making a copy of the list.

+ All stats logs now have the same event tag, the atom id is the first elem in the log.

Test: statsd_test
Change-Id: I4419380f2ee1c2b2155d427b9f2adb78883b337f
2017-11-17 11:53:15 -08:00
Yao Chen
a81a0ffa52 Sync proto from google3
Default initial value for Condition changed to false

Test: statsd_test

Change-Id: I73ad80d873f2ff1264d18b1eec6d882ad0e9bb6f
2017-11-16 10:15:46 -08:00
Yao Chen
d41c4220d0 Fix some bugs found in statsd
+ in log matcher, condition tracker and duration metric

Test: added unit test
Change-Id: Id633e856ba5453842487321d7ddc0c64100e4bb8
2017-11-16 09:25:53 -08:00
Yao Chen
967b2051fe Make SimpleCondition satisfactorily complicated.
+ Support nested counting
+ Support StopAll
+ Added default_condition to SimpleCondition config
+ Some refactoring/clean up
+ Added unit tests

Test: Added unit tests, statsd_test
Change-Id: I6564ac2e068ce6810e8090c0818064c625c7847a
2017-11-10 08:49:05 -08:00