Commit Graph

20 Commits

Author SHA1 Message Date
Ruchir Rastogi
2250fa1593 Fix LogEventQueue_tests
These tests were broken by ag/10444161, which deleted a test-only
LogEvent constructor (the tests still built though because there was
another constructor that accepted the same parameter types but with much
different semantics).

Also fixes a bug in TestSlicedCondition that was introduced by one of
the later patchsets in ag/10444161.

Test: bit statsd_test:*
Bug: 149590301
Change-Id: If7de80922680c63505a4b30a1dcefdc47be13219
2020-02-27 17:00:52 -08:00
Ruchir Rastogi
dfd63d4522 Use new socket schema within TestSlicedCondition
This CL serves as a prototype for future efforts to move statsd tests to
the new socket schema.

We also significantly refactor the LogEvent class by introducing two new
functions:
    - LogEvent::LogEvent(int32_t uid, int32_t pid)
    - bool LogEvent::parseBuffer(uint8_t* buf, size_t size)

The goal is that these two functions will be the only two entry points
into the class. Separating these two is helpful because (a) it allows
test and source code to use the same codepath and (b) it allows the
calling function to see if the atom encoding was successfully parsed
(note that constructors can't return values).

P.S. This CL breaks LogEventQueueTests, but that will be resolved once
all tests are moved to the new constructors.

Test: m statsd
Test: bit statsd_test:SimpleConditionTrackerTest#TestSlicedCondition
Bug: 149590301
Change-Id: Id3c9522c4467c4869ec97226734a556dd9dfb169
Merged-In: Id3c9522c4467c4869ec97226734a556dd9dfb169
(cherry picked from commit 14f56f6bc4)
2020-02-27 17:33:53 +00:00
Jeffrey Huang
1e4368aa43 Comment out Statsd tests
Added todos to make them ues the new schema.

Bug: 145923087
Test: m -j && bit statsd_test:*
Change-Id: I0749760eb3123407b78b9ace9a93967bac727bf5
2020-02-18 18:36:02 -08:00
tsaichristine
7685337fe7 Remove dimensions_in_condition from statsd metrics
The dimensions_in_condition field is not used for any configs in statsd.
The functionality that it was intended to provide will be replaced by
StateTracker which is being tracked in b/136566566.

go/remove-dic contains background information on what was
removed/changed and why

Test: bit statsd_test:*
&& atest CtsStatsdHostTestCases
&& atest GtsStatsdHostTestCases

Change-Id: Ic328fa5fe027377380ba57363d9bc77985f18376
2019-08-22 10:03:33 -07:00
Yangster
13fb7e4eea Statsd cpu optimizaton
- cache the changed dimensions in condition tracker.
- avoid query condition wizard when unnecessary.
- avoid copy dimension keys in condition key generation.

Test: statsd tests.

BUG: b/73959649
Change-Id: I17d68e2a82643de3f421309841e75f84c6fd8f43
2018-03-07 17:36:07 -08:00
Yao Chen
9c1debe330 Add annotation to atoms that represent a state change in atoms.proto
+ A state change atom can have one exclusive state field, and any
  number of primary key fields.

  When there is primary key in the atom, it means the state belongs to the primary key.
  For example,
  message UidProcessStateChanged {
    optional int32 uid = 1 [(stateFieldOption).option = PRIMARY];
    optional android.app.ProcessStateEnum state = 2 [(stateFieldOption).option = EXCLUSIVE];
  }

  When there is no primary key fields in the atom, the state is global.
  For example,
  message ScreenStateChanged {
     optional android.view.DisplayStateEnum state = 1 [(stateFieldOption).option = EXCLUSIVE];
  }

+ The annotation is consumed by stats_log_api_gen to generate a static map from the state
  atoms to its primary fields, and exclusive fields

+ stats_log.proto is splitted into 2 proto files, because statsd needs proto lite, and c++
  lite proto library cannot properly ignore the field options which requires full proto.

 This CL doesn't change any logic in the statsd yet. A separate CL will use the field option
 information to correctly track the state.

Test: added unit tests in stats_log_api_gen_test. and statsd_test pases.
Change-Id: I9e8a979fe81ba60efd4d854bb7087ce4b2b147ec
2018-02-21 16:46:56 -08:00
Yao Chen
8a8d16ceea Statsd CPU optimization.
The key change is to revamp how we parse/store/match a log event, especially how we match repeated
field and attribution nodes, and how we construct dimensions and compare them.

+ We use a integer to encode the field of a log element. And also encode the FieldMatcher into an
integer and a bit mask. The log matching becomes 2 integer operations.

+ Dimension is stored as encoded field and value pair. Checking if 2 dimensions are equal is then
  becoming checking if the underlying integers are equal. The integers are stored contiguously
  in memory, so it's much faster than previous tree structure.

Start review from FieldValue.h

Test: statsd_test + new unit tests

Bug: 72659059

Change-Id: Iec8daeacdd3f39ab297c10ab9cd7b710a9c42e86
2018-02-12 10:38:45 -08:00
Yangster-mac
9369446f0b Support dimension in condition in metric producers.
Test: added e2e tests for count/duration metrics sliced by fields in condition and with/without links.

Change-Id: Ie34deba68e6780abdde458be3f0ce5284e76a1a2
2018-01-30 13:33:57 -08:00
Yangster-mac
94e197cceb 1/ Change all "name" to id in statsD.
2/ Handle Subscription for alert.
3/ Support no_report_metric

Bug: 69522276
Test: all statsd unit tests passed.
Change-Id: I851b235f2d149b8602b0cad632d5bf541962f40a
2018-01-03 15:34:00 -08:00
Yangster-mac
2087716f2b 1/ Support nested message and repeated fields in statsd.
2/ Filter gauge fields by FieldMatcher.
3/ Wire up wakelock attribution chain.
4/ e2e test: wakelock duration metric with aggregated predicate dimensions.
5/ e2e test: count metric with multiple metric condition links for 2 predicates and 1 non-sliced predicate.

Test: statsd unit test passed.

Change-Id: I89db31cb068184a54e0a892fad710966d3127bc9
2018-01-01 10:01:36 -08:00
Yao Chen
d5aa01b371 Remove the hacky HashableDimensionKey.
+ Add a real HashableDimensionKey as a wrapper of the dimension.
So we can get rid of the maps that we kept.

Pay down technical debt and reduce memory usage.

Test: statsd_test & manual
Change-Id: I233280cf1e2ce93da6a8cd4e8514abb066f4016d
2017-12-20 18:45:43 -08:00
Stefan Lafon
12d01fa7c1 Rename more things in statsd_config.proto.
SimpleCondition -> SimplePredicate
simple_condition -> simple_predicate
Condition -> Predicate
condition -> predicate

Test: Ran statsd and statsd_test.

Change-Id: I3115fb51a4dc3cbe5f23535dfc448d5755b9ed7f
2017-12-05 09:20:28 -08:00
TreeHugger Robot
97d43d525c Merge "Fix a bug in test that cause the flakyness in SimpleConditionTracker_test." 2017-11-27 22:13:20 +00:00
Yao Chen
9975205ded Fix a bug in test that cause the flakyness in SimpleConditionTracker_test.
Test: statsd_test
Change-Id: I9f785d26c857023fbe44d71eb3490a50fcc6139a
2017-11-27 11:40:45 -08:00
Yao Chen
b356151e63 Add StatsdStats and guardrail.
+ StatsdStats is the global class that tracks the stats about statsd.

+ Added guardrail for classes that have a map which could potentially grow
  unboundedly with the number of logs.

TODO: add unit tests & CTS for StatsdStats, and guardrail
      add stats for pulled atoms.

Test: statsd_test

Change-Id: I0ea562de4dd3f6162f7923a9c193420b482c1d51
2017-11-27 10:52:54 -08:00
TreeHugger Robot
a82d460baf Merge "Sync proto from google3" 2017-11-21 19:16:21 +00:00
Yao Chen
80235403d2 statsd: parse the new format of stats log
+ Changed how we construct LogEvent, now it's based on the context from log_msg
  without making a copy of the list.

+ All stats logs now have the same event tag, the atom id is the first elem in the log.

Test: statsd_test
Change-Id: I4419380f2ee1c2b2155d427b9f2adb78883b337f
2017-11-17 11:53:15 -08:00
Yao Chen
a81a0ffa52 Sync proto from google3
Default initial value for Condition changed to false

Test: statsd_test

Change-Id: I73ad80d873f2ff1264d18b1eec6d882ad0e9bb6f
2017-11-16 10:15:46 -08:00
Yao Chen
d41c4220d0 Fix some bugs found in statsd
+ in log matcher, condition tracker and duration metric

Test: added unit test
Change-Id: Id633e856ba5453842487321d7ddc0c64100e4bb8
2017-11-16 09:25:53 -08:00
Yao Chen
967b2051fe Make SimpleCondition satisfactorily complicated.
+ Support nested counting
+ Support StopAll
+ Added default_condition to SimpleCondition config
+ Some refactoring/clean up
+ Added unit tests

Test: Added unit tests, statsd_test
Change-Id: I6564ac2e068ce6810e8090c0818064c625c7847a
2017-11-10 08:49:05 -08:00