Commit Graph

6 Commits

Author SHA1 Message Date
Yangster-mac
9def8e3995 Reduce statsd log data size.
1. Hash the strings in metric dimensions.
2. Optimize the timestamp encoding in bucket.
   Use bucket num for full bucket and millis for
   partial bucket.
3. Encode the dimension path per metric and avoid
   deduping it across dimensons.

Test: statsd test
Change-Id: I18f69654de85edb21a9c835c73edead756295e05
BUG: b/77813755
2018-04-26 04:30:18 -07:00
Yangster-mac
e06cfd777a Support slicing by chain.
BUG: b/73975181

Test: statsd test
Change-Id: I913ae0f68ff21ed0703bb5da9c60d3eaa3bf5981
2018-03-14 15:36:24 -07:00
Yangster
13fb7e4eea Statsd cpu optimizaton
- cache the changed dimensions in condition tracker.
- avoid query condition wizard when unnecessary.
- avoid copy dimension keys in condition key generation.

Test: statsd tests.

BUG: b/73959649
Change-Id: I17d68e2a82643de3f421309841e75f84c6fd8f43
2018-03-07 17:36:07 -08:00
Yangster-mac
f520492279 Avoid creating temporary objects in FiterValue().
It reduces the cpu time from 1000ns to 750ns

Test: statsd test.
Change-Id: Ifa7e98e3368f8d55f85c7b09d05a6c416482981d
2018-02-23 20:53:23 -08:00
Yao Chen
4c959cb99e Remove unused variables in statsd, and make more warnings show.
Test: statsd_test

Change-Id: I2c7b674cb615f22c5de90c2de5f2d58108ab2e7f
2018-02-13 15:31:22 -08:00
Yao Chen
8a8d16ceea Statsd CPU optimization.
The key change is to revamp how we parse/store/match a log event, especially how we match repeated
field and attribution nodes, and how we construct dimensions and compare them.

+ We use a integer to encode the field of a log element. And also encode the FieldMatcher into an
integer and a bit mask. The log matching becomes 2 integer operations.

+ Dimension is stored as encoded field and value pair. Checking if 2 dimensions are equal is then
  becoming checking if the underlying integers are equal. The integers are stored contiguously
  in memory, so it's much faster than previous tree structure.

Start review from FieldValue.h

Test: statsd_test + new unit tests

Bug: 72659059

Change-Id: Iec8daeacdd3f39ab297c10ab9cd7b710a9c42e86
2018-02-12 10:38:45 -08:00