Commit Graph

293 Commits

Author SHA1 Message Date
TreeHugger Robot
14b9335393 Merge "consolidate subsystem sleep state atoms" 2018-01-18 17:56:39 +00:00
Yangster-mac
468ff04fd4 Proto change for dimension from condition.
Test: all statsd unit tests and cts tests passed.

Change-Id: I955143aca7c76e2e22a10d7ba42eb797bd32e147
2018-01-17 21:19:45 -08:00
Chenjie Yu
c8b7f2277c consolidate subsystem sleep state atoms
Test: cts test
Change-Id: Ib9ff746eaa17b6c2c8f9fe5860dd0b480ec85186
2018-01-17 11:42:07 -08:00
Yao Chen
29f79b5623 Update the AID -> UID mapping.
Test: statsd_test
Change-Id: I43c004b765b0cc221ec2e02c9979890e5d55d570
2018-01-17 11:00:20 -08:00
Yangster-mac
87718e283a Gauge metric e2e test
Test: new test passed
Change-Id: I6a8bceb43fbb6e7b82f47951e71b5620779f2ceb
2018-01-15 20:41:08 -08:00
yro
079cea9a7e Create /data/misc/stats-data/ and /data/misc/stats-service/ in statsd.rc
rather than during the runtime of statsd

The purpose of this change is to prevent causing selinux violation by
trying to mkdir to /data/misc/ directory when statsd doesn't have
permission to do so.

Bug: 71537285
Test: manually tested to make sure that there's no sepolicy violation

Change-Id: I9c4ccecc416f41923c9b24dd44a388d135fecc07
2018-01-12 13:51:48 -08:00
TreeHugger Robot
964be9ab42 Merge "pull absolute value for cpu time per frequency." 2018-01-12 00:54:34 +00:00
Yangster-mac
afad8c6b43 Log attribution to statsd for pushed atoms.
Will avoid allocating temp objects in a following CL after the new StatsLog API is in.

Test: all statsd unit test passed.
Change-Id: Id5f6a908d9d42732d83d9985b1b34b29230a7f20
2018-01-11 09:33:13 -08:00
Chenjie Yu
1ee9b7413c pull absolute value for cpu time per frequency.
Test: cts test
Change-Id: I519616905ed8ec6afdaa7e1a0743e279009aa0e5
2018-01-11 09:27:28 -08:00
Yangster-mac
d40053eb8b Map isolated uid to host uid when processing log event in statsD.
Test: added test case for isolated uid in Attribution e2e test.
Change-Id: I63d16ebee3e611b1ef0c910e5154cf27766cb330
2018-01-09 21:45:46 -08:00
TreeHugger Robot
d6808dc0c0 Merge "E2e test: attribution matching and slicing by attribution." 2018-01-10 00:22:45 +00:00
Bookatz
1bf94382d0 Anomaly detection is per dimension
Anomalies are now detected and declared per dimension. This means that
declareAnomaly now gets access to the key that is responsible for the
anomaly. Moreover, the refractory period is per dimension, not overall.
So a second anomaly for the same metric but a different dimension can
fire within the first dimension's refractory period. Thus, if app A
misbehaves and app B misbehaves shortly thereafter, they will both be
detected.

Eventually this key will be passed to the subscribers, although this cl
doesn't do anything with it.

Test: adb shell data/nativetest64/statsd_test/statsd_test
Change-Id: Id76856dc44fe9ecf91ac81a423e84f97c81d30ab
2018-01-09 11:29:41 -08:00
TreeHugger Robot
26d5b41fdb Merge "Cap the size of ice box in StatsdStats." 2018-01-09 17:03:17 +00:00
Olivier Gaillard
aed7f12d4a Add atoms for app startup time.
These atoms match exactly what we currently send through Tron.

Test: manual
Change-Id: I36809417c8d5c62d60035be696ddf506243ee2a9
2018-01-09 14:33:55 +00:00
TreeHugger Robot
c131a9297b Merge changes Ibe8c8d3c,I4a475d6f
* changes:
  Thread-safety at log processor level.
  Handle null string in jni and c++ stats-log-api interfaces.
2018-01-09 10:03:01 +00:00
Yao Chen
f6723dfcf0 Cap the size of ice box in StatsdStats.
+ only keep the last 20 old config stats.

Test: statsd_test
Change-Id: I5bff68299bb9e4def7f7b54aa1d347133bc7a768
2018-01-08 15:18:00 -08:00
Yangster-mac
b0d0628a29 Thread-safety at log processor level.
Test: statsd unit test passed.

Change-Id: Ibe8c8d3cc8297875b16ee385c077b71c87353147
2018-01-08 14:59:42 -08:00
Yao Chen
d59a6589fa Some fixes in duration metrics
+ Don't need to keep the condition key if the condition is not sliced.
+ Whiten the hash before returning.

Test: statsd_test
Change-Id: Ie0bcb14b3adc4a15b79ba3be050431a672a68c28
2018-01-08 11:17:11 -08:00
Yangster-mac
b5bc74122f E2e test: attribution matching and slicing by attribution.
Test: statsd unit tests passed.
Change-Id: I05875b80a9a1ab5f87a478d1f63543f9798254a5
2018-01-06 23:23:19 -08:00
TreeHugger Robot
a7afd1311a Merge "Fix a few things with the loadtest app." 2018-01-06 04:37:59 +00:00
Stefan Lafon
c6f2fa2700 Fix a few things with the loadtest app.
Test: Loadtest app is running fine.

Change-Id: I834f6c618f5a6b0b17ecd5592e2ab2209ce7b94a
2018-01-05 17:02:56 -08:00
Chenjie Yu
b038b70956 pullers now cache data to throttle frequent pull requests.
all pullers have a default 1s cool down before next pull.
We can adjust these later.
Also add puller stats in StatsdStats

Test: unit test
Change-Id: I71894a24c41e059d841591312dbb852f54387b7d
2018-01-05 15:49:59 -08:00
TreeHugger Robot
4735b97d84 Merge "Updates atoms.proto for statsd." 2018-01-05 18:44:54 +00:00
Howard Ro
7266e82cf8 Merge "Use the configs from disk after reboot and separate out ConfigManager startup for unit test to avoid reading configs for tests" 2018-01-05 06:44:38 +00:00
Yangster-mac
a070b6a40c Bug fix: gauge metric producer does not need the atom id parameter.
Test: statsd unit test passed
Change-Id: Ifd1e2f7a05a03886eb227ac1ae0e30b884e49f0c
2018-01-05 04:34:49 +00:00
Yangster-mac
d19bcae42d Move Field proto out of stats_log.proto to keep it clean.
Test: statsd unit test passed.

Change-Id: I31ffea40cf1b82c16bf37c706969bfb159639c7b
2018-01-04 16:39:52 -08:00
yro
469cd80840 Use the configs from disk after reboot and separate out ConfigManager
startup for unit test to avoid reading configs for tests

Bug: 70667694
Test: manual testing conducted, statsd, statsd_test
Change-Id: I49cb82d2b1d7c178b7a4c30c089d8d93e7902faa
2018-01-04 15:00:48 -08:00
TreeHugger Robot
efc331b287 Merge "Use TimeUnit enum to specify the bucket size." 2018-01-04 22:53:29 +00:00
Yangster-mac
b814481ad1 Use TimeUnit enum to specify the bucket size.
Test: all statsd unit test passed

Change-Id: I4f6b80ba2f8c984b06e46e6de6df3e546e99a968
2018-01-04 13:01:14 -08:00
Yao Chen
147ce60278 use only string type in the log source whitelist.
+ predefined "AID_X" will be provided as string type to statsd, and we will translate
  to integer uid using the static map.

Test: statsd_test

Change-Id: Ie47d8481e0c456457e6881ebb9cb4ce008e772b8
2018-01-04 09:57:03 -08:00
Yangster-mac
a7fb12d2d2 1/ Use FieldMatcher to specify the value fields in value metric.
2/ rename number_of_buckets as num_buckets
3/ use double for the Alert's threshold

Test: statsd unit tests passed.
Change-Id: Id1f55f14d3712eddee561681e3cd77343f086c7a
2018-01-03 23:49:36 -08:00
David Chen
6e3e6cbe07 Updates atoms.proto for statsd.
Updates numbering for pulled atoms to start at 10,000 so we have more
room to add more pushed atoms. Also fixes definition for dropbox
error added to more efficiently encode -1, and include the package
name when it's available.

Test: Tested the dropbox atom is logged without crashing on marlin.
Change-Id: I096911e3fe134320f2b1c6f545b0f479c7199612
2018-01-03 17:20:14 -08:00
TreeHugger Robot
48989c2744 Merge "1/ Change all "name" to id in statsD. 2/ Handle Subscription for alert. 3/ Support no_report_metric" 2018-01-03 23:36:21 +00:00
Yangster-mac
94e197cceb 1/ Change all "name" to id in statsD.
2/ Handle Subscription for alert.
3/ Support no_report_metric

Bug: 69522276
Test: all statsd unit tests passed.
Change-Id: I851b235f2d149b8602b0cad632d5bf541962f40a
2018-01-03 15:34:00 -08:00
TreeHugger Robot
5459651bf6 Merge "Checks in statsd that fields are within range." 2018-01-03 20:43:48 +00:00
Yangster-mac
2087716f2b 1/ Support nested message and repeated fields in statsd.
2/ Filter gauge fields by FieldMatcher.
3/ Wire up wakelock attribution chain.
4/ e2e test: wakelock duration metric with aggregated predicate dimensions.
5/ e2e test: count metric with multiple metric condition links for 2 predicates and 1 non-sliced predicate.

Test: statsd unit test passed.

Change-Id: I89db31cb068184a54e0a892fad710966d3127bc9
2018-01-01 10:01:36 -08:00
TreeHugger Robot
303f84faa4 Merge "Splits AnomalyTracker into two files" 2017-12-29 05:18:41 +00:00
David Chen
daa9f3a679 Checks in statsd that fields are within range.
We check that both the developer-chosen label and the state fields
are within the expected ranges.

Test: Test that statsd still builds.

Change-Id: I9f76dfbf16edc4540b28a778385f416cb1306d4e
2017-12-28 17:01:49 -08:00
David Chen
7d8aa4ded5 Small fix to addconfig in statsd.
AddConfiguration now returns false if the incoming byte array can't
even be parsed into the expected proto format. Unfortunately, we
can't fully evaluate if this configuration is valid. Still, this
gives the caller a chance to know that the parameters it sent can't
even be parsed.

Test: Tested that statsd still builds.
Change-Id: I7ba7c64adc4bc77549d5e9133e212a4870645786
2017-12-27 13:37:01 -08:00
Bookatz
857aaa5208 Splits AnomalyTracker into two files
Splits out DurationAnomalyTracker-specific functions into their own
subclass.

Test: the unit tests and CTS tests
Change-Id: Id6eb74d232b4a9c3a932d805d1ba3f0ba43a88b1
2017-12-22 15:05:31 -08:00
Yao Chen
d5aa01b371 Remove the hacky HashableDimensionKey.
+ Add a real HashableDimensionKey as a wrapper of the dimension.
So we can get rid of the maps that we kept.

Pay down technical debt and reduce memory usage.

Test: statsd_test & manual
Change-Id: I233280cf1e2ce93da6a8cd4e8514abb066f4016d
2017-12-20 18:45:43 -08:00
Yao Chen
d10f7b1c7b Add log source filtering in statsd to filter out spams.
+ Add log source whitelist in StatsdConfig
+ Some changes in UidMap API. Listener needs to be wp instead of sp.
+ Update dogfood app config to have log source
+ Increase the stats service thread pool size to 10 (9+1).

TODO: add unit tests(b/70805664). This unit test takes some time to write.

Test: statsd_test & manual

Change-Id: I129b1cc13db5114db7417580962bd7cc4438519d
2017-12-20 18:45:43 -08:00
TreeHugger Robot
b80dcf259f Merge "1/ index app name. 2/ fix a bug in app update." 2017-12-20 17:50:24 +00:00
TreeHugger Robot
4e102f07b6 Merge "Statsd anomaly detection atom" 2017-12-19 07:28:27 +00:00
Bookatz
8fcd09a3e9 Statsd anomaly detection atom
Creates an atom to represent when statsd's anomaly detection alert
fires, so that statsd can log when it detects an anomaly (in the output
and for any further processing).

Test: manually confirmed that atom was logged
Change-Id: I3376e4038bdc3402a536aab7cebad24b104a9aee
2017-12-18 15:59:41 -08:00
Chenjie Yu
2b4fc9d657 align metrics start time to the previous 5 min start, instead
of the next 5 min start. Also refactor config check condition
to be more readable.

Test: unit test
Change-Id: Iaa6b4a59d27c52996723862d92080d90c9e794cb
2017-12-18 15:53:18 -08:00
Yangster
9df9a7f78d 1/ index app name.
2/ fix a bug in app update.

Test: statsd unit tests passed.
Change-Id: I0a1c82411d89705ceaa0a2badfdd4bbb9859df3a
2017-12-18 14:48:35 -08:00
TreeHugger Robot
b5c1fe39c0 Merge "rename pulled atoms to shorter names" 2017-12-18 16:41:15 +00:00
TreeHugger Robot
e6abcbed99 Merge "Turn off verbose logging, remove the fake config." 2017-12-17 17:01:36 +00:00
TreeHugger Robot
dc4e426a17 Merge "add feature: GaugeMetricProducer now takes repeated list of fields bug fix: GaugeMetricProducer now works better with pulled events. unit test also includes GaugeMetricProducer_test" 2017-12-17 16:58:46 +00:00