Commit Graph

2575 Commits

Author SHA1 Message Date
Bookatz
1bf94382d0 Anomaly detection is per dimension
Anomalies are now detected and declared per dimension. This means that
declareAnomaly now gets access to the key that is responsible for the
anomaly. Moreover, the refractory period is per dimension, not overall.
So a second anomaly for the same metric but a different dimension can
fire within the first dimension's refractory period. Thus, if app A
misbehaves and app B misbehaves shortly thereafter, they will both be
detected.

Eventually this key will be passed to the subscribers, although this cl
doesn't do anything with it.

Test: adb shell data/nativetest64/statsd_test/statsd_test
Change-Id: Id76856dc44fe9ecf91ac81a423e84f97c81d30ab
2018-01-09 11:29:41 -08:00
TreeHugger Robot
26d5b41fdb Merge "Cap the size of ice box in StatsdStats." 2018-01-09 17:03:17 +00:00
Olivier Gaillard
aed7f12d4a Add atoms for app startup time.
These atoms match exactly what we currently send through Tron.

Test: manual
Change-Id: I36809417c8d5c62d60035be696ddf506243ee2a9
2018-01-09 14:33:55 +00:00
TreeHugger Robot
c131a9297b Merge changes Ibe8c8d3c,I4a475d6f
* changes:
  Thread-safety at log processor level.
  Handle null string in jni and c++ stats-log-api interfaces.
2018-01-09 10:03:01 +00:00
TreeHugger Robot
c99dea0a4b Merge "Revert "Enable incidentd, it will just be idle while no one triggers it."" 2018-01-09 03:28:11 +00:00
Yi Jin
44ed76eb9f Revert "Enable incidentd, it will just be idle while no one triggers it."
This reverts commit d612df381e.

Reason for revert: Selinux denials, see b/71591707 for more details.

Change-Id: I864cd74e7ba57d67e2ed8328a7df9c5ff335508c
2018-01-09 01:44:47 +00:00
TreeHugger Robot
9761f47e84 Merge "Some fixes in duration metrics" 2018-01-08 23:42:26 +00:00
Yao Chen
f6723dfcf0 Cap the size of ice box in StatsdStats.
+ only keep the last 20 old config stats.

Test: statsd_test
Change-Id: I5bff68299bb9e4def7f7b54aa1d347133bc7a768
2018-01-08 15:18:00 -08:00
Yangster-mac
b0d0628a29 Thread-safety at log processor level.
Test: statsd unit test passed.

Change-Id: Ibe8c8d3cc8297875b16ee385c077b71c87353147
2018-01-08 14:59:42 -08:00
Yi Jin
fbef27ab4a Merge "Implement a new type of section which reads from logd and dumps proto." 2018-01-08 20:52:54 +00:00
Yao Chen
d59a6589fa Some fixes in duration metrics
+ Don't need to keep the condition key if the condition is not sliced.
+ Whiten the hash before returning.

Test: statsd_test
Change-Id: Ie0bcb14b3adc4a15b79ba3be050431a672a68c28
2018-01-08 11:17:11 -08:00
Yi Jin
3c034c987e Implement a new type of section which reads from logd and dumps proto.
And implement a file section which reads from event-log-tags for
decoding binary logs.

Bug: 70936599
Test: atest incidentd_test && atest incident_helper_test and flush on
device and test log sections and event_log_tag_map
Change-Id: Ib3d35e317f355de69f01ded012482486e9a43da6
2018-01-06 20:33:14 -08:00
TreeHugger Robot
a7afd1311a Merge "Fix a few things with the loadtest app." 2018-01-06 04:37:59 +00:00
Stefan Lafon
c6f2fa2700 Fix a few things with the loadtest app.
Test: Loadtest app is running fine.

Change-Id: I834f6c618f5a6b0b17ecd5592e2ab2209ce7b94a
2018-01-05 17:02:56 -08:00
Chenjie Yu
b038b70956 pullers now cache data to throttle frequent pull requests.
all pullers have a default 1s cool down before next pull.
We can adjust these later.
Also add puller stats in StatsdStats

Test: unit test
Change-Id: I71894a24c41e059d841591312dbb852f54387b7d
2018-01-05 15:49:59 -08:00
TreeHugger Robot
4735b97d84 Merge "Updates atoms.proto for statsd." 2018-01-05 18:44:54 +00:00
TreeHugger Robot
a068cd68c0 Merge "Binding on-demand #8: Miscellaneous usages" 2018-01-05 18:21:53 +00:00
Bernardo Rufino
fa51853ae5 Binding on-demand #8: Miscellaneous usages
Migrate usages of the transport binder to binding on-demand:
* getDestinationString()
* isAppEligibleForBackup()
* dump()

For getDestinationString() we'll be introducing an invisible bug for
people that haven't updated GMSCore to include the usage of
updateTransportAttributes() API introduced in earlier CL. The bug is
that that text won't change, it'll remain constant. It's invisible
because currently only place that uses that method is Settings in some
circumstances that depend on the transport, and those circunstances
don't happen with our transports. Check http://ag/1831025.

For isAppEligibleForBackup(), a new filterAppsEligibleForBackup() is
created and there we bind on-demand.

Change-Id: Idc9e31f0e8eda8531e204c05a84fafdaf0247d08
Ref: http://go/br-binding-on-demand
Bug: 17140907
Test: adb shell dumpsys backup, observe destination of transports
Test: adb shell bmgr backupnow --all, observe only eligible apps got backed-up
Test: Force-loaded settings screen and observed destination string
Test: m -j RunFrameworksServicesRoboTests
2018-01-05 16:33:33 +00:00
Howard Ro
7266e82cf8 Merge "Use the configs from disk after reboot and separate out ConfigManager startup for unit test to avoid reading configs for tests" 2018-01-05 06:44:38 +00:00
Yangster-mac
a070b6a40c Bug fix: gauge metric producer does not need the atom id parameter.
Test: statsd unit test passed
Change-Id: Ifd1e2f7a05a03886eb227ac1ae0e30b884e49f0c
2018-01-05 04:34:49 +00:00
Jeff Sharkey
c9cbc3e5fc Merge "FBE devices now fully support adoptable storage." 2018-01-05 03:53:56 +00:00
Jeff Sharkey
8eb783b21e FBE devices now fully support adoptable storage.
We've finished all the underlying work to support adoptable storage
on FBE devices, so remove the code that was disabling it by default.

Introduce feature flag to make it easier to detect devices that
support adoptable storage.

Test: cts-tradefed run commandAndExit cts-dev -m CtsAppSecurityHostTestCases -t android.appsecurity.cts.AdoptableHostTest
Bug: 29923055, 25861755, 33252673, 37289651
Change-Id: I3630d690c9e66c7e41e316a4263ea2eb1e752ad3
2018-01-04 18:50:14 -07:00
Yangster-mac
d19bcae42d Move Field proto out of stats_log.proto to keep it clean.
Test: statsd unit test passed.

Change-Id: I31ffea40cf1b82c16bf37c706969bfb159639c7b
2018-01-04 16:39:52 -08:00
yro
469cd80840 Use the configs from disk after reboot and separate out ConfigManager
startup for unit test to avoid reading configs for tests

Bug: 70667694
Test: manual testing conducted, statsd, statsd_test
Change-Id: I49cb82d2b1d7c178b7a4c30c089d8d93e7902faa
2018-01-04 15:00:48 -08:00
TreeHugger Robot
efc331b287 Merge "Use TimeUnit enum to specify the bucket size." 2018-01-04 22:53:29 +00:00
Yangster-mac
b814481ad1 Use TimeUnit enum to specify the bucket size.
Test: all statsd unit test passed

Change-Id: I4f6b80ba2f8c984b06e46e6de6df3e546e99a968
2018-01-04 13:01:14 -08:00
Yao Chen
147ce60278 use only string type in the log source whitelist.
+ predefined "AID_X" will be provided as string type to statsd, and we will translate
  to integer uid using the static map.

Test: statsd_test

Change-Id: Ie47d8481e0c456457e6881ebb9cb4ce008e772b8
2018-01-04 09:57:03 -08:00
TreeHugger Robot
64d467ec09 Merge "1/ Use FieldMatcher to specify the value fields in value metric. 2/ rename number_of_buckets as num_buckets 3/ use double for the Alert's threshold" 2018-01-04 11:19:42 +00:00
Yangster-mac
a7fb12d2d2 1/ Use FieldMatcher to specify the value fields in value metric.
2/ rename number_of_buckets as num_buckets
3/ use double for the Alert's threshold

Test: statsd unit tests passed.
Change-Id: Id1f55f14d3712eddee561681e3cd77343f086c7a
2018-01-03 23:49:36 -08:00
TreeHugger Robot
e85fac24db Merge "Enable incidentd, it will just be idle while no one triggers it." 2018-01-04 01:40:41 +00:00
David Chen
6e3e6cbe07 Updates atoms.proto for statsd.
Updates numbering for pulled atoms to start at 10,000 so we have more
room to add more pushed atoms. Also fixes definition for dropbox
error added to more efficiently encode -1, and include the package
name when it's available.

Test: Tested the dropbox atom is logged without crashing on marlin.
Change-Id: I096911e3fe134320f2b1c6f545b0f479c7199612
2018-01-03 17:20:14 -08:00
TreeHugger Robot
48989c2744 Merge "1/ Change all "name" to id in statsD. 2/ Handle Subscription for alert. 3/ Support no_report_metric" 2018-01-03 23:36:21 +00:00
Yangster-mac
94e197cceb 1/ Change all "name" to id in statsD.
2/ Handle Subscription for alert.
3/ Support no_report_metric

Bug: 69522276
Test: all statsd unit tests passed.
Change-Id: I851b235f2d149b8602b0cad632d5bf541962f40a
2018-01-03 15:34:00 -08:00
Yi Jin
d612df381e Enable incidentd, it will just be idle while no one triggers it.
Test: N/A
Change-Id: I169108f0d61fc1fffea629eb2e862bfa7aad65f9
2018-01-03 15:22:18 -08:00
TreeHugger Robot
5459651bf6 Merge "Checks in statsd that fields are within range." 2018-01-03 20:43:48 +00:00
Kweku Adams
f5cc5759d5 incidentd: parsing ps dump into proto.
Also changing from execv to execvp so that we don't have to specify the full command path.

Bug: 65750831
Test: atest incident_helper_test
Change-Id: I92191afff4e7f9a6d08ea22ecfc2de5623d3bde5
2018-01-02 15:49:23 -08:00
TreeHugger Robot
b944bc8607 Merge "incidentd: Parsing battery_type data to proto." 2018-01-02 23:33:44 +00:00
TreeHugger Robot
ea713a3882 Merge "Add "write" comment to content tool." 2018-01-01 20:59:22 +00:00
Jeff Sharkey
e0ee5cefd6 Add "write" comment to content tool.
This gives developers an easy way to push temporary testing
configuration data into their apps.  We also now have symmetry with
the existing "read" command.

Test: builds, boots, example commands work
Bug: 64672411
Change-Id: I7bf6360915e1e4eb0d6ceaaec674f09931e28c6c
2018-01-01 12:29:58 -07:00
Yangster-mac
2087716f2b 1/ Support nested message and repeated fields in statsd.
2/ Filter gauge fields by FieldMatcher.
3/ Wire up wakelock attribution chain.
4/ e2e test: wakelock duration metric with aggregated predicate dimensions.
5/ e2e test: count metric with multiple metric condition links for 2 predicates and 1 non-sliced predicate.

Test: statsd unit test passed.

Change-Id: I89db31cb068184a54e0a892fad710966d3127bc9
2018-01-01 10:01:36 -08:00
TreeHugger Robot
303f84faa4 Merge "Splits AnomalyTracker into two files" 2017-12-29 05:18:41 +00:00
David Chen
daa9f3a679 Checks in statsd that fields are within range.
We check that both the developer-chosen label and the state fields
are within the expected ranges.

Test: Test that statsd still builds.

Change-Id: I9f76dfbf16edc4540b28a778385f416cb1306d4e
2017-12-28 17:01:49 -08:00
Kweku Adams
59e6fd792d incidentd: Parsing battery_type data to proto.
Bug: 65750835
Test: atest incident_helper_test
Change-Id: I3afefcfbd68924f8b6bd384cf1c7a41daeb243e3
2017-12-27 15:48:14 -08:00
David Chen
7d8aa4ded5 Small fix to addconfig in statsd.
AddConfiguration now returns false if the incoming byte array can't
even be parsed into the expected proto format. Unfortunately, we
can't fully evaluate if this configuration is valid. Still, this
gives the caller a chance to know that the parameters it sent can't
even be parsed.

Test: Tested that statsd still builds.
Change-Id: I7ba7c64adc4bc77549d5e9133e212a4870645786
2017-12-27 13:37:01 -08:00
Bookatz
857aaa5208 Splits AnomalyTracker into two files
Splits out DurationAnomalyTracker-specific functions into their own
subclass.

Test: the unit tests and CTS tests
Change-Id: Id6eb74d232b4a9c3a932d805d1ba3f0ba43a88b1
2017-12-22 15:05:31 -08:00
TreeHugger Robot
25f5d01738 Merge "Modify Loadtest app to be able to exclude certain metric types." 2017-12-22 18:44:21 +00:00
Yao Chen
0d7d4d4f30 Modify Loadtest app to be able to exclude certain metric types.
Test: manual
Change-Id: Iba025e4d6cd7c15a231550d382768aa7661ccc86
2017-12-22 09:13:37 -08:00
TreeHugger Robot
382ac3757d Merge "Modify system properties proto" 2017-12-21 22:41:33 +00:00
Yi Jin
526aa056bd Modify system properties proto
Test: atest incident_helper_test
Change-Id: Id81922e7f4412090e3b72b3a0fc1151995635885
2017-12-21 12:57:08 -08:00
Yao Chen
d5aa01b371 Remove the hacky HashableDimensionKey.
+ Add a real HashableDimensionKey as a wrapper of the dimension.
So we can get rid of the maps that we kept.

Pay down technical debt and reduce memory usage.

Test: statsd_test & manual
Change-Id: I233280cf1e2ce93da6a8cd4e8514abb066f4016d
2017-12-20 18:45:43 -08:00