Commit Graph

2210 Commits

Author SHA1 Message Date
TreeHugger Robot
a69b6f29e0 Merge "AnomalyMonitor can get alarms based on timestamp" 2017-10-11 17:58:15 +00:00
Yi Jin
956ab6ac91 Merge "Implement go/streaming-proto native libs, part 2" 2017-10-11 17:51:37 +00:00
Bookatz
ece5f705d5 AnomalyMonitor can get alarms based on timestamp
When StatsCompanion informs statsd that an anomaly alarm has fired,
statsd will need to check to see whether an anomaly has indeed occurred.
This will require determining which anomaly alarms have times in the
past (since that indicates that the anomaly did indeed occur). This
capability is now added.

Test: adb shell data/nativetest64/statsd_test/statsd_test
Change-Id: I5464c94634df70832d2723d8c4718277776d58aa
2017-10-11 09:26:20 -07:00
David Pursell
1de1197993 Merge "iot bootaction: provide default lib name" 2017-10-11 15:49:24 +00:00
TreeHugger Robot
6edd1686e0 Merge "Fix a bug in SimpleConditionTracker." 2017-10-11 00:32:21 +00:00
Yi Jin
974a9c2885 Implement go/streaming-proto native libs, part 2
Implement ProtoOutputStream.h to write protobuf format data.

Usage of dumping proto:
ProtoOutputStream proto(fd);

proto.write(fieldId, value1); // dump a single value

// start to dump a message:
long long token = proto.start(messageFieldId);
proto.write(nestedField1, nestedValue1);
...
proto.end(token);

fieldId will be generated by protoc-cpp plugin(TBD). It is an encoded uint64_t
value, with 0 - 32 bits as its proto number, 33 - 40 bits reserved for
field type, int32, bool, string, message, etc. and 41 - 43 bits for
single, repeated or packed type. Currently packed field is not
supported.

Bug: 65641021
Test: N/A, need to wait for protoc-cpp plugin and will test in
incident_helper
Change-Id: Ic188615b950235aae0edeee4876b78d31feb5619
2017-10-10 17:16:34 -07:00
David Pursell
be09c954cb iot bootaction: provide default lib name
We want to provide a sane default for the bootaction library name so
that the developer isn't required to mess with read-only properties to
get this working.

Also small cleanup to remove duplicate and unused libs from Android.mk.

Bug: 67644323
Test: builds (master)
Test: properly loads default lib (nyc-iot-dev)
Test: logs and exits when lib doesn't exist (nyc-iot-dev)
Change-Id: I865a45f43b3594c99419e7e27c9798ee944d0db2
2017-10-10 16:51:01 -07:00
Yao Chen
4b1468538f Fix a bug in SimpleConditionTracker.
copy-paste bug

Test: manual. Will add unit tests for it.
Change-Id: I9494271a8a407b0f3c5fc0138c1afb4925c69b79
2017-10-10 15:34:42 -07:00
David Chen
de70169109 UID mapping to provide app name and version.
The UID map is updated by StatsCompanionService, which listens to broadcast
updates indicating that an app was updated/installed or removed. Also,
the entire map is updated when statsd first connects to the companion
service. Also, there is a way for metrics producers to subscribe to
updates, so that they can know when an app was upgraded.

Test: Created new unit-test for mapping and manually tested for install
and remove. Did not manually test the app upgrade.

Change-Id: I6676ae5c93b75c72d9badabb36aa9c40006db07d
2017-10-10 14:33:23 -07:00
Yao Chen
caf339d004 More complete implementation for condition and log matchers in statsd.
+ also synced proto from google3 to fix the LogEntryMatcher proto

+ MetricsManager represents StatsdConfig, it's responsible for initializing and managing all
LogEntryMatcher, Condition, and Metrics. Start review from here.

+ Added more complete StatsdConfig initialization, including building the map for:
    LogEntryMatcher -> Metrics
    LogEntryMatcher -> Condition
    Condition       -> Metrics.

    All the maps use index(int). The extra amount of memory for storing mappings help us
    quickly process log events.

  The StatsdConfig initialization process detects malformed config
  - Circle dependency
  - Missing definition
  etc.

 And once we detect ANY error, statsd will reject the config. And the resources related to this
 config will be released.

Test: Added unit tests
Change-Id: I2c4aefdbf3e2aa1701eacbb2fb5e653819ec1fbb
2017-10-10 10:03:46 -07:00
Chenjie Yu
16e9b10aa2 Merge "refactor statspuller" 2017-10-09 21:48:51 +00:00
TreeHugger Robot
4f99ad1e7b Merge "Statsd Anomaly tracking for CountMetricProducer" 2017-10-09 20:54:32 +00:00
Chenjie Yu
1a317baeda refactor statspuller
Test: manual test on device
Change-Id: Ibdec6a821e47cd2b2e7435002219c0b2e3f4c5d2
2017-10-09 12:47:45 -07:00
Bookatz
a4bc9c4a1e Statsd Anomaly tracking for CountMetricProducer
CountMetricProducer now has a CountAnomalyTracker which stores past
bucket information. Anomalies can be determined by seeing if the
information from the past and current buckets exeeds a threshold.

Test: manual
Change-Id: I35103c01dd32dcc31cb155f5685161cbaf969d03
2017-10-04 14:03:02 -07:00
David Pursell
6a3078672d Merge "iot: extract boot parameter logic." 2017-10-03 18:32:20 +00:00
Stefan Lafon
3e595b07c4 Check in new version of stats_events.proto. Rename a field.
Test: The code compiles, and most changes are cosmetic.

Change-Id: Iae06d7bbbe0212067de4302d4004d89d45812176
2017-10-02 20:55:52 -07:00
David Pursell
54a8fe4bbe iot: extract boot parameter logic.
Currently the boot parameters are tied pretty tightly to the bootaction
functionality, but volume and brightness need to be set on the
bootanimation regardless of whether there's a bootaction or not.

Extract boot parameters into a separate class to make it easier to apply
volume/brightness in a future CL.

Bug: 65462981
Test: Manual test, can succesfully read params on boot.
Change-Id: I32daad64cb8aab39fcd0ca17503218e0605ccd27
(cherry picked from commit f78561e7bbe580d0f0dbca7a615c575973ef6ce4)
2017-10-02 17:52:24 -07:00
TreeHugger Robot
6214ad9f5f Merge "Statsd can pull kernel wakelock data" 2017-10-02 18:22:53 +00:00
Yi Jin
667ded2fc9 Merge "Implement c++ native lib for streaming proto, part 1" 2017-09-30 00:08:44 +00:00
Bookatz
c68a9d21b4 Statsd can pull kernel wakelock data
When statsd is told that it is time to poll data, it asks
StatsCompanionService to pull kernel wakelock data, receives the result
(as a string), and outputs it to screen.

Still to do:
1. don't use a string; use a parcel instead
2. don't output it to screen; do something useful instead
3. do more than just kernel wakelocks
4. pull data on demand, in addition to just on periodic pulling

Test: added setPollingAlarms to statsd.main and confirmed that kernel
wakelock information was written to screen
Change-Id: I35f5164420699dea1a00c9e530b938904f1d3055
2017-09-29 15:10:47 -07:00
Yao Chen
44cf27c148 Add metric computation skeleton to statsd.
This cl is to let statsd understand statsd_config, and compute metrics
 defined in the config.

+ StatsLogProcessor is given a StatsdConfig (hard coded right now).
  We construct a MetricProducer for each of the metric, and the metrics
  share Condition and LogEntryMatchers

+ Added the CountMetricProducer type for CountMetric.

We can now count times of SCREEN_ON events given a config.

TODO: 1) conditions are not implemented.
      2) slicings are not implemented in CountMetric
      3) move the interaction to dropbox to a separate thread
      4) decide how the in memory metrics would be used by anomaly detection

Test: manual test.

      $ adb shell /system/bin/statsd

      $ cat config_file.dat | adb shell cmd stats config

Change-Id: I38f4059c0dc5a827c338131d4a6fa7d4cbe865db
2017-09-29 12:52:41 -07:00
TreeHugger Robot
078208d235 Merge "Check in new protos and constants. Test: Started statsd and verified it outputs data. Also ran statsd tests." 2017-09-29 04:46:12 +00:00
Yi Jin
b5e4ddd16b Merge "Enable Diskstats Section" 2017-09-29 00:52:18 +00:00
TreeHugger Robot
dbc2bfeaf4 Merge "Finish removing ASEC." 2017-09-28 22:37:40 +00:00
Stefan Lafon
cdb1a0ed95 Check in new protos and constants.
Test: Started statsd and verified it outputs data. Also ran statsd tests.

Change-Id: I2a438b2ddfcb1576e21acb6159bea607fed7caaa
2017-09-28 14:38:31 -07:00
Yi Jin
a5c5e8a99b Enable Diskstats Section
Bug: 67019205
Test: manually tested in incident_report tool
Change-Id: I291d4b0b0c01385e57c6ad83a12678c0f8d8746b
2017-09-28 14:09:02 -07:00
Jeff Sharkey
f8bb2445ff Finish removing ASEC.
Awhile back we explicitly blocked any new ASEC installs, with the
expectation that we'd eventually remove the logic entirely.  We've
had them disabled for about a week now without incident, so let's
rip out the remaining code.

Test: bit FrameworksCoreTests:android.content.pm.PackageHelperTests
Test: bit FrameworksCoreTests:android.content.pm.PackageManagerTests
Bug: 32913676
Change-Id: I1ecc35487420731f5c4bdf03bca5751548ce51b3
2017-09-28 11:32:57 -06:00
David Pursell
80ac4aeaa7 Merge "iot bootaction: use libandroidthings types." 2017-09-28 16:01:06 +00:00
David Pursell
1ecfdbda28 iot bootaction: use libandroidthings types.
We now provide some parameters that can be set on reboot and passed into
the bootaction library. This becomes part of the public libandroidthings
native API, so we need to use the public type.

Bug: 65462981
Test: `mma` builds successfully
Test: Test bootaction.so can receive boot params (tested on nyc-iot-dev)
Change-Id: Ibf6548730e0bac023f6a0a3aef925b0938418a10
2017-09-27 16:24:44 -07:00
TreeHugger Robot
353a02a5af Merge "Adds new utility functions for evaluating log entry matching. Includes matching for both simple and compound matchers." 2017-09-27 22:30:09 +00:00
Yi Jin
c23fad2f90 Implement c++ native lib for streaming proto, part 1
Extract protobuf class out and creates EncodedBuffer class
which holds protobuf data.
Next step is to create a ProtoOutputStream and let incident helper
adapt the change as well.
please see frameworks/base/core/java/android/util/proto

Bug: 65641021
Test: unit tested
Change-Id: I0dd343b2e62d60f091c8f857fae3452ec8da6b96
2017-09-27 12:23:51 -07:00
David Chen
dd89694f5e Adds new utility functions for evaluating log entry matching.
Includes matching for both simple and compound matchers.

Change-Id: Id913d2625d205ad9a529019e98ae805f730f4d48
Test: Added new unit-tests and checked on marlin device.
2017-09-27 10:24:01 -07:00
yro
0eee308138 Extract tag ID from log_msg and fill in stats log proto
Test: tested locally

Change-Id: Ic6b88995dc9eec367125d687e4af9277016a30cb
2017-09-26 18:20:19 -07:00
TreeHugger Robot
9e0b54848d Merge "Delay bootanimation exit until told to shutdown." 2017-09-26 23:47:07 +00:00
Ed Coyne
aa599b90f7 Delay bootanimation exit until told to shutdown.
We delay shutting down the animation if there is a boot action present
until we are told by the system to shutdown.

This addresses an issue we are seeing where we switched to a very short
bootanimation (bootanimation_mini.zip) to save CPU but this kills the
boot action prematurely.

Bug: 37992717
Test: Ran locally against imx7d.
Change-Id: I23556b21128b80d08cc55eaa761439a570b8eebe
2017-09-26 14:28:08 -07:00
Braden Kell
6dd64f3a1a Pass saved parameters to boot action
Bug: http://b/65462981
Test: Parameters in next_boot.json are passed to
      boot action; next_boot.json is moved to
      last_boot.json to allow reading by
      DeviceManagementService.

Change-Id: Ie290711ea48a3a221cfad2e9266215b76631ecbd
2017-09-25 17:53:52 -07:00
TreeHugger Robot
0121af6189 Merge "Merge multiple eventMetricData entries into a single StatsLogReport when writing to dropbox" 2017-09-25 16:56:08 +00:00
Derek Sollenberger
71b36912e0 Merge "Reduce overhead by combining libskia and libhwui into a single library." 2017-09-25 15:45:34 +00:00
yro
fbeb4cb3af Merge multiple eventMetricData entries into a single StatsLogReport
when writing to dropbox

Test: manual test.

Change-Id: I8ce323002ae1cad62703ece92bd99a5058b16fba
2017-09-24 22:42:44 -07:00
Yao Chen
3f0c1f2368 Fix a timestamp bug.
Test: manual
Change-Id: I8067fa8029c0b5702cae09202f8f9f7e443e902d
2017-09-22 21:25:11 -07:00
Yao Chen
ef99c4fa23 clang-format existing code in statsd.
Added .clang-format, because there isn't an official .clang-format file for Android framework code.

before we upload changes, do:

clang-format -style=file -i [file list]

to format the files that you touched.

Test: formatting only. NO code changes.
Change-Id: I90e87f1ee6618da8ea9bc2221c609c415a4046a8
2017-09-22 16:26:54 -07:00
TreeHugger Robot
4a2fed98c5 Merge "Add missing key for temporary battery process event and missing tag id for process Test: tested locally" 2017-09-21 09:03:05 +00:00
yro
a5c807ae80 Add timestamp pair to stats log
Test: tested locally

Change-Id: I70dfb0856c8410d55ea9cc6eaf1377961139ecd8
2017-09-21 00:06:46 -07:00
yro
e03583e261 Add missing key for temporary battery process event and missing tag id
for process
Test: tested locally

Change-Id: I1ef376ba9478e1e88ff09fb201078d4972d8d51b
2017-09-20 23:34:46 -07:00
Bookatz
906a35c814 Statsd namespace is defined and used
Statsd code now lives in android::os::statsd namespace. Existing files
are largely modified to follow this convention.

Exception: parse_util, since it seems a bit different.

Test: code compiles and existing statsd_tests still pass
Change-Id: Idf92a071b9ed172d01eb3087a4fa3609d67a038c
2017-09-20 16:58:14 -07:00
TreeHugger Robot
e63e6fc643 Merge "statsd & statscompanion communication more robust" 2017-09-20 23:11:13 +00:00
Bookatz
b487b5533e statsd & statscompanion communication more robust
If statsd or statsdcompanion crashes, or if one loads
before the other, the other will be able to accomodate.

When statsd loads, it will attempt to tell statscompanion that it's
alive, and then get on to its business, while assuming that
statscompanion is not alive. Only when statscompanion tells statsd
that it is alive, statsd will then start to use it.

When statscompanion loads, it will attempt to tell statsd that it's
alive and then do nothing (since it has nothing to do). When statsd
tells statscompanion that statsd is alive, statscompanion will respond,
telling statsd that it is alive and, if that binder call returns, will
get to work.

This way, if statsd loads first, it can work unobstructed until
statscompanion informs statsd that it is alive, at which point they
shake hands and work. Conversely, if statscompanion loads first, it will
do nothing until statsd contacts it, at which point they will shake
hands and work.

Test: manual
Change-Id: I969ad47fb8060e27814d05ad37433a02711cfa6a
2017-09-20 14:13:44 -07:00
TreeHugger Robot
8f24283631 Merge "StatsCompanionService sends messages to statsd" 2017-09-20 21:13:07 +00:00
TreeHugger Robot
45a1064bb7 Merge "incidentd: fix memory leaks in a test" 2017-09-20 20:47:06 +00:00
Bookatz
1b0b114abc StatsCompanionService sends messages to statsd
StatsCompanionService can now inform statsd that an alarm (for anomaly
alerting and for polling) has fired, so that statsd can act accordingly.

Test: manual created an alarm from statsd.main and checked logcat that
statsd received the broadcast that it fired
Change-Id: I1d33dfbee0d3e213c91dd6973d2622ecacc890c8
2017-09-20 07:51:53 -07:00