Commit Graph

3633 Commits

Author SHA1 Message Date
Chenjie Yu
bb0fed59fd Merge "log connectivity state change" 2018-11-14 15:02:50 +00:00
Chenjie Yu
ae9dfacfc1 log connectivity state change
log when networks become available and lost, for all transport types
In statsd, we can use net id to uniquely identify a network. We should
be able to count connectivity state changes similar to what BatteryStats
does.

Bug: 115561340
Test: cts test
Change-Id: Ieac4d243e27d5d368a77e5607dec51964912240d
2018-11-13 19:39:02 -08:00
TreeHugger Robot
94b33fb532 Merge "Statsd pulls on-device power measurements" 2018-11-13 22:29:27 +00:00
Risan
f8771d362c Merge "Introducing StubVolume in StorageManager" am: b499af866c am: eb871cc916
am: ee32e0d473

Change-Id: I4098132df9833b7cbac0db9bdadac297d324a822
2018-11-13 12:12:23 -08:00
Risan
ee32e0d473 Merge "Introducing StubVolume in StorageManager" am: b499af866c
am: eb871cc916

Change-Id: I69240d32659b511aec0e1e029fb6c6f1ab455c9c
2018-11-13 11:57:12 -08:00
TreeHugger Robot
fddeab3668 Merge "add UNKNOWN to proto enum" 2018-11-13 18:57:35 +00:00
Bookatz
92da2835b1 Statsd pulls on-device power measurements
Pulls power rail measurements into statslog.

Bug: 119034725
Test: cts-tradefed run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.atom.HostAtomTests
(after enabling OPTINAL_TESTS_ENABLED)

Change-Id: If083ae50160d508a9d180f748f9515557d35d8a1
2018-11-13 10:42:17 -08:00
Chenjie Yu
e3eab89df9 add UNKNOWN to proto enum
Test: manual test
Change-Id: Ibb6817d2c597eb03d30c7b05433da51bc81ce501
2018-11-13 09:24:01 -08:00
Todd Kennedy
d65a1323c0 Merge "idmap2: silence irrelevant warning during 'idmap2 scan'" 2018-11-13 17:01:32 +00:00
TreeHugger Robot
e6730cbd83 Merge "filter pulled events for ValueMetric" 2018-11-13 03:55:22 +00:00
TreeHugger Robot
3ab40f1866 Merge "Add sm set-isolated-storage [true|false]" 2018-11-13 01:19:24 +00:00
Chenjie Yu
054ce9cc99 filter pulled events for ValueMetric
Bug: 118153041
Test: unit test
Change-Id: I528b0a82ad8a8c32f92fadf5251b39d3d0256a2c
2018-11-12 15:27:29 -08:00
Mårten Kongstad
e63a4a28ab idmap2: silence irrelevant warning during 'idmap2 scan'
Even if an idmap file fails the is-up-to-date check during 'idmap2
scan', there is no need to print a warning: this is an expected scenario
and just means that the idmap file needs to be re-created. Remove the
print.

Useful commands for testing:

  1. adb shell rm -f /data/resource-cache/*
  2. adb shell idmap2 scan                                         \
             --recursive                                           \
             --target-package-name android                         \
             --target-apk-path /system/framework/framework-res.apk \
             --output-directory /data/resource-cache               \
             --input-directory /vendor/overlay

Test: manual: put an overlay package with isStatic="true" and       \
              targetPackage="android" in /vendor/overlay/<dir>, run \
              the commands above, verify that nothing is printed    \
              on stderr
Change-Id: I9a458a805936d0c1f6d64b0c3983365e0f0c4696
2018-11-12 13:37:01 -08:00
Chenjie Yu
32717c3b6f multi-value aggregation in ValueMetric
Allow aggregation on multiple fields, instead of one at a time.
All these fields should use the same aggregation time, use_diff,
direction, etc.
The config reuses value_field but allows multiple fields to be
specified.
The order they are specified determines the "index" of a value in the
output.

Bug: 119217634
Test: unit test
Change-Id: I38b1465d13723a897b30ee0b4f868498f60ad4db
2018-11-12 12:04:16 -08:00
TreeHugger Robot
252c7c43c9 Merge "Pass the last error code when statslog.write fails" 2018-11-12 19:27:58 +00:00
Tej Singh
719a8643db Merge "Thermal shutdown fix: cool down on writing to disk" into pi-dev am: 0cc63ef090
am: 6752e11cdc

Change-Id: I57fe90e8411b6131cec4e86507c70fea25db9670
2018-11-09 17:03:50 -08:00
Tej Singh
6752e11cdc Merge "Thermal shutdown fix: cool down on writing to disk" into pi-dev
am: 0cc63ef090

Change-Id: Ieb9fe51c3a729413a1aa7f0c044246eb1c7c836c
2018-11-09 14:41:26 -08:00
Tej Singh
42f9e06418 Thermal shutdown fix: cool down on writing to disk
Currently, statsd uses wall clock seconds to write data to disk. This
issue affects both thermal and normal shutdowns, because if two writes
occur in the same second, the more recent write will overwrite the older
write, erasing the actual data that we want.

For thermal shutdowns, we write twice. Once because of termination
signal received, and once because of binder death recipient from
statscompanion service.

For normal clean shutdowns, we write 3 times. In addition to the two
above, we write for the shutdown received signal.

This fix introduces a cool down period of 3 seconds between writing to
disk.

Bug: 112432890
Test: statsd unit tests
Test: statsd cts tests
Test: manually verified normal shutdown had 1 file written to disk
Test: manually verified thermal shutdown had 1 file written to disk
Change-Id: I4cd39de9063935e762ff7d00051ccc915f31e89a
2018-11-09 20:37:42 +00:00
Yao Chen
39b679925a Pass the last error code when statslog.write fails
The error code will give us some clue on what caused the loss (e.g., EBUSY, or ENOENT)

Test: manual
Bug: 80538532

Change-Id: I446c6e2255bdae063dfb8803ad0b702ead87c645
2018-11-09 09:56:36 -08:00
Risan
05c41e6a0a Introducing StubVolume in StorageManager
Bug: 110380403
Test: Tested in ARC++ (with Settings and vold changes in separate CLS)
- able to see the external storage under StorageSettings. Also tested
the sm command to print stubvolumes.

Change-Id: I7517260a40399bd9800424bb394512601f6af617
2018-11-08 21:48:35 +00:00
Sudheer Shanka
be0febe40a Add sm set-isolated-storage [true|false]
Bug: 119038726
Test: manual
Change-Id: I29eeec7872584f1173e9b6d31434b36487515d9e
2018-11-08 11:36:04 -08:00
Wei Wang
c2dcd92c33 Merge "Log the thermal throttling event to enable conditional temperature pulling." into pi-dev
am: e905bdb194

Change-Id: Iaa3d1aa70fcf7d1cb00f79986cfe0d6a6279c834
2018-11-07 15:26:53 -08:00
Wei Wang
e905bdb194 Merge "Log the thermal throttling event to enable conditional temperature pulling." into pi-dev 2018-11-07 22:51:12 +00:00
Chenjie Yu
e894f95053 Merge "ValueMetric overhaul" 2018-11-07 05:11:44 +00:00
Chenjie Yu
c715b9e0cc ValueMetric overhaul
This is savaged from the large cl.

1. Simplify the logic in ValueMetricProducer.
1.1 for pull data on bucket boundary, we pull on bucket end, instead of
playing with timestamp twice.
1.2 for data that require diffing, we keep a rolling diff base that gets
updated.
1.3 Now we check condition in onMatchedLogEventInternalLocked for pushed atoms. For pulled atoms, check before commit time. This was very error prone in P and caused multiple bugs. It is much simpler now.
2. Treat pushed and pulled atoms the same way and share the same
aggregation types.
4. Allow decreasing values for diffing.
5. Allow diffing for pushed atoms.
6. For diff based aggregation, if the diff value is zero, we skip
output.

Bug: 117224984
Bug: 115683963
Bug: 117975256
Bug: 113268259
Test: unit test
Change-Id: I6ee306e9f6e5a166b392c443594704e7d2792ef5
2018-11-06 17:07:29 -08:00
Jia-yi Chen
560055e29d Merge "Persist data to disk when system shuts down" into pi-dev
am: 29226627d0

Change-Id: I18a5f507f0e50d1212ee40573a3a6a687e5d0052
2018-11-06 16:52:37 -08:00
Adam Bookatz
3f5b30c42e Merge "Minor comment fix in StatsLogProcessor" 2018-11-06 22:41:45 +00:00
Colin Cross
872fd51397 Merge changes from topic "revert-protobuf3.5.2" am: a4680e1ebe am: 3e312d20cd
am: e217b6222c

Change-Id: Ib4c806582d5445c8387054dbc6fe2f6e173f1a97
2018-11-06 11:41:02 -08:00
Colin Cross
43ac27281b Revert "Convert statsd_test and statsd_benchmark to proto lite" am: 47046b80d2 am: 65200cf70e
am: bd7c4efbb6

Change-Id: Id4bab0ce60f08ce358db99c67c58cbef0fe6e589
2018-11-06 11:34:29 -08:00
Colin Cross
e217b6222c Merge changes from topic "revert-protobuf3.5.2" am: a4680e1ebe
am: 3e312d20cd

Change-Id: I096f3b5a5a985ce7926ccb71fa07eda69d078ed3
2018-11-06 11:21:11 -08:00
Colin Cross
bd7c4efbb6 Revert "Convert statsd_test and statsd_benchmark to proto lite" am: 47046b80d2
am: 65200cf70e

Change-Id: I952a9cbcb9bf9d2d479efecc28c5411e31dc9cf6
2018-11-06 11:15:44 -08:00
Bookatz
9cc7b660a9 Minor comment fix in StatsLogProcessor
Test: none; it's just a comment
Change-Id: Iafd9a5264eb13502509e7466c2773b0d5536170c
2018-11-06 10:39:21 -08:00
Adam Bookatz
bab22bc860 Merge "Statsd can dump data as proto to bugreport" 2018-11-06 08:35:47 +00:00
Bookatz
ff71cadecc Statsd can dump data as proto to bugreport
* Creates an incident section for statsd data.
* Allows dump to output statsd data, in proto format.
* Hooks up two statsd outputs to bugreports:
  -statsd report data in proto format
  -statsd metadata (statsdstats) in text format

The incident section does not import stats_log.proto because that turns
out to be extremely difficult: stats_log.proto imports atoms.proto,
which imports more things and is enormous and causes all sorts of
problems. atoms.proto was purposefully never compiled in AOSP, so to
retain that feature, the incident section uses 'bytes' instead of an
actual message. Since this isn't ever read in AOSP (other than testing),
this should be fine.

Bug: 115678461
Test: take a bug report and confirm valid proto
Test: cts-tradefed run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.atom.HostAtomTests#testDumpsysStats

Change-Id: I1c370af7678d1dc7440ce299ea5ea4da6d33832b
2018-11-05 18:52:49 -08:00
Colin Cross
44f74d111f Merge changes from topic "protobuf3.5.2" am: 1d899f6be8 am: 11a4d77516
am: c836696a92

Change-Id: I158185738327cee8f501dcba7e85de66240eaf42
2018-11-04 18:39:37 -08:00
Colin Cross
5d258093a8 Convert statsd_test and statsd_benchmark to proto lite am: e7908d7b62 am: 5742f9ffa7
am: 85201e02df

Change-Id: Id99366ec5900c71ba712dbee99fbefb5b785b827
2018-11-04 18:33:00 -08:00
Colin Cross
c836696a92 Merge changes from topic "protobuf3.5.2" am: 1d899f6be8
am: 11a4d77516

Change-Id: I6ad3772bc16f236c2c5e7ba381593b10292aabde
2018-11-04 18:13:52 -08:00
Colin Cross
85201e02df Convert statsd_test and statsd_benchmark to proto lite am: e7908d7b62
am: 5742f9ffa7

Change-Id: I5ef22cce9bbebe4355915ae3ab158faf14bf2708
2018-11-04 18:08:27 -08:00
Colin Cross
054b0c0e06 Revert "Adapt to google::protobuf::uint64 type change"
This reverts commit fa6bc27df0.

Change-Id: I9f778e6daa9ae0c6080016ab0bd1e6e414ae6061
2018-11-04 17:24:27 -08:00
Colin Cross
47046b80d2 Revert "Convert statsd_test and statsd_benchmark to proto lite"
This reverts commit e7908d7b62.

Change-Id: I585991ab1d5b35930bee6e32b82714aa11fbcb41
2018-11-04 17:24:26 -08:00
Colin Cross
e7908d7b62 Convert statsd_test and statsd_benchmark to proto lite
libplatformprotos for the device is compiled using proto lite, and
statsd_test and statsd_benchmark have protos that reference the
libplatformprotos.  With protobuf 3.5.2 compiling a full proto
against a lite proto doesn't work.  Convert them to lite.  Requires
adding the protobuf internal protos to the sources,
libprotobuf-cpp-lite does not contain the compiled version of
field_options.proto.

Bug: 117607748
Test: m checkbuild
Change-Id: I6a618c4118972c0e5ffb07a361ac6612b9069c5d
2018-11-02 11:35:48 -07:00
Colin Cross
fa6bc27df0 Adapt to google::protobuf::uint64 type change
Protobuf 3.5.2 redefines google::protobuf::uint64 from unsigned long
long to uint64_t, which is sometimes unsigned long and sometimes
unsigned long long.  Use PRIu64 to print it, and add an implementation
of ProtoOutputStream::write for long.

Bug: 117607748
Test: m checkbuild
Change-Id: I48912f58e2f63f4a98c3aefaa4f78cc2c719bb68
2018-11-02 11:35:48 -07:00
TreeHugger Robot
5f2a965757 Merge "record atom pull time" 2018-11-01 21:33:30 +00:00
Chenjie Yu
48ed1cc74f record atom pull time
output looks like this:
********Pulled Atom stats***********
Atom 10003->(total pull)2, (pull from cache)0, (min pull interval)0,
(average pull time nanos)528047, (max pull time nanos)568386, (average
pull delay nanos)4222422, (max pull delay nanos)4498125
Atom 10019->(total pull)2, (pull from cache)0, (min pull interval)0,
(average pull time nanos)4455885, (max pull time nanos)6245938, (average
pull delay nanos)12153985, (max pull delay nanos)13518387
Atom 10020->(total pull)2, (pull from cache)0, (min pull interval)0,
(average pull time nanos)3092005, (max pull time nanos)3223594, (average
pull delay nanos)18301668, (max pull delay nanos)19805783
Atom 10021->(total pull)3, (pull from cache)0, (min pull interval)0,
(average pull time nanos)3151944, (max pull time nanos)4931719, (average
pull delay nanos)17319897, (max pull delay nanos)24455211

Bug: 118757386
Test: unit test
Change-Id: Idbe0e4cc822d75813d2e4511be5e7593fff370d4
2018-11-01 11:30:52 -07:00
Rafal Slawik
272a816155 Add process start time to ProcessMemoryState atom
Also, add the start time to non-additive in statsd.

Bug: 118249210
Test: manually verified values are included in a report
Change-Id: Ib91d4d0f416a1cbd298f4a010e8264a3e8f0ee16
2018-11-01 16:32:58 +00:00
Howard Ro
1260bb4739 Manifest for default implementation of stats hal
Bug: 118509430
Test: builds successfully. logging tests will be conducted once client
implementation is in.

Change-Id: If0085273b43316bce4c266829012a618193d6bd8
2018-10-31 15:08:51 -07:00
Colin Cross
bd26d4363f Convert statsd and incidentd to Android.bp
am: 05c9e5c24a

Change-Id: Ic5b6353777baf66eb77f7fc6b9846fa4adbca186
2018-10-31 14:12:36 -07:00
Olivier Gaillard
936b54b261 Merge "Update the binder calls atom to track the calling app of a binder call." 2018-10-31 16:00:27 +00:00
Colin Cross
05c9e5c24a Convert statsd and incidentd to Android.bp
See build/soong/README.md for more information.

Test: m checkbuild
Merged-In: I8c80878cec46c9299234c6f4c1e0e19efc0a221e
Change-Id: I8c80878cec46c9299234c6f4c1e0e19efc0a221e
2018-10-31 08:01:59 -07:00
Colin Cross
b7fddfc6a6 Merge "Convert statsd and incidentd to Android.bp" 2018-10-30 23:28:28 +00:00