Commit Graph

3581 Commits

Author SHA1 Message Date
Bookatz
ff71cadecc Statsd can dump data as proto to bugreport
* Creates an incident section for statsd data.
* Allows dump to output statsd data, in proto format.
* Hooks up two statsd outputs to bugreports:
  -statsd report data in proto format
  -statsd metadata (statsdstats) in text format

The incident section does not import stats_log.proto because that turns
out to be extremely difficult: stats_log.proto imports atoms.proto,
which imports more things and is enormous and causes all sorts of
problems. atoms.proto was purposefully never compiled in AOSP, so to
retain that feature, the incident section uses 'bytes' instead of an
actual message. Since this isn't ever read in AOSP (other than testing),
this should be fine.

Bug: 115678461
Test: take a bug report and confirm valid proto
Test: cts-tradefed run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.atom.HostAtomTests#testDumpsysStats

Change-Id: I1c370af7678d1dc7440ce299ea5ea4da6d33832b
2018-11-05 18:52:49 -08:00
TreeHugger Robot
5f2a965757 Merge "record atom pull time" 2018-11-01 21:33:30 +00:00
Chenjie Yu
48ed1cc74f record atom pull time
output looks like this:
********Pulled Atom stats***********
Atom 10003->(total pull)2, (pull from cache)0, (min pull interval)0,
(average pull time nanos)528047, (max pull time nanos)568386, (average
pull delay nanos)4222422, (max pull delay nanos)4498125
Atom 10019->(total pull)2, (pull from cache)0, (min pull interval)0,
(average pull time nanos)4455885, (max pull time nanos)6245938, (average
pull delay nanos)12153985, (max pull delay nanos)13518387
Atom 10020->(total pull)2, (pull from cache)0, (min pull interval)0,
(average pull time nanos)3092005, (max pull time nanos)3223594, (average
pull delay nanos)18301668, (max pull delay nanos)19805783
Atom 10021->(total pull)3, (pull from cache)0, (min pull interval)0,
(average pull time nanos)3151944, (max pull time nanos)4931719, (average
pull delay nanos)17319897, (max pull delay nanos)24455211

Bug: 118757386
Test: unit test
Change-Id: Idbe0e4cc822d75813d2e4511be5e7593fff370d4
2018-11-01 11:30:52 -07:00
Rafal Slawik
272a816155 Add process start time to ProcessMemoryState atom
Also, add the start time to non-additive in statsd.

Bug: 118249210
Test: manually verified values are included in a report
Change-Id: Ib91d4d0f416a1cbd298f4a010e8264a3e8f0ee16
2018-11-01 16:32:58 +00:00
Howard Ro
1260bb4739 Manifest for default implementation of stats hal
Bug: 118509430
Test: builds successfully. logging tests will be conducted once client
implementation is in.

Change-Id: If0085273b43316bce4c266829012a618193d6bd8
2018-10-31 15:08:51 -07:00
Olivier Gaillard
936b54b261 Merge "Update the binder calls atom to track the calling app of a binder call." 2018-10-31 16:00:27 +00:00
Colin Cross
b7fddfc6a6 Merge "Convert statsd and incidentd to Android.bp" 2018-10-30 23:28:28 +00:00
Colin Cross
bc2dc31c4d Convert statsd and incidentd to Android.bp
See build/soong/README.md for more information.

Fixes: 110563449
Test: m checkbuild
Change-Id: I8c80878cec46c9299234c6f4c1e0e19efc0a221e
2018-10-30 13:30:23 -07:00
Olivier Gaillard
720ec5baf9 Update the binder calls atom to track the calling app of a binder call.
Test: manual
Change-Id: I0c63298e56b85eb69ac7f909c12414cc43a964a9
2018-10-30 17:33:00 +00:00
TreeHugger Robot
0b4e17bee2 Merge "idmap2: initial code drop" 2018-10-30 17:00:24 +00:00
Misha Wagner
5a51e004b1 Add atom to StatsCompanionService for retrieving per-thread CPU usage data
N.B.: This calls CpuThreadProcReader synchronously in
StatsCompanionService::pullData. This call takes approximately 50.079ms on a
Pixel 2.

Bug: 111534779
Test: `adb shell cmd stats pull-source 10035` returns correct data

Change-Id: I6fe6d178e28669f10ba9c076cbf19dc443d171c9
2018-10-30 15:34:07 +00:00
Mårten Kongstad
0275123eef idmap2: initial code drop
idmap2 is a reboot of the idmap project. The project aims to

  - use modern C++
  - greatly improve test and debug support
  - interface towards AssetManager2 (instead of AssetManager)
  - provide a solid foundation to add support for new features

To make it easier to verify correctness, this first version of idmap2 is
feature equivalent to idmap. Later versions will add support for new
features such as <overlayable>.

Bug: 78815803
Test: make idmap2_tests
Change-Id: I1d806dc875a493e730ab55d2fdb027618e586d16
2018-10-30 04:37:41 -07:00
Chenjie Yu
c3c30c0b30 add drop byte info in statsdstats
Test: manual test and unit test
Change-Id: I8675b399f83b2bb43be4fc2e6c8f28db37b8863b
2018-10-29 21:48:38 -07:00
TreeHugger Robot
761b0bd7c9 Merge "Adding tests and move launcher.proto" 2018-10-29 21:52:52 +00:00
Rafal Slawik
bf67d07be2 Record process start time for native processes
Process start time is recorded to detect whether two memory samples
come from the same process and how long the process was alive.

Bug: 118249210
Test: atest MemoryStatUtilTest and manually verified that data is in
statsd report

Change-Id: I7f49cd8bfc81c5e7e70e4f8b49729632eeec5798
2018-10-29 14:36:37 +00:00
Yao Chen
8c43386145 Adding tests and move launcher.proto
Test: statsd_test

Change-Id: I0655f70098d78f8f6dc9381921c4a687d2195bd4
2018-10-26 10:28:05 -07:00
Hyunyoung Song
4f8f072f7c Merge "Add launcher.proto to atoms.proto" 2018-10-26 04:57:58 +00:00
TreeHugger Robot
1cfaeed73b Merge "Log uid info to KeyValuePairAtom" 2018-10-25 23:57:34 +00:00
Hyunyoung Song
c6d6b77ddc Add launcher.proto to atoms.proto
Test: blank
Bug: 113043444

Change-Id: I7e0e72cdf68dcf195188dbab8fe2567fcbd5964e
2018-10-25 15:22:12 -07:00
Howard Ro
1a2a399419 Log uid info to KeyValuePairAtom
Bug: 118153464
Test: verified the data via print-logs in logcat
Change-Id: I74fcde1a7d2b2c37eb6d63e40fb5cb40ebb9cff9
2018-10-25 13:26:42 -07:00
TreeHugger Robot
03b8c0f49c Merge "Allow atoms to log fields in bytes format." 2018-10-25 19:04:17 +00:00
Wei Wang
86d0d9d8e5 Merge "BootAnimation: Use system log" 2018-10-25 03:56:48 +00:00
TreeHugger Robot
c242dac241 Merge "[Screenshot] Accept data space and pixel format for screenshot." 2018-10-25 03:29:31 +00:00
TreeHugger Robot
fd5117d680 Merge "make testdrive log message less verbose" 2018-10-25 02:16:35 +00:00
Yangster-mac
308ea0cda8 Pull process cpu stats atom.
Bug: 113353129
Test: manual + statsd_test

Change-Id: I4dde181e328506c119aa3994b1a243d7a9d61e25
2018-10-25 01:01:16 +00:00
Chenjie Yu
328432d887 make testdrive log message less verbose
1. Use custom TestDriveFormatter for log output.
It should be easy to fine tune it from here.
2. Fine tune logging level

Now the output looks like this:

./out/host/linux-x86/bin/statsd_testdrive 10
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.Utf8$UnsafeProcessor (file:/android2/master/out/host/linux-x86/framework/statsd_testdrive.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.Utf8$UnsafeProcessor
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Pushed the following config to statsd:
id: 12345
event_metric {
  id: 1111
  what: 1234567
}
atom_matcher {
  id: 1234567
  simple_atom_matcher {
    atom_id: 10
  }
}

......

hash_strings_in_metric_report: false

Now please play with the device to trigger the event. All events should be dumped after 1 min ...
Success!
Got following metric data dump:
metric_id: 1111
event_metrics {
  data {

.....

        state: RELEASE
      }
    }
  }
}

Test: manual test
Change-Id: I9fcb09bbd1663f0eae85d428ed45a3b139f62782
2018-10-24 16:26:05 -07:00
Yao Chen
bbdd67d19f Allow atoms to log fields in bytes format.
There are an increasing number of requests to log data in complex format to statsd, while the data
is not expected to be parsed or aggregated by statsd and only to be uploaded as events.

Instead of making an exception for each of these cases in a hard coded way, this CL add a feature to
annotate these field in atoms.proto and the stats-log-api-gen tool will produce byte array
interfaces for them.

Note that log_msg does not have byte array type, and only has string type, when statsd receives the
log, these fields are in string type. Only when the atom is written to proto, we will check if this
field should be bytes field and write it to protobuf in message format.

Change-Id: If53dd95c5826710c76d7fe982bf951a435dfc738
Fix: 118386797
Test: unit test & manual test
2018-10-24 13:12:13 -07:00
Peiyong Lin
10a34d1074 [Screenshot] Accept data space and pixel format for screenshot.
Previously we always take screenshot in SRGB data space. It's been working well
since the screen only has sRGB color. However, since we are moving towards
wider color, we can't make this assumption any more. Thus we need to pass the
data space and pixel format of the screenshot buffer into SurfaceFlinger and
render screenshot in the desired data space and pixel format.

BUG: 111436479
BUG: 116112787
Test: Build, flash, boot and take screenshot, observe there's no color shift.
Test: Take screenshot, verified the color profile is correct of the png.
Change-Id: I9b1a1c878f8099715c2a2c296aaf3419ca6e9bd5
2018-10-24 11:34:08 -07:00
Wei Wang
159a410d7b BootAnimation: Use system log
To match the rest of booting log in system_server,
e.g. SystemServerTiming and SystemUIBootTiming.

Test: build
Bug: 118241273
Change-Id: Idf6aca0c503b85c645a7c3d5f430c321a8d83cca
2018-10-23 23:15:58 -07:00
TreeHugger Robot
5989449661 Merge "Move Settings atom enums to its own file." 2018-10-23 23:12:07 +00:00
Howard Ro
d845abbac0 Merge "Notification atoms changes" 2018-10-23 21:57:49 +00:00
Fan Zhang
f837b8e442 Move Settings atom enums to its own file.
Bug: 117860032
Test: rebuild
Change-Id: I26f634c36de0cf946951d85f0000468e3726db23
2018-10-23 13:04:07 -07:00
TreeHugger Robot
845a41cb8a Merge "Migrate a few metrics from metrics_contant to atom proto" 2018-10-23 18:24:57 +00:00
Howard Ro
183c2bd09a Notification atoms changes
Test: Adding an atom doesn't break Android!

Change-Id: I43d5bbc808a58856fd27132824bea1d022022da0
2018-10-22 19:00:25 -07:00
Fan Zhang
2e6d4243ec Migrate a few metrics from metrics_contant to atom proto
Bug: 117860032
Test: rebuild
Change-Id: Idfd48123beed4c5be02b12f38bafa975e89f7dfb
2018-10-22 15:51:39 -07:00
Fan Zhang
74494b5172 Merge "Add a Settings atom" 2018-10-22 22:24:11 +00:00
Sudheer Shanka
23bab6c766 Merge "Add instrumentation flag to get access to full exteranl storage." 2018-10-22 21:43:29 +00:00
Chih-Hung Hsieh
e1afb6cb29 Allow implicit-fallthrough warnings locally.
It will be a global error by default.
Cherry pick lost changes from AOSP, for Android.mk and main.cpp.

Test: make checkbuild
Bug: 112564944
Change-Id: I6f1024fa5e31302af27fa9d5bac195e511f4ba20
Exempt-From-Owner-Approval: do not block on new warnings
2018-10-22 12:25:50 -07:00
Fan Zhang
916c13b5f5 Add a Settings atom
Bug: 117860032
Test: rebuild
Change-Id: I0cd0422891ca9a18168eb57cf4b61ad1089263f6
2018-10-22 11:34:50 -07:00
Yang Lu
1587e13496 Revert "Pull process cpu stats into statsd"
This reverts commit b35b5f7c6b.

Reason for revert: move the refactoring/renaming to another CL.

Test: statsd test

Change-Id: Ia3445beb0df813adba0d3d6e137a363e694b5da1
2018-10-22 09:31:04 -07:00
Antoan Angelov
07a9c42c39 Merge "Add UserRestrictionChanged atom and log to track user restriction events." 2018-10-22 11:59:25 +00:00
Sudheer Shanka
8f99bff2e4 Add instrumentation flag to get access to full exteranl storage.
When starting a process under instrumentation using this new
"--no-isolated-storage" flag, the process would be able to see the
full external storage.

Bug: 117229024
Test: adb shell am instrument --no-isolated-storage \
      -e class android.media.cts.EncodeDecodeTest#testEncodeDecodeVideoFromBufferToBuffer720p \
      -w android.media.cts/android.test.InstrumentationTestRunner
Change-Id: I7973b123cf4fc08e8ce2b05bd4c23fa41b1cdcdf
2018-10-21 16:30:52 -07:00
arangelov
d5db50ee93 Add UserRestrictionChanged atom and log to track user restriction events.
Test: none
Bug: 114382154
Change-Id: I20dcdce657e1fed3057d84df5f427542bbabf2cc
2018-10-19 17:26:09 +01:00
Rafal Slawik
37c3c958d7 Merge "Rename pgfault and pgmajfault fields" 2018-10-19 07:41:20 +00:00
TreeHugger Robot
7dd9e171c2 Merge "metric activation and TTL." 2018-10-18 21:32:29 +00:00
Yangster-mac
849dfdc25b metric activation and TTL.
Currently, once a metric config is pushed to statsD, it will
start to collect metrics immediately. This CL introduces the metric
activation logic. When metric needs an activation, the metric producer
will hold until the activation event is detected. Then the metric producer
starts metric generation until the TTL expires (timebomb).

This is to support Mainline where it wants to collect a few metrics for
a few hours when the binary push starts or flag flips.

Test: statsd test
BUG: b/117858835
Change-Id: I992ae98f4303d5b79932eb94eddf6c19ded3727e
2018-10-18 10:45:31 -07:00
Rafal Slawik
298e07fb04 Rename pgfault and pgmajfault fields
Bug: 115968899
Test: existing tests pass
Change-Id: Iff5932c55941772dda5d85ee1ccdd243ec8856c8
2018-10-18 14:44:44 +01:00
Yangster-mac
32f07af29c Match pulled events in gauge metric.
Bug: b/117703265

Test: statsd unit tests
Change-Id: Ia4c67ebfdb4f9647d4135c6879f04faa4efdd471
2018-10-17 17:47:11 -07:00
TreeHugger Robot
7c5c113f53 Merge "Snapshot memory state for some native processes" 2018-10-17 16:30:55 +00:00
Rafal Slawik
0862158f13 Snapshot memory state for some native processes
Which processes to snapshot is controlled by a whitelist.

Benchmark for taking the snapshot:
https://docs.google.com/spreadsheets/d/1vG9ku8Uu8104CmKbO4cNeEKVeeByvHY--p0_dK1GAdA/edit?usp=sharing
(The difference between the first two sheets.)
~20ms constant cost plus ~4ms per process.

Bug: 115968899
Test: manually verified that statsd is included in the report
Change-Id: Iba680531c563ba28fae849e44044313866b2103f
2018-10-17 10:34:27 +01:00