Commit Graph

3597 Commits

Author SHA1 Message Date
Adam Bookatz
bab22bc860 Merge "Statsd can dump data as proto to bugreport" 2018-11-06 08:35:47 +00:00
Bookatz
ff71cadecc Statsd can dump data as proto to bugreport
* Creates an incident section for statsd data.
* Allows dump to output statsd data, in proto format.
* Hooks up two statsd outputs to bugreports:
  -statsd report data in proto format
  -statsd metadata (statsdstats) in text format

The incident section does not import stats_log.proto because that turns
out to be extremely difficult: stats_log.proto imports atoms.proto,
which imports more things and is enormous and causes all sorts of
problems. atoms.proto was purposefully never compiled in AOSP, so to
retain that feature, the incident section uses 'bytes' instead of an
actual message. Since this isn't ever read in AOSP (other than testing),
this should be fine.

Bug: 115678461
Test: take a bug report and confirm valid proto
Test: cts-tradefed run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.atom.HostAtomTests#testDumpsysStats

Change-Id: I1c370af7678d1dc7440ce299ea5ea4da6d33832b
2018-11-05 18:52:49 -08:00
Colin Cross
44f74d111f Merge changes from topic "protobuf3.5.2" am: 1d899f6be8 am: 11a4d77516
am: c836696a92

Change-Id: I158185738327cee8f501dcba7e85de66240eaf42
2018-11-04 18:39:37 -08:00
Colin Cross
5d258093a8 Convert statsd_test and statsd_benchmark to proto lite am: e7908d7b62 am: 5742f9ffa7
am: 85201e02df

Change-Id: Id99366ec5900c71ba712dbee99fbefb5b785b827
2018-11-04 18:33:00 -08:00
Colin Cross
c836696a92 Merge changes from topic "protobuf3.5.2" am: 1d899f6be8
am: 11a4d77516

Change-Id: I6ad3772bc16f236c2c5e7ba381593b10292aabde
2018-11-04 18:13:52 -08:00
Colin Cross
85201e02df Convert statsd_test and statsd_benchmark to proto lite am: e7908d7b62
am: 5742f9ffa7

Change-Id: I5ef22cce9bbebe4355915ae3ab158faf14bf2708
2018-11-04 18:08:27 -08:00
Colin Cross
e7908d7b62 Convert statsd_test and statsd_benchmark to proto lite
libplatformprotos for the device is compiled using proto lite, and
statsd_test and statsd_benchmark have protos that reference the
libplatformprotos.  With protobuf 3.5.2 compiling a full proto
against a lite proto doesn't work.  Convert them to lite.  Requires
adding the protobuf internal protos to the sources,
libprotobuf-cpp-lite does not contain the compiled version of
field_options.proto.

Bug: 117607748
Test: m checkbuild
Change-Id: I6a618c4118972c0e5ffb07a361ac6612b9069c5d
2018-11-02 11:35:48 -07:00
Colin Cross
fa6bc27df0 Adapt to google::protobuf::uint64 type change
Protobuf 3.5.2 redefines google::protobuf::uint64 from unsigned long
long to uint64_t, which is sometimes unsigned long and sometimes
unsigned long long.  Use PRIu64 to print it, and add an implementation
of ProtoOutputStream::write for long.

Bug: 117607748
Test: m checkbuild
Change-Id: I48912f58e2f63f4a98c3aefaa4f78cc2c719bb68
2018-11-02 11:35:48 -07:00
TreeHugger Robot
5f2a965757 Merge "record atom pull time" 2018-11-01 21:33:30 +00:00
Chenjie Yu
48ed1cc74f record atom pull time
output looks like this:
********Pulled Atom stats***********
Atom 10003->(total pull)2, (pull from cache)0, (min pull interval)0,
(average pull time nanos)528047, (max pull time nanos)568386, (average
pull delay nanos)4222422, (max pull delay nanos)4498125
Atom 10019->(total pull)2, (pull from cache)0, (min pull interval)0,
(average pull time nanos)4455885, (max pull time nanos)6245938, (average
pull delay nanos)12153985, (max pull delay nanos)13518387
Atom 10020->(total pull)2, (pull from cache)0, (min pull interval)0,
(average pull time nanos)3092005, (max pull time nanos)3223594, (average
pull delay nanos)18301668, (max pull delay nanos)19805783
Atom 10021->(total pull)3, (pull from cache)0, (min pull interval)0,
(average pull time nanos)3151944, (max pull time nanos)4931719, (average
pull delay nanos)17319897, (max pull delay nanos)24455211

Bug: 118757386
Test: unit test
Change-Id: Idbe0e4cc822d75813d2e4511be5e7593fff370d4
2018-11-01 11:30:52 -07:00
Rafal Slawik
272a816155 Add process start time to ProcessMemoryState atom
Also, add the start time to non-additive in statsd.

Bug: 118249210
Test: manually verified values are included in a report
Change-Id: Ib91d4d0f416a1cbd298f4a010e8264a3e8f0ee16
2018-11-01 16:32:58 +00:00
Howard Ro
1260bb4739 Manifest for default implementation of stats hal
Bug: 118509430
Test: builds successfully. logging tests will be conducted once client
implementation is in.

Change-Id: If0085273b43316bce4c266829012a618193d6bd8
2018-10-31 15:08:51 -07:00
Colin Cross
bd26d4363f Convert statsd and incidentd to Android.bp
am: 05c9e5c24a

Change-Id: Ic5b6353777baf66eb77f7fc6b9846fa4adbca186
2018-10-31 14:12:36 -07:00
Olivier Gaillard
936b54b261 Merge "Update the binder calls atom to track the calling app of a binder call." 2018-10-31 16:00:27 +00:00
Colin Cross
05c9e5c24a Convert statsd and incidentd to Android.bp
See build/soong/README.md for more information.

Test: m checkbuild
Merged-In: I8c80878cec46c9299234c6f4c1e0e19efc0a221e
Change-Id: I8c80878cec46c9299234c6f4c1e0e19efc0a221e
2018-10-31 08:01:59 -07:00
Colin Cross
b7fddfc6a6 Merge "Convert statsd and incidentd to Android.bp" 2018-10-30 23:28:28 +00:00
Colin Cross
bc2dc31c4d Convert statsd and incidentd to Android.bp
See build/soong/README.md for more information.

Fixes: 110563449
Test: m checkbuild
Change-Id: I8c80878cec46c9299234c6f4c1e0e19efc0a221e
2018-10-30 13:30:23 -07:00
Olivier Gaillard
720ec5baf9 Update the binder calls atom to track the calling app of a binder call.
Test: manual
Change-Id: I0c63298e56b85eb69ac7f909c12414cc43a964a9
2018-10-30 17:33:00 +00:00
TreeHugger Robot
0b4e17bee2 Merge "idmap2: initial code drop" 2018-10-30 17:00:24 +00:00
Misha Wagner
5a51e004b1 Add atom to StatsCompanionService for retrieving per-thread CPU usage data
N.B.: This calls CpuThreadProcReader synchronously in
StatsCompanionService::pullData. This call takes approximately 50.079ms on a
Pixel 2.

Bug: 111534779
Test: `adb shell cmd stats pull-source 10035` returns correct data

Change-Id: I6fe6d178e28669f10ba9c076cbf19dc443d171c9
2018-10-30 15:34:07 +00:00
Mårten Kongstad
0275123eef idmap2: initial code drop
idmap2 is a reboot of the idmap project. The project aims to

  - use modern C++
  - greatly improve test and debug support
  - interface towards AssetManager2 (instead of AssetManager)
  - provide a solid foundation to add support for new features

To make it easier to verify correctness, this first version of idmap2 is
feature equivalent to idmap. Later versions will add support for new
features such as <overlayable>.

Bug: 78815803
Test: make idmap2_tests
Change-Id: I1d806dc875a493e730ab55d2fdb027618e586d16
2018-10-30 04:37:41 -07:00
Chenjie Yu
c3c30c0b30 add drop byte info in statsdstats
Test: manual test and unit test
Change-Id: I8675b399f83b2bb43be4fc2e6c8f28db37b8863b
2018-10-29 21:48:38 -07:00
TreeHugger Robot
761b0bd7c9 Merge "Adding tests and move launcher.proto" 2018-10-29 21:52:52 +00:00
Gopinath Elanchezhian
4e57b2c35c Merge "Add statsdprotolite in Android.bp" am: f469bd2f68
am: 748b449887

Change-Id: I94293a1cd5ca457224eccaf98ee1c265a3c3ce06
2018-10-29 11:47:54 -07:00
Gopinath Elanchezhian
f469bd2f68 Merge "Add statsdprotolite in Android.bp" 2018-10-29 17:14:46 +00:00
Rafal Slawik
bf67d07be2 Record process start time for native processes
Process start time is recorded to detect whether two memory samples
come from the same process and how long the process was alive.

Bug: 118249210
Test: atest MemoryStatUtilTest and manually verified that data is in
statsd report

Change-Id: I7f49cd8bfc81c5e7e70e4f8b49729632eeec5798
2018-10-29 14:36:37 +00:00
gopinath
49dd4c02e8 Merge "allow statsd pull based on event trigger." am: bd4ae82f66
am: 161b7028a4

Change-Id: Ibf8b0ac7e6f5f0deffb2e0c687eb9aeef4da9423
2018-10-26 21:39:51 -07:00
gopinath
17593b0395 Add statsdprotolite in Android.bp
Bug: b/118482428

Test: make statsdprotolite
Change-Id: I5c27f3017be043b6809699158a94b32155163535
Merged-In: I857957e816506162496192fe0ad1da4dbdfd4efd
2018-10-26 23:16:39 +00:00
gopinath
9d426e141d allow statsd pull based on event trigger.
Bug: b/118482428

Test: make statsdprotolite
Change-Id: I5a40d5e6a409b65ca1639029d5da83c327ed1798
Merged-In: Ia15b1f209945f022edffb9ec5d673317d55d9e4f
2018-10-26 22:22:51 +00:00
Yao Chen
8c43386145 Adding tests and move launcher.proto
Test: statsd_test

Change-Id: I0655f70098d78f8f6dc9381921c4a687d2195bd4
2018-10-26 10:28:05 -07:00
Hyunyoung Song
4f8f072f7c Merge "Add launcher.proto to atoms.proto" 2018-10-26 04:57:58 +00:00
TreeHugger Robot
1cfaeed73b Merge "Log uid info to KeyValuePairAtom" 2018-10-25 23:57:34 +00:00
Hyunyoung Song
c6d6b77ddc Add launcher.proto to atoms.proto
Test: blank
Bug: 113043444

Change-Id: I7e0e72cdf68dcf195188dbab8fe2567fcbd5964e
2018-10-25 15:22:12 -07:00
Howard Ro
1a2a399419 Log uid info to KeyValuePairAtom
Bug: 118153464
Test: verified the data via print-logs in logcat
Change-Id: I74fcde1a7d2b2c37eb6d63e40fb5cb40ebb9cff9
2018-10-25 13:26:42 -07:00
TreeHugger Robot
03b8c0f49c Merge "Allow atoms to log fields in bytes format." 2018-10-25 19:04:17 +00:00
Wei Wang
86d0d9d8e5 Merge "BootAnimation: Use system log" 2018-10-25 03:56:48 +00:00
TreeHugger Robot
c242dac241 Merge "[Screenshot] Accept data space and pixel format for screenshot." 2018-10-25 03:29:31 +00:00
TreeHugger Robot
fd5117d680 Merge "make testdrive log message less verbose" 2018-10-25 02:16:35 +00:00
Yangster-mac
308ea0cda8 Pull process cpu stats atom.
Bug: 113353129
Test: manual + statsd_test

Change-Id: I4dde181e328506c119aa3994b1a243d7a9d61e25
2018-10-25 01:01:16 +00:00
Chenjie Yu
328432d887 make testdrive log message less verbose
1. Use custom TestDriveFormatter for log output.
It should be easy to fine tune it from here.
2. Fine tune logging level

Now the output looks like this:

./out/host/linux-x86/bin/statsd_testdrive 10
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.Utf8$UnsafeProcessor (file:/android2/master/out/host/linux-x86/framework/statsd_testdrive.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.Utf8$UnsafeProcessor
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Pushed the following config to statsd:
id: 12345
event_metric {
  id: 1111
  what: 1234567
}
atom_matcher {
  id: 1234567
  simple_atom_matcher {
    atom_id: 10
  }
}

......

hash_strings_in_metric_report: false

Now please play with the device to trigger the event. All events should be dumped after 1 min ...
Success!
Got following metric data dump:
metric_id: 1111
event_metrics {
  data {

.....

        state: RELEASE
      }
    }
  }
}

Test: manual test
Change-Id: I9fcb09bbd1663f0eae85d428ed45a3b139f62782
2018-10-24 16:26:05 -07:00
Chih-hung Hsieh
eb919bc560 Merge "Allow implicit-fallthrough warnings locally." am: dd68cf76de
am: 0db2c27df7

Change-Id: Iad6403f231cd7c125a57ec0913e996224ecc3b24
2018-10-24 14:23:59 -07:00
Yao Chen
bbdd67d19f Allow atoms to log fields in bytes format.
There are an increasing number of requests to log data in complex format to statsd, while the data
is not expected to be parsed or aggregated by statsd and only to be uploaded as events.

Instead of making an exception for each of these cases in a hard coded way, this CL add a feature to
annotate these field in atoms.proto and the stats-log-api-gen tool will produce byte array
interfaces for them.

Note that log_msg does not have byte array type, and only has string type, when statsd receives the
log, these fields are in string type. Only when the atom is written to proto, we will check if this
field should be bytes field and write it to protobuf in message format.

Change-Id: If53dd95c5826710c76d7fe982bf951a435dfc738
Fix: 118386797
Test: unit test & manual test
2018-10-24 13:12:13 -07:00
Peiyong Lin
10a34d1074 [Screenshot] Accept data space and pixel format for screenshot.
Previously we always take screenshot in SRGB data space. It's been working well
since the screen only has sRGB color. However, since we are moving towards
wider color, we can't make this assumption any more. Thus we need to pass the
data space and pixel format of the screenshot buffer into SurfaceFlinger and
render screenshot in the desired data space and pixel format.

BUG: 111436479
BUG: 116112787
Test: Build, flash, boot and take screenshot, observe there's no color shift.
Test: Take screenshot, verified the color profile is correct of the png.
Change-Id: I9b1a1c878f8099715c2a2c296aaf3419ca6e9bd5
2018-10-24 11:34:08 -07:00
Wei Wang
159a410d7b BootAnimation: Use system log
To match the rest of booting log in system_server,
e.g. SystemServerTiming and SystemUIBootTiming.

Test: build
Bug: 118241273
Change-Id: Idf6aca0c503b85c645a7c3d5f430c321a8d83cca
2018-10-23 23:15:58 -07:00
TreeHugger Robot
5989449661 Merge "Move Settings atom enums to its own file." 2018-10-23 23:12:07 +00:00
Howard Ro
d845abbac0 Merge "Notification atoms changes" 2018-10-23 21:57:49 +00:00
Fan Zhang
f837b8e442 Move Settings atom enums to its own file.
Bug: 117860032
Test: rebuild
Change-Id: I26f634c36de0cf946951d85f0000468e3726db23
2018-10-23 13:04:07 -07:00
TreeHugger Robot
845a41cb8a Merge "Migrate a few metrics from metrics_contant to atom proto" 2018-10-23 18:24:57 +00:00
Howard Ro
183c2bd09a Notification atoms changes
Test: Adding an atom doesn't break Android!

Change-Id: I43d5bbc808a58856fd27132824bea1d022022da0
2018-10-22 19:00:25 -07:00
Fan Zhang
2e6d4243ec Migrate a few metrics from metrics_contant to atom proto
Bug: 117860032
Test: rebuild
Change-Id: Idfd48123beed4c5be02b12f38bafa975e89f7dfb
2018-10-22 15:51:39 -07:00