Commit Graph

36 Commits

Author SHA1 Message Date
Chenjie Yu
159e4f8f46 allow use of atom definition outside of atoms.proto in pulled atoms
There will be followup cls to add support of new types that can be
nested and repeated.
+ skip StatsLog.write signature generation for pulled atoms.

Bug: 113872139
Test: manual test
Change-Id: I69d0ee0b2fe8e286e48eac20ec2c8b280c477bf0
2018-09-10 18:26:33 -07:00
Yangster-mac
e124e42582 Interface of writing key value pair atom to socket and parsing from statsd.
Test: statsd unit test
BUG: b/114231161

Change-Id: I3543900934b5e8e0677bf1e7cc454d61064a2475
2018-09-07 11:09:37 -07:00
Yao Chen
0f5d661803 Add more accurate logging for stats log loss.
+ Only record eventual failure after the retries to get a accurate log loss count.
+ Record all types of failures which lead to log loss
+ Change the timestamp from elapsedRealtime to wallclock time for easier debugging server side
+ Also log the count too.

Bug: 80538532
Test: manually tested

Change-Id: I4fcccae3fa39c9e280a842e27c6432bb0a090b85
2018-08-22 14:21:42 -07:00
Yangster-mac
48b3d62bfe Create log event from key value maps.
BUG: b/112816333
Test: statsd test.
Change-Id: Ib66f06186abfacd77807436379e1e142a5b87c99
2018-08-19 22:37:59 +00:00
Yao Chen
39c8fe12e4 Merge "Disable statsd and make StatsLog no-op if ro.statsd.enable=false" into pi-dev
am: 38b92986df

Change-Id: Ia90bed14900a2c944165aefe00661d3979c08aed
2018-06-07 14:53:43 -07:00
Yao Chen
cf3829a696 Disable statsd and make StatsLog no-op if ro.statsd.enable=false
Bug: 79324611
Test: manually tested on marlin_svelte
Change-Id: Ic7fbf86d7c295cb6b2f20e7f9a957d1ed74c42c0
2018-06-06 17:25:08 -07:00
Bookatz
0bd972083a Wean statsd off batterystats: workchain (cp)
Modified cherry pick from master:
(cherry picked from commit d888df2ba9)
covering ag/4210168 and ag/4230692.

The tool that generates StatsLogInternal.java now includes the following
type of code, to accomodate WorkSources automatically.

    public static void write(int code, WorkSource ws, int arg2) {
        for (int i = 0; i < ws.size(); ++i) {
            write_non_chained(code, ws.get(i), ws.getName(i), arg2);
        }
        ArrayList<WorkSource.WorkChain> workChains = ws.getWorkChains();
        if (workChains != null) {
            for (WorkSource.WorkChain wc : workChains) {
                write(code, wc.getUids(), wc.getTags(), arg2);
            }
        }
    }

This will reduce a lot of code repitition for the StatsLog.write calls
in BatteryStats. Consequently, doing so will make it easier to remove
these StatsLog.write calls out of BatteryStats.

Bug: 80308558
Bug: 80314737
Test: manual verification
Test: com.android.internal.os.BatteryStatsTests
Change-Id: Ied537a6195121f16ef49b6bccf58c8c31964909c
Merged-In: Ibd28297345f4ab926dec01a89de0f1bfff9f8871
2018-06-05 19:46:23 +00:00
Yao Chen
c9c242e5cf Move libstatssocket to system/ so that lmkd can build on PDK.
Bug: 79349329
Test: builds locally
Change-Id: I9e767639f39fcd317ba0d8daf353d9af3bb73359
2018-05-07 17:00:46 -07:00
Yao Chen
88e881017c Support stats log in lmkd.
This CL changes the relevant cpp files to c files. And added functionalities to reset the log
context for reuse.

Background:
+ lmkd doesn't use the generated statslog code because:
   1. lmkd doesn't want to create a 4K log context for each log. lmkd is single threaded and can
      reuse the same log context.
   2. lmkd is written in c, and the existing statslog.h/.cpp are not compatible

Bug: 78603347
Test: manually tested with alloc-stress
Change-Id: Ife6f5c69248ecf5af730269e67f229ba4c72f37f
2018-05-03 15:42:17 -07:00
Yao Chen
f7bc6ab7fa Allow StatsLog to directly write to statsd's socket.
+ Reuse the log_event_list from liblog. StatsLog's binary format remains unchanged
+ Copied socket write code from liblog, including the retry logic.
+ Added build flags to control the StatsLog channel (logd, statsd, or both for debugging)

Bug: 78239479
Test: locally tested and saw logs being written to statsd

Change-Id: I7b1f0069ead00bbf3c29e4bd5b7f363a7ce26abe
2018-04-20 14:52:01 -07:00
Yangster-mac
ca5c0869f0 Logd retry throttling.
Test: statsd test

BUG: b/77852322
Change-Id: I3b4c46aaa592e21bfed13330c2ca72fac8e8320a
2018-04-10 11:20:57 -07:00
Yangster-mac
b8382a10a0 Retry logs write when it fails.
Report skipped event in statsd.

Test: manual test
BUG: b/77222120
Change-Id: I257f5e76d557893c4eb4a8e8a13396d8b5d1afc0
2018-04-04 17:53:48 -07:00
Yao Chen
97e21ec340 Surface the log write errors to StatsLog.write
Test: statsd_test
Bug: 77222120
Change-Id: I1c6e41a3949ce21fbec6893da7f5f94f5aec8c36
2018-03-29 11:00:38 -07:00
Yao Chen
c40a19d2e4 Add uid field annotation in atoms.proto and statd memory usage optimization.
[memory]
  statsd binary size from 664k -> 600k
  memory usage 1978k -> 1813k (with no configs)
  + Avoid initialize any static map in statslog.h to avoid many copies of the map in each include.
    - Do it in cpp so that it is initialized only in places that use them

[Uid annotation]
+ Uid annotation is needed for extracting uid from dimension for UidCpuPuller.
+ After the change, stand-alone uids don't need to be in field 1 anymore.
+ Also added exclusive bit annotation in AppDied
+ Currently only allow one uid field in an Atom. This is to keep things simple until
  we find an exception.

Test: statsd_test
Bug: 73958484
Bug: 72129300

Change-Id: I8a916d5c00d5930e24ae7e0825a57dea19c0e744
2018-03-16 13:56:38 -07:00
TreeHugger Robot
868688ed16 Merge "Add annotation to atoms that represent a state change in atoms.proto" 2018-02-22 05:36:45 +00:00
Yao Chen
9c1debe330 Add annotation to atoms that represent a state change in atoms.proto
+ A state change atom can have one exclusive state field, and any
  number of primary key fields.

  When there is primary key in the atom, it means the state belongs to the primary key.
  For example,
  message UidProcessStateChanged {
    optional int32 uid = 1 [(stateFieldOption).option = PRIMARY];
    optional android.app.ProcessStateEnum state = 2 [(stateFieldOption).option = EXCLUSIVE];
  }

  When there is no primary key fields in the atom, the state is global.
  For example,
  message ScreenStateChanged {
     optional android.view.DisplayStateEnum state = 1 [(stateFieldOption).option = EXCLUSIVE];
  }

+ The annotation is consumed by stats_log_api_gen to generate a static map from the state
  atoms to its primary fields, and exclusive fields

+ stats_log.proto is splitted into 2 proto files, because statsd needs proto lite, and c++
  lite proto library cannot properly ignore the field options which requires full proto.

 This CL doesn't change any logic in the statsd yet. A separate CL will use the field option
 information to correctly track the state.

Test: added unit tests in stats_log_api_gen_test. and statsd_test pases.
Change-Id: I9e8a979fe81ba60efd4d854bb7087ce4b2b147ec
2018-02-21 16:46:56 -08:00
Yangster-mac
c4aa760428 Remove a few atoms from the timestamp truncating whitelist.
Test: statsd test
Change-Id: Ic6a152cb400bef33139357d60f6d0feca855ade2
2018-02-20 13:20:54 -08:00
Yangster-mac
d5c356214f Timestamp Truncating whitelist.
Test: manually tested.
Change-Id: I46da375b6c0773ffc611bc06fff12cb4f9a7fc18
2018-02-17 21:11:54 -08:00
Yangster-mac
31ddcde690 Log the elapsed realtime in non-chained interface.
Test: statsd test
Change-Id: I61e457bd069147fd1b1fc964b27d42d4ec19230d
2018-02-14 16:09:35 -08:00
Yangster-mac
330af58f2b Use elapsed realtime instead of times based on wall clock, which can jump around and go backwards.
Test: statsd unit test passed

Change-Id: Ib541df99231e171b3be2a24f75632693e36da90e
2018-02-13 23:30:39 -08:00
Yangster-mac
68985805f2 Avoid processing log event when there is no uid field.
Test: all statsd unit test passed

Change-Id: Id434d86586950a485b30a244f3c030e8202c1c6d
2018-01-23 16:43:07 +00:00
Yangster-mac
ba5b9e44bb Stats log interface for single node attribution chain.
Usually the input parameters for attribution chain is list of int64/string.
The new c++interface takes one int64 and one string as input and write
as a single node into logd.

This will help avoid allocating temp objects in battery stats java code.

Test: all statsd unit test passed.
Change-Id: Ia0f7f26b71a570bd3628c42726a5afb0a500d073
2018-01-15 16:41:29 -08:00
Yangster-mac
20ac944ea0 Handle null string in jni and c++ stats-log-api interfaces.
Test: statsd unit test passed.
Change-Id: I4a475d6fcc0b4a0293450bc0ccdd718b362c498b
2018-01-08 14:54:48 -08:00
Yangster-mac
2087716f2b 1/ Support nested message and repeated fields in statsd.
2/ Filter gauge fields by FieldMatcher.
3/ Wire up wakelock attribution chain.
4/ e2e test: wakelock duration metric with aggregated predicate dimensions.
5/ e2e test: count metric with multiple metric condition links for 2 predicates and 1 non-sliced predicate.

Test: statsd unit test passed.

Change-Id: I89db31cb068184a54e0a892fad710966d3127bc9
2018-01-01 10:01:36 -08:00
TreeHugger Robot
58a6cf6afe Merge "Stats log api for attribution chain." 2017-12-16 04:15:11 +00:00
Yangster-mac
7604aeaf99 Stats log api for attribution chain.
Test: all unit test passed.
Change-Id: I628d409e517f4f95c8da1d0c7fd4d514c1d9196d
2017-12-15 16:53:29 -08:00
David Chen
0a368b2c39 Adds API for apps to push events to statsd.
This API allows app to construct custom metrics based on labels
chosen by the app developers. Also added some buttons to manually
test this functionality in the dogfood app.

Test: Verified that Android can be built and tested with custom app.
Bug: 69522276
Change-Id: Ifb7abea4c1d62fb435a9cb6f32df12bc2234d82f
2017-12-14 16:19:29 -08:00
Yao Chen
b356151e63 Add StatsdStats and guardrail.
+ StatsdStats is the global class that tracks the stats about statsd.

+ Added guardrail for classes that have a map which could potentially grow
  unboundedly with the number of logs.

TODO: add unit tests & CTS for StatsdStats, and guardrail
      add stats for pulled atoms.

Test: statsd_test

Change-Id: I0ea562de4dd3f6162f7923a9c193420b482c1d51
2017-11-27 10:52:54 -08:00
Yao Chen
80235403d2 statsd: parse the new format of stats log
+ Changed how we construct LogEvent, now it's based on the context from log_msg
  without making a copy of the list.

+ All stats logs now have the same event tag, the atom id is the first elem in the log.

Test: statsd_test
Change-Id: I4419380f2ee1c2b2155d427b9f2adb78883b337f
2017-11-17 11:53:15 -08:00
Chih-hung Hsieh
5209abad9b Merge "Use -Werror in frameworks/base/tools/stats_log_api_gen" 2017-11-16 19:09:45 +00:00
Stefan Lafon
ae2df01aae Rename "stats_events" as "atom".
Also make atom WakelockStateChanged use the "Type" enum instead of int32.

Test: Unittests are passing. Statsd is working.

Change-Id: I0909e2d97297f78996a81366d66aae62d5bf5ce1
2017-11-14 11:56:05 -08:00
Chih-Hung Hsieh
ca189f4db6 Use -Werror in frameworks/base/tools/stats_log_api_gen
* Remove unused local variables.

Bug: 66996870
Test: build with WITH_TIDY=1
Change-Id: I24b33facec0bfd73eb5c2a9759b42f9c98ee285f
2017-11-10 15:10:34 -08:00
Stefan Lafon
9478f3515b Generate constants for enum values.
Test: Builds successfully, tests pass and statsd works (it seems).

This will allow us to use those constants instead of literals.
The generated code only augmentes the java constant file.
If needed, the same can be done for the C++ file.
Some of the constant names are very long, but this is due to enum value names that are unnecessarily redundant with the enum names, i.e.
enum ENUM_NAME {
  ENUM_NAME_UNKNOWN = 0;
  ENUM_NAME_VALUE1 = 1;
  ENUM_NAME_VALUE2 = 2;
  ...
}
which can be fixed by avoiding the 'ENUM_NAME_' part in the value names above.
So, when possible, we should use shorter value names in stats_events.proto.

Change-Id: I1ad19b86e28d0df0f8c15d4c995d101423cff4c2
2017-10-31 11:23:09 -07:00
Yao Chen
d54f9dd625 Revert "Revert "Start auto-generating the stats log API.""
Test: builds successfully

This reverts commit 9319453998.

Change-Id: I22bca4a32adf86040b9d72ad5b45999aba28f586
2017-10-17 10:42:44 -07:00
Joe Onorato
9319453998 Revert "Start auto-generating the stats log API."
This reverts commit b81d1a7b9a.

Change-Id: I189684d8913ffffca42d9514ac88ea5cc4a44f05
2017-10-16 22:41:28 +00:00
Joe Onorato
b81d1a7b9a Start auto-generating the stats log API.
Both native and java bindings.

TODOs:
- Finish WorkSources.
- Clean up the package names for the protos.
- Put the protos in a more suitable location.

Test: stats-log-api-gen-test
Change-Id: Idf4022225e2be05106dbcf7de8e97a3337fc63e2
2017-10-15 20:25:19 -07:00