Commit Graph

27 Commits

Author SHA1 Message Date
Muhammad Qureshi
f77913fbc6 Remove kBytesFieldAtoms from atoms_info
Bug: 150415347
Test: m
Test: bit statsd_test:*
Change-Id: I2e55b445121a39e3bf12164cd9123561e8b5bd4c
2020-03-20 10:55:46 -07:00
Muhammad Qureshi
b13a3214a1 Store annotation during collation
- Only collate atoms in the specified module.

- Replace signature_to_modules with signatureInfoMap. This maps each
signature to another map keyed by field number and whose values are vector of
annotations.

Bug: 151102006
Test: stats-log-api-gen-test
Change-Id: I25bbe4883c8f7f86a06d04d27cd425367b6d65a0
2020-03-17 20:19:04 -07:00
Muhammad Qureshi
f8460f70d0 Support multiple module annotations per atom
Bug: 150864291
Test: m stats-log-api-gen-test &&
out/host/linux-x86/nativetest/stats-log-api-gen-test/stats-log-api-gen-test

Change-Id: Ibf8869078c6f1eae13e687f0196d6ed91ec35c64
2020-03-05 14:52:05 -08:00
Tej Singh
c88fbf1e1c Fail if there are repeated fields in atoms.proto
Repeated fields arent allowed outside of attribution node and key value
pairs, so we should fail to build if someone attempts to add one.

Bug: 149956370
Test: atest stats-log-api-gen-test
Change-Id: I0caa6fcf74dabe8e962360ecccfd3a87fcaa4909
Merged-In: I0caa6fcf74dabe8e962360ecccfd3a87fcaa4909
2020-03-05 01:06:57 -08:00
Christine Tsai
f7e56a64e2 Merge "Allow default state, reset state, and nesting for binary states" 2020-02-03 22:46:06 +00:00
Muhammad Qureshi
ad94425b15 Add flag to stats-log-api-gen for WorkSource.
As part of getting rid of StatsLogInternal and moving every atom to be
logged from a module, this CL addresses 2 issues:
1. Add WorkSource logging to modules.
2. Gate WorkSource logging via a CLI flag to prevent WorkSource methods
from being included in modules that are Mainline modules and can't have
@hide calls.

Bug: 147541828
Test: m
Test: atest android.cts.statsd.atom.UidAtomTests

Change-Id: Icce2f6ed7298be5e5ceebfb05d87e6fa7cd6bdbb
2020-01-17 08:36:52 -08:00
tsaichristine
5adc7e008b Allow default state, reset state, and nesting for binary states
Every state atom can have a customized default state and reset state by
annotating atoms.proto.

Binary state atoms (such as WakelockStateChanged)  can turn on nested
counting by annotating atoms.proto as well.

Generated atoms_info.h before change: https://paste.googleplex.com/4626190063108096
Generated atoms_info.h after change: https://paste.googleplex.com/5410938863747072
Generated atoms_info.cpp before change: https://paste.googleplex.com/5726061016907776
Generated atoms_info.cpp after change: https://paste.googleplex.com/5540983737417728

Test: bit statsd_test:*
Change-Id: I845616f103e013a7927de869b8e8228cfb244090
2020-01-16 17:35:29 -08:00
tsaichristine
ed6156442d Allow first uid in attribution chain as primary field
Test: bit statsd_test:*
Bug: 142108433
Change-Id: I6a2f8e6198765d4205dac95e5f54666496b78808
2020-01-13 09:35:38 -08:00
Muhammad Qureshi
bcf6606f46 Rename log_from_module to module
Bug: 145951577
Test: m
Test: flashes successfully
Test: inspect adb logcat "*:S statsd:*"
Change-Id: I4f5fedb63f1d4c743d8dd474f14950c7f36c6999
2020-01-02 14:54:59 -08:00
Muhammad Qureshi
c8e226610c Move statsd metadata from statslog to atoms_info
Clients don't need to know about statsd metadata. Extract out metadata
from statslog.h/cpp into atoms_info.h/cpp which is only used by statsd.

Generated atoms_info.h: https://paste.googleplex.com/6303016724463616
Generated atoms_info.cpp: https://paste.googleplex.com/5717940978581504

Test: m -j
Test: old metadata in statslog matches the metadata in
atoms_info
Test: Flashes successfully
Test: adb logcat "*:S statsd:*"

Change-Id: I56ef3cc4ea1fbd2cd0130d4e9576b242efb9f627
2019-12-17 12:59:01 -08:00
Tej Singh
2910d5aa16 Java code for mainline modules to log to statsd.
Adds support for generating app code to allow java mainline modules to
call the StatsLog.writeRaw public API. Supports primitives, enums,
attribution chains, MODE_BYTES. Does not support key value pairs,
worksource methods (because worksource uses hidden apis in the
worksource object).

Test: manually tested atom logging from DocumentsUI
Test: existing autogenerated code is not modified
Test: cts will follow
Bug: 126134616

Change-Id: Ia321cf2d9952e3875ed0c7a28db1f4113711513f
2019-04-06 01:26:12 +00:00
Tej Singh
810eeb3eed Mainline logging to statsd: native static libs
Creates an annotation to specify if an atom needs to be logged from a
specific mainline module. Creates options in stats-log-api-gen so that
if the same module name is passed in, cpp/h files will be generated with
only those atoms that are specified. These files can be used to create a
static library per mainline module to log to statsd.

Test: builds
Test: atest stats-log-api-gen-test
Test: does not affect existing shared libstatslog
Test: manually created a static lib for netd and used it to log to
statsd. Used testdrive to validate the data was properly flowing.
Bug: 126134616
Change-Id: I78064f81fb4971eede8e97dacce5424e3eefd8bb
2019-03-15 18:54:02 -07:00
Andrei Onea
da01ea5b1c Add whitelist atom field option
Whitelisted atoms can be triggered from any source
Test: stats-log-api-gen-test
Bug: 119217680
Change-Id: Ia5faed04d696b59ba4ffaab13f5046f943d8a8b7
2019-02-06 18:55:06 +00:00
Yao Chen
bbdd67d19f Allow atoms to log fields in bytes format.
There are an increasing number of requests to log data in complex format to statsd, while the data
is not expected to be parsed or aggregated by statsd and only to be uploaded as events.

Instead of making an exception for each of these cases in a hard coded way, this CL add a feature to
annotate these field in atoms.proto and the stats-log-api-gen tool will produce byte array
interfaces for them.

Note that log_msg does not have byte array type, and only has string type, when statsd receives the
log, these fields are in string type. Only when the atom is written to proto, we will check if this
field should be bytes field and write it to protobuf in message format.

Change-Id: If53dd95c5826710c76d7fe982bf951a435dfc738
Fix: 118386797
Test: unit test & manual test
2018-10-24 13:12:13 -07:00
Yangster-mac
b6b77c6fb4 Make state atom field annotation name consistent between google3 and android.
Test: statsd test
Fix: b/117681672

Change-Id: I3b0145516c594cc23565df67acc1cfa9f79d306e
2018-10-13 10:40:09 -07:00
Chenjie Yu
159e4f8f46 allow use of atom definition outside of atoms.proto in pulled atoms
There will be followup cls to add support of new types that can be
nested and repeated.
+ skip StatsLog.write signature generation for pulled atoms.

Bug: 113872139
Test: manual test
Change-Id: I69d0ee0b2fe8e286e48eac20ec2c8b280c477bf0
2018-09-10 18:26:33 -07:00
Yangster-mac
e124e42582 Interface of writing key value pair atom to socket and parsing from statsd.
Test: statsd unit test
BUG: b/114231161

Change-Id: I3543900934b5e8e0677bf1e7cc454d61064a2475
2018-09-07 11:09:37 -07:00
Yangster-mac
48b3d62bfe Create log event from key value maps.
BUG: b/112816333
Test: statsd test.
Change-Id: Ib66f06186abfacd77807436379e1e142a5b87c99
2018-08-19 22:37:59 +00:00
Yao Chen
c40a19d2e4 Add uid field annotation in atoms.proto and statd memory usage optimization.
[memory]
  statsd binary size from 664k -> 600k
  memory usage 1978k -> 1813k (with no configs)
  + Avoid initialize any static map in statslog.h to avoid many copies of the map in each include.
    - Do it in cpp so that it is initialized only in places that use them

[Uid annotation]
+ Uid annotation is needed for extracting uid from dimension for UidCpuPuller.
+ After the change, stand-alone uids don't need to be in field 1 anymore.
+ Also added exclusive bit annotation in AppDied
+ Currently only allow one uid field in an Atom. This is to keep things simple until
  we find an exception.

Test: statsd_test
Bug: 73958484
Bug: 72129300

Change-Id: I8a916d5c00d5930e24ae7e0825a57dea19c0e744
2018-03-16 13:56:38 -07:00
Yao Chen
9c1debe330 Add annotation to atoms that represent a state change in atoms.proto
+ A state change atom can have one exclusive state field, and any
  number of primary key fields.

  When there is primary key in the atom, it means the state belongs to the primary key.
  For example,
  message UidProcessStateChanged {
    optional int32 uid = 1 [(stateFieldOption).option = PRIMARY];
    optional android.app.ProcessStateEnum state = 2 [(stateFieldOption).option = EXCLUSIVE];
  }

  When there is no primary key fields in the atom, the state is global.
  For example,
  message ScreenStateChanged {
     optional android.view.DisplayStateEnum state = 1 [(stateFieldOption).option = EXCLUSIVE];
  }

+ The annotation is consumed by stats_log_api_gen to generate a static map from the state
  atoms to its primary fields, and exclusive fields

+ stats_log.proto is splitted into 2 proto files, because statsd needs proto lite, and c++
  lite proto library cannot properly ignore the field options which requires full proto.

 This CL doesn't change any logic in the statsd yet. A separate CL will use the field option
 information to correctly track the state.

Test: added unit tests in stats_log_api_gen_test. and statsd_test pases.
Change-Id: I9e8a979fe81ba60efd4d854bb7087ce4b2b147ec
2018-02-21 16:46:56 -08:00
Yangster-mac
ba5b9e44bb Stats log interface for single node attribution chain.
Usually the input parameters for attribution chain is list of int64/string.
The new c++interface takes one int64 and one string as input and write
as a single node into logd.

This will help avoid allocating temp objects in battery stats java code.

Test: all statsd unit test passed.
Change-Id: Ia0f7f26b71a570bd3628c42726a5afb0a500d073
2018-01-15 16:41:29 -08:00
Yangster-mac
2087716f2b 1/ Support nested message and repeated fields in statsd.
2/ Filter gauge fields by FieldMatcher.
3/ Wire up wakelock attribution chain.
4/ e2e test: wakelock duration metric with aggregated predicate dimensions.
5/ e2e test: count metric with multiple metric condition links for 2 predicates and 1 non-sliced predicate.

Test: statsd unit test passed.

Change-Id: I89db31cb068184a54e0a892fad710966d3127bc9
2018-01-01 10:01:36 -08:00
Yangster-mac
7604aeaf99 Stats log api for attribution chain.
Test: all unit test passed.
Change-Id: I628d409e517f4f95c8da1d0c7fd4d514c1d9196d
2017-12-15 16:53:29 -08:00
Stefan Lafon
9478f3515b Generate constants for enum values.
Test: Builds successfully, tests pass and statsd works (it seems).

This will allow us to use those constants instead of literals.
The generated code only augmentes the java constant file.
If needed, the same can be done for the C++ file.
Some of the constant names are very long, but this is due to enum value names that are unnecessarily redundant with the enum names, i.e.
enum ENUM_NAME {
  ENUM_NAME_UNKNOWN = 0;
  ENUM_NAME_VALUE1 = 1;
  ENUM_NAME_VALUE2 = 2;
  ...
}
which can be fixed by avoiding the 'ENUM_NAME_' part in the value names above.
So, when possible, we should use shorter value names in stats_events.proto.

Change-Id: I1ad19b86e28d0df0f8c15d4c995d101423cff4c2
2017-10-31 11:23:09 -07:00
Yao Chen
d54f9dd625 Revert "Revert "Start auto-generating the stats log API.""
Test: builds successfully

This reverts commit 9319453998.

Change-Id: I22bca4a32adf86040b9d72ad5b45999aba28f586
2017-10-17 10:42:44 -07:00
Joe Onorato
9319453998 Revert "Start auto-generating the stats log API."
This reverts commit b81d1a7b9a.

Change-Id: I189684d8913ffffca42d9514ac88ea5cc4a44f05
2017-10-16 22:41:28 +00:00
Joe Onorato
b81d1a7b9a Start auto-generating the stats log API.
Both native and java bindings.

TODOs:
- Finish WorkSources.
- Clean up the package names for the protos.
- Put the protos in a more suitable location.

Test: stats-log-api-gen-test
Change-Id: Idf4022225e2be05106dbcf7de8e97a3337fc63e2
2017-10-15 20:25:19 -07:00