Commit Graph

123 Commits

Author SHA1 Message Date
Muhammad Qureshi
23daf2656c Only send reset state annotation when needed
Only send reset state annotation when reset state occurs.

Bug: 151776731
Test: m libstatslog

Change-Id: I6f6b4d784d3741c0059085421565eba81db5527c
2020-03-27 02:23:38 -07:00
Muhammad Qureshi
c6c38632af Group annotations for the same atom id
Group annotations for the same atom id inside one if-block in generated
code.

Use shared_ptr to store AtomDecls in multiple data structures.

Store a mapping of field numbers to atoms that have annotations at
corresponding field numbers in Collation.h

Bug: 151744250
Test: stats-log-api-gen-test
Test: m stats-log-api-gen
Test: m libstatsmetadata
Test: m statslog-framework-java-gen
Test: m libstatslog

Change-Id: I874696cfb5c27141017b4293bec809ab510ceb98
2020-03-26 14:51:28 -07:00
Muhammad Qureshi
a345af9b40 Run clang-format in stats_log_api_gen
Bug: 152253223
Test: m stats-log-api-gen
Test: stats-log-api-gen-test
Change-Id: Iad76baea8cc9ae7b3978f1133ab94d8e96bbbafe
2020-03-24 17:35:08 -07:00
Muhammad Qureshi
9b995809fb Add truncate_timestamp annotation
Mark privacy-sensitive atoms with truncate_timestamp annotation.

Factor out annotation collation to a helper method.

Add truncate_timestamp annotation support in stats-log-api-gen.

Add writeAnnotation* calls in Java and native generated code for
atom id annotations.

TODO: remove kTruncatingTimestampAtoms from atoms_info.
TODO: use truncate_timestamp annotation inside statsd.

Bug: 151111680
Test: stats-log-api-gen-test
Test: m statslog-framework-java-gen
Test: m libstatsmetadata

Change-Id: I3db5f4ffbf959bd36c62f890cc88606912798d40
2020-03-24 16:49:36 -07:00
Muhammad Qureshi
f77913fbc6 Remove kBytesFieldAtoms from atoms_info
Bug: 150415347
Test: m
Test: bit statsd_test:*
Change-Id: I2e55b445121a39e3bf12164cd9123561e8b5bd4c
2020-03-20 10:55:46 -07:00
Muhammad Qureshi
29613e8ff8 Merge "StatsEvent annotations java autogen" into rvc-dev 2020-03-19 17:20:19 +00:00
Muhammad Qureshi
38bdeb5977 Merge "Write annotations in native generated code." into rvc-dev 2020-03-19 17:19:50 +00:00
Muhammad Qureshi
8a69088a36 Merge "Store annotation during collation" into rvc-dev 2020-03-19 17:19:27 +00:00
Jeffrey Huang
f960305a31 StatsEvent annotations java autogen
Autogenerate annotations for atoms logged in java.

generated FrameworkStatsLog.java:
https://paste.googleplex.com/4789132691767296

Bug: 151102209
Test: m stats-log-api-gen
Test: m statslog-framework-java-gen
Change-Id: I50b01feaacfcb8ad5d0236432ceb0e97c8c907ab
2020-03-17 22:01:58 -07:00
Muhammad Qureshi
a31a6229dd Write annotations in native generated code.
Generated statslog.cpp: https://paste.googleplex.com/6345695252447232

Bug: 151101995
Test: m stats-log-api-gen
Test: m libstatslog
Change-Id: I745a9beb7be07a1890f811256c51627e6c5976dc
2020-03-17 21:57:35 -07:00
Muhammad Qureshi
b13a3214a1 Store annotation during collation
- Only collate atoms in the specified module.

- Replace signature_to_modules with signatureInfoMap. This maps each
signature to another map keyed by field number and whose values are vector of
annotations.

Bug: 151102006
Test: stats-log-api-gen-test
Change-Id: I25bbe4883c8f7f86a06d04d27cd425367b6d65a0
2020-03-17 20:19:04 -07:00
Jeffrey Huang
74fc4354b5 Remove apex_available from libstatslog
Test: m -j
Bug: 149781190
Change-Id: I765852b1719a3056c06a0965ea6a05a8e8410693
2020-03-10 15:14:27 -07:00
Muhammad Qureshi
d7944d6573 Merge "Remove libstatslog from libstatsmetadata" into rvc-dev 2020-03-06 08:04:46 +00:00
Muhammad Qureshi
513453c253 Merge "Support multiple module annotations per atom" into rvc-dev 2020-03-06 08:04:09 +00:00
Muhammad Qureshi
9e0f728fe4 Remove libstatslog from libstatsmetadata
Instead of relying on the constants in libstatslog, hardcode them in
atoms_info to get rid of the dependency on libstatslog

New generated atoms_info.cpp:
https://paste.googleplex.com/5779947622760448

Bug: 150417465
Test: m libstatsmetadata
Change-Id: I954c963f1883f889053b63d308c648548de71e56
2020-03-05 17:20:58 -08:00
Muhammad Qureshi
f8460f70d0 Support multiple module annotations per atom
Bug: 150864291
Test: m stats-log-api-gen-test &&
out/host/linux-x86/nativetest/stats-log-api-gen-test/stats-log-api-gen-test

Change-Id: Ibf8869078c6f1eae13e687f0196d6ed91ec35c64
2020-03-05 14:52:05 -08:00
Muhammad Qureshi
4341dbb253 Merge "Delete unused code in stats-log-api-gen" into rvc-dev 2020-03-05 21:41:17 +00:00
Muhammad Qureshi
4ef3c3bd26 Delete unused code in stats-log-api-gen
Bug: 145100015
Test: m
Change-Id: Idbbe35fbce0d751a5973128406596678e1ead5d3
2020-03-05 10:21:36 -08:00
Tej Singh
c88fbf1e1c Fail if there are repeated fields in atoms.proto
Repeated fields arent allowed outside of attribution node and key value
pairs, so we should fail to build if someone attempts to add one.

Bug: 149956370
Test: atest stats-log-api-gen-test
Change-Id: I0caa6fcf74dabe8e962360ecccfd3a87fcaa4909
Merged-In: I0caa6fcf74dabe8e962360ecccfd3a87fcaa4909
2020-03-05 01:06:57 -08:00
Tej Singh
eb7d8f4675 Make libstatssocket shared
Also make private versions of the libraries available to link statically
for tests. We should try to figure out a better long term solution.

Test: m
Test: bit statsd_test:*
Test: atest LibStatsPullTests
Test: bit libstatssocket_test:*
Test: atest GtsStatsdHostTestCases
Bug: 149340100

Change-Id: I05b91efab2a657aec75d436575aff4373f86ed3f
2020-02-27 11:38:41 -08:00
Tej Singh
d87589549f Turn off cpp aidlgen, link libstatssocket as share
Test: make
Change-Id: I052fa53544e47eff6d1d8f409b62569e3d59fd04
2020-02-18 18:32:56 -08:00
Jeffrey Huang
b41dacd813 Allow statsd to link against libstatslog
Bug: 145923087
Test: m -j
Change-Id: I44f8a24284b4ef8445ebb17edac72c33f72e1e39
2020-02-18 14:52:54 -08:00
TreeHugger Robot
d1c1a97b1b Merge "Turn on new socket protocol" 2020-02-06 05:38:36 +00:00
Muhammad Qureshi
deebbd9382 Turn on new socket protocol
Bug: 148909969
Bug: 143979391
Test: m
Test: fastboot flashall
Test: atest android.cts.statsd.atom.UidAtomTests#testLmkKillOccurred
Test: atest LibStatsPullTests
Test: atest GtsStatsdHostTestCases
Change-Id: I40bf643ac9e10e5b8caeb1d1b1a161d768e3bb99
2020-02-05 17:17:46 -08:00
Tej Singh
2bead545e7 Fix key value pairs logging on old schema
There was an extra level of lists caused by
FrameworkStatsLog.java. This fixes that.

Test: statsd_testdrive 83
Change-Id: I2afe6c739c786f886210f658c8ae8840774f8092
2020-02-05 17:17:16 -08:00
Howard Ro
1f0c52287b Merge "Update libstatssocket/pull API council feedback" 2020-02-05 18:28:12 +00:00
Howard Ro
6269383520 Turn back to the old socket schema
Bug: 148909969
Test: GTS tests pass after this change (they fail otherwise)
Change-Id: Iee8d6506c4fb2fbf3acef474f9ee13df1b764c41
2020-02-04 21:53:40 -08:00
Howard Ro
2688fe61c4 Merge "Turn on new socket schema" 2020-02-04 05:09:57 +00:00
Tej Singh
b26d044a75 Update libstatssocket/pull API council feedback
ALso, removes KeyValuePairs support from libstatssocket

Test: make
Test: atest libstatssocket_test
Test: atest statsd_test
Test: atest LibStatsPullTests
Test: statsd_testdrive 10068
Change-Id: I3a5d0e66f00cea5897e0db33cb769bf6ea67f320
2020-02-03 18:01:31 -08:00
Christine Tsai
f7e56a64e2 Merge "Allow default state, reset state, and nesting for binary states" 2020-02-03 22:46:06 +00:00
Muhammad Qureshi
af77b79ab6 Turn on new socket schema
This switches stats-log-api-gen to use StatsEvent.java and
stats_event.c/StatsLogCompat.cpp in generated StatsLog write methods.

Bug: 143979391
Test: m
Test: fastboot flashall
Test: atest CtsStatsdHostTestCases
Change-Id: I56f58d5545e51a0d7a34ec835110f5bc1b069d57
2020-02-03 02:22:48 +00:00
Automerger Merge Worker
338963acbe Merge "Generate compile-time Q-compatible StatsLog" am: d8a5039db0 am: f4eb2201ae am: 79e6c0d075
Change-Id: I21fed9646eeb8c936584490c252bc020bbe68107
2020-01-31 16:31:25 +00:00
Muhammad Qureshi
bb699b68c0 Generate compile-time Q-compatible StatsLog
Bug: 148388745
Test: m NetworkStack
Test: m NetworkStackNext
Change-Id: Ic37bb67ce625d0a583c424d4775e9f0fa132c418
2020-01-29 14:27:04 -08:00
Muhammad Qureshi
b6b3995f17 Add KeyValuePairs support for modules.
Add KeyValuePairs support to StatsLog autogen for modules.

Bug: 147674597
Test: m
Test: fastboot flashall
Test: adb logcat "*:S statsd:*" and inspect KeyValuePair atoms.
Test: statsd_testdrive 83
Change-Id: Ie40ea52fde494ec11da6b73f430c7a112711b55b
2020-01-28 16:34:22 -08:00
Muhammad Qureshi
b11e00d655 Add flag to stats-log-api-gen for WorkSource.
As part of getting rid of StatsLogInternal and moving every atom to be
logged from a module, this CL addresses 2 issues:
1. Add WorkSource logging to modules.
2. Gate WorkSource logging via a CLI flag to prevent WorkSource methods
from being included in modules that are Mainline modules and can't have
@hide calls.

Bug: 147541828
Test: m
Test: atest android.cts.statsd.atom.UidAtomTests

Change-Id: Icce2f6ed7298be5e5ceebfb05d87e6fa7cd6bdbb
Merged-In: Icce2f6ed7298be5e5ceebfb05d87e6fa7cd6bdbb
2020-01-23 11:39:38 -08:00
Muhammad Qureshi
17c204de33 Make writeKeyValuePairs accept Nullable args
API Review requested to remove unneeded allocations so pass null for
arguments that are empty when writing KeyValuePairs

Bug: 146383532
Test: m
Change-Id: Ic1788f65e8c7f837d6800d59c186519a82882bf7
Merged-In: Ic1788f65e8c7f837d6800d59c186519a82882bf7
(cherry picked from commit 996afa633c)
2020-01-23 11:32:07 -08:00
Muhammad Qureshi
0c8cdf54db Use preview_sdk_int for version code checks
Don't use Build.VERSION.CODENAME to check for R.
Use Build.VERSION.SDK_INT > Q || (Build.VERSION.SDK_INT == Q &&
        Build.VERSION.PREVIEW_SDK_INT > 0)

Bug: 146235828
Test: m DocumentsUIGoogle

Change-Id: I59e327d4e92d41a93b6bea145b8497fd4f85108c
Merged-In: I59e327d4e92d41a93b6bea145b8497fd4f85108c
(cherry picked from commit b6f32d9fe2)
2020-01-23 11:29:21 -08:00
Muhammad Qureshi
ad94425b15 Add flag to stats-log-api-gen for WorkSource.
As part of getting rid of StatsLogInternal and moving every atom to be
logged from a module, this CL addresses 2 issues:
1. Add WorkSource logging to modules.
2. Gate WorkSource logging via a CLI flag to prevent WorkSource methods
from being included in modules that are Mainline modules and can't have
@hide calls.

Bug: 147541828
Test: m
Test: atest android.cts.statsd.atom.UidAtomTests

Change-Id: Icce2f6ed7298be5e5ceebfb05d87e6fa7cd6bdbb
2020-01-17 08:36:52 -08:00
tsaichristine
5adc7e008b Allow default state, reset state, and nesting for binary states
Every state atom can have a customized default state and reset state by
annotating atoms.proto.

Binary state atoms (such as WakelockStateChanged)  can turn on nested
counting by annotating atoms.proto as well.

Generated atoms_info.h before change: https://paste.googleplex.com/4626190063108096
Generated atoms_info.h after change: https://paste.googleplex.com/5410938863747072
Generated atoms_info.cpp before change: https://paste.googleplex.com/5726061016907776
Generated atoms_info.cpp after change: https://paste.googleplex.com/5540983737417728

Test: bit statsd_test:*
Change-Id: I845616f103e013a7927de869b8e8228cfb244090
2020-01-16 17:35:29 -08:00
Muhammad Qureshi
f29d5c88da Merge "Make writeKeyValuePairs accept Nullable args" 2020-01-16 01:25:22 +00:00
Muhammad Qureshi
996afa633c Make writeKeyValuePairs accept Nullable args
API Review requested to remove unneeded allocations so pass null for
arguments that are empty when writing KeyValuePairs

Bug: 146383532
Test: m
Change-Id: Ic1788f65e8c7f837d6800d59c186519a82882bf7
2020-01-15 13:35:41 -08:00
tsaichristine
ed6156442d Allow first uid in attribution chain as primary field
Test: bit statsd_test:*
Bug: 142108433
Change-Id: I6a2f8e6198765d4205dac95e5f54666496b78808
2020-01-13 09:35:38 -08:00
Muhammad Qureshi
6a6bd7ee22 Remove dependency on libutils in native autogen
Right now, Q schema generated code has a dependency on libutils.
Turning on R schema for clients will remove libutils as a required
dependency. Once libutils is removed from clients, turning the flag off
again will be complicated as all clients will need to re-include
libutils. To fix this problem, remove dependency on libutils in Q
schema.

Right now, we only use SystemClock.h from libutils in Q schema autogen.
Use timespec from C standard library instead.

Bug: 146808265
Test: m
Test: flashes successfully
Test: adb logcat "*:S statsd:*"
Test: bit statsd_test:*
Change-Id: I00efeab1be351a95083f3b6e2dbc9067849bee20
Merged-In: I00efeab1be351a95083f3b6e2dbc9067849bee20
2020-01-08 17:08:05 -08:00
Muhammad Qureshi
7182159154 Add --supportQ flag to java StatsLog codegen
Add support to generate Q schema code generation only for modules that
ship to Q.

Bug: 145606209
Test: m
Test: flashes successfully
Test: adb logcat "*:S statsd:*"
Change-Id: I978ac321415f1e8d76e4ec73445894965845aaea
Merged-In: I978ac321415f1e8d76e4ec73445894965845aaea
2020-01-08 17:07:18 -08:00
Muhammad Qureshi
6f65881d62 Support new socket schema in native codegen
New code generation implementation is flag guarded by
STATS_SCHEMA_LEGACY

Support for Q schema can be added by passing --supportQ flag in
stats_log_api_gen. (Only needed for statslog_resolv.cpp)

Q schema is supported through StatsEventCompat.

Generated R schema statslog.h:
https://paste.googleplex.com/4986214782337024

Generated R schema statslog.cpp:
https://paste.googleplex.com/4856851575341056

Generated R schema statslog_resolv.h:
https://paste.googleplex.com/6062978921136128

Generated R schema statslog_resolv.cpp:
https://paste.googleplex.com/4752329251225600

Generated Q schema statslog.h:
https://paste.googleplex.com/4766729873915904

Generated Q schema statslog.cpp:
https://paste.googleplex.com/5018563779756032

Generated Q schema statslog_resolv.h:
https://paste.googleplex.com/5338897498243072

Generated Q schema statslog_resolv.cpp:
https://paste.googleplex.com/5191011011657728

Test: m -j
Test: flashes successfully
Test: adb logcat inspection
Change-Id: I5675a80c03ca3fbd5cd4a02c04a4b9cb89ec32ab
Merged-In: I5675a80c03ca3fbd5cd4a02c04a4b9cb89ec32ab
2020-01-08 17:06:18 -08:00
Muhammad Qureshi
c29064e14c Move statsd metadata from statslog to atoms_info
Clients don't need to know about statsd metadata. Extract out metadata
from statslog.h/cpp into atoms_info.h/cpp which is only used by statsd.

Generated atoms_info.h: https://paste.googleplex.com/6303016724463616
Generated atoms_info.cpp: https://paste.googleplex.com/5717940978581504

Test: m -j
Test: old metadata in statslog matches the metadata in
atoms_info
Test: Flashes successfully
Test: adb logcat "*:S statsd:*"

Change-Id: I56ef3cc4ea1fbd2cd0130d4e9576b242efb9f627
Merged-In: I56ef3cc4ea1fbd2cd0130d4e9576b242efb9f627
2020-01-08 17:05:15 -08:00
Muhammad Qureshi
2a4c0782b5 Fix StatsEvent memory usage for pulled events
Add usePooledBuffer flag to the Builder which determines whether to reuse
the Buffer's byte array in StatsEvent or use a copy.
The build() function also calls release() on the Buffer
if a copy of the Buffer's byte array is passed to StatsEvent.

Also, for pushed events, release the StatsEvent object and consequently,
the Buffer in StatsLog.write(StatsEvent)

Fixes: 145026572
Fixes: 144126444
Test: bit FrameworksCoreTests:android.util.StatsEventTest
Change-Id: I1cdaf0027b69281cb7cb6f3c8ca923d03829b4dd
Merged-In: I1cdaf0027b69281cb7cb6f3c8ca923d03829b4dd
2020-01-08 17:04:27 -08:00
Muhammad Qureshi
01268dec13 Handle nulls in Java autogeneration
Bug: 143979391
Test: m -j
Change-Id: I93146faa1976cac195c2250832d12ee4a09ae969
Merged-In: I93146faa1976cac195c2250832d12ee4a09ae969
2020-01-08 17:02:22 -08:00
Muhammad Qureshi
ef19b08b05 Use StatsEvent in Java autogenerated code
The change is guarded by STATS_SCHEMA_LEGACY flag in the autogenerator
Android.bp.

The flag can only be removed once rest of statsd switches over to the
new socket schema.

The new autogeneration uses StatsEvent for both Mainline module
clients and the default platform client. For Mainline modules, the
autogenerated code uses a runtime dessert version check to use new
socket schema on R+ platforms and the legacy schema on the older
platforms.

The legacy schema is encapsulated in a separate autogenerated inner
class called QLogger.

generated DocumentsStatsLog.java: https://paste.googleplex.com/4665805503463424
generated StatsLogInternal.java: https://paste.googleplex.com/5955095055302656
generated android_util_StatsLogInternal.cpp:
    https://paste.googleplex.com/6737331711115264

Bug: 142811546
Test: m -j && m -j DocumentsUIGoogle
Test: Flashes successfully and events are logged to statsd
Change-Id: I4c804eaf4d5ae78001146c89ebe46dfb0a453853
Merged-In: I4c804eaf4d5ae78001146c89ebe46dfb0a453853
2020-01-08 17:01:43 -08:00
Muhammad Qureshi
b6f32d9fe2 Use preview_sdk_int for version code checks
Don't use Build.VERSION.CODENAME to check for R.
Use Build.VERSION.SDK_INT > Q || (Build.VERSION.SDK_INT == Q &&
        Build.VERSION.PREVIEW_SDK_INT > 0)

Bug: 146235828
Test: m DocumentsUIGoogle

Change-Id: I59e327d4e92d41a93b6bea145b8497fd4f85108c
2020-01-08 16:10:57 -08:00