Commit Graph

135 Commits

Author SHA1 Message Date
Muhammad Qureshi
741d31898a Remove libstatsmetadata
Remove kWhitelistedAtoms usage in MetricsManager and remove:
- atoms_info references from statsd
- libstatsmetadata usages from statsd
- libstatsmetadata library
- atoms_info_writer.h/.cpp
- references to atoms_info in rest of stats-log-api-gen
- allow_from_any_uid annotation

Fixes: 147600720
Fixes: 154856835
Fixes: 148993016

Test: m stats-log-api-gen
Test: m stats-log-api-gen-test &&
out/host/linux-x86/nativetest/stats-log-api-gen-test/stats-log-api-gen-test
Test: m statsd

Change-Id: I2e0579609ff257da934b95cdda397f3ca0ffa1f0
2020-06-18 18:59:12 -07:00
Ruchir Rastogi
c480dd54ba Use proper platform version checks (apigen)
Test: m statslog-docsui-java-gen
Bug: 146235828
Change-Id: I5b1a2e0fa356368b91548bd050ec2c493b72f37c
2020-05-07 12:32:54 -07:00
Muhammad Qureshi
e3ef95cf13 Merge "Remove kMaxPushedAtomId from atoms_info." into rvc-dev 2020-04-18 00:13:52 +00:00
Muhammad Qureshi
cdf291d297 Remove kMaxPushedAtomId from atoms_info.
Hardcode kMaxPushedAtomId in StatsdStats.

Bug: 154164020
Test: bit statsd_test:*
Change-Id: Ibca731d9783686a9e3caf0ad580e1d62038d68e2
2020-04-17 10:10:17 -07:00
Muhammad Qureshi
7fee54bb27 Remove kAtomsWithAttributionChain from atoms_info
Use LogEvent::getAttributionChainIndex() to check presence of
attribution chains

Bug: 150414252
Test: bit statsd_test:*
Change-Id: Ic798d3b182c2b5aaf73444f2796d539cd3f7bf40
2020-04-16 21:51:44 -07:00
TreeHugger Robot
cbd9f03e82 Merge "Remove kTruncatingTimestampAtomBlackList" into rvc-dev 2020-04-16 13:07:00 +00:00
Muhammad Qureshi
a7de00022b Remove kTruncatingTimestampAtomBlackList
Use shouldTruncateTimestamp flag in LogEvent instead of the blacklist
to determine whether timestamp should be truncated.

Bug: 150414373
Test: bit statsd_test:*
Change-Id: I4e21ae4909484dd898b9fcd3ce7a018f5e9d982f
2020-04-15 11:34:35 -07:00
Muhammad Qureshi
bfc4bdb594 Remove kStateAtomsInfo from atoms_info.
Use the annotation information in LogEvent/FieldValue instead.

- Initialize nested to true in FieldValue.mAnnotations

- Add filterPrimaryKey function to HashableDimensionKey for populating a
HashableDimensionKey with all FieldValues that have the primary key
annotation set.

- Create StateTrackers without checking if atom is a state atom. (We
  can't do this check anymore)

- Remove mAtomId, mStateField, mNested, mPrimaryFields, mDefaultState,
    mResetState members from StateTracker. Use the information passed in
    LogEvent in onLogEvent instead.

- Update tests to log annotations when logging events.

- Remote kStateAtomsFieldOptions from atoms_info.

- Make MetricProducer::mStateGroupMap const

- Rename handlePartialReset to clearStateForPrimaryKey

- Store "default" states in mStateMap. An entry in mStateMap means the
state is not kStateUnknown. Before, an entry in mStateMap meant the
state is not the "default" state.

- Consolidate all state change logic in updateStateForPrimaryKey()
    - remote StateTracker::updateState
    - handleReset and clearStateForPrimaryKey call
    updateStateForPrimaryKey for resetting and clearing of states
    respectively.

- Create a helper method for notifying StateTracker listeners

- Make StateManager::registerListener void

Bug: 151110842
Test: bit statsd_test:*
Change-Id: Ifb8371b213a178fcccaa484086fbdd283dbaec49
2020-04-14 17:22:15 -07:00
Ruchir Rastogi
ffa34f05cf Remove kAtomsWithUidField from atoms_info
Instead of using the map from atoms_info, we now receive uid information
in the form of StatsEvent annotations. The isUid annotation is exposed
from both LogEvent and FieldValue.

Test: bit statsd_test:*
Test: atest GtsStatsdHostTestCases
Bug: 150414601
Change-Id: Iebbe4ce5668de1ab91485daa1be9197cde6e8309
2020-04-08 19:09:33 -07:00
Muhammad Qureshi
79e10620ee Initialize nested to true
This fixes random nested values in atoms_info.

Bug: 153119080
Test: m libstatsmetadata
Change-Id: I1c1900dae1701b28a8db6ee257f5730e71e574eb
2020-04-07 07:23:21 -07:00
Muhammad Qureshi
3f9c330eef Make state_option top-level boolean annotations
- Add AnnotationId enum stats-log-api-gen
- Rename RESET_STATE annotation to TRIGGER_STATE_RESET. Also rename the proto annotation.

Bug: 152412767
Test: stats-log-api-gen-test
Test: m statslog-framework-java-gen
Change-Id: I2c12ea0b9222ef28b8cc11ea36b272e518a96259
2020-04-03 17:00:55 -07:00
Muhammad Qureshi
4b141386b0 Use constant names instead of literals
Use constant names instead of literals when writing annotations to
increase readability.

FrameworkStatsLog.java: https://paste.googleplex.com/4789132691767296
statslog.h: https://paste.googleplex.com/4986214782337024
statslog.cpp: https://paste.googleplex.com/4856851575341056

Bug: 151786433
Test: m statslog-framework-java-gen
Test: m libstatslog
Change-Id: I47111fe02319bebf41f429642fb675f7a856710b
2020-03-31 17:52:07 -07:00
Muhammad Qureshi
23daf2656c Only send reset state annotation when needed
Only send reset state annotation when reset state occurs.

Bug: 151776731
Test: m libstatslog

Change-Id: I6f6b4d784d3741c0059085421565eba81db5527c
2020-03-27 02:23:38 -07:00
Muhammad Qureshi
c6c38632af Group annotations for the same atom id
Group annotations for the same atom id inside one if-block in generated
code.

Use shared_ptr to store AtomDecls in multiple data structures.

Store a mapping of field numbers to atoms that have annotations at
corresponding field numbers in Collation.h

Bug: 151744250
Test: stats-log-api-gen-test
Test: m stats-log-api-gen
Test: m libstatsmetadata
Test: m statslog-framework-java-gen
Test: m libstatslog

Change-Id: I874696cfb5c27141017b4293bec809ab510ceb98
2020-03-26 14:51:28 -07:00
Muhammad Qureshi
a345af9b40 Run clang-format in stats_log_api_gen
Bug: 152253223
Test: m stats-log-api-gen
Test: stats-log-api-gen-test
Change-Id: Iad76baea8cc9ae7b3978f1133ab94d8e96bbbafe
2020-03-24 17:35:08 -07:00
Muhammad Qureshi
9b995809fb Add truncate_timestamp annotation
Mark privacy-sensitive atoms with truncate_timestamp annotation.

Factor out annotation collation to a helper method.

Add truncate_timestamp annotation support in stats-log-api-gen.

Add writeAnnotation* calls in Java and native generated code for
atom id annotations.

TODO: remove kTruncatingTimestampAtoms from atoms_info.
TODO: use truncate_timestamp annotation inside statsd.

Bug: 151111680
Test: stats-log-api-gen-test
Test: m statslog-framework-java-gen
Test: m libstatsmetadata

Change-Id: I3db5f4ffbf959bd36c62f890cc88606912798d40
2020-03-24 16:49:36 -07:00
Muhammad Qureshi
f77913fbc6 Remove kBytesFieldAtoms from atoms_info
Bug: 150415347
Test: m
Test: bit statsd_test:*
Change-Id: I2e55b445121a39e3bf12164cd9123561e8b5bd4c
2020-03-20 10:55:46 -07:00
Muhammad Qureshi
29613e8ff8 Merge "StatsEvent annotations java autogen" into rvc-dev 2020-03-19 17:20:19 +00:00
Muhammad Qureshi
38bdeb5977 Merge "Write annotations in native generated code." into rvc-dev 2020-03-19 17:19:50 +00:00
Muhammad Qureshi
8a69088a36 Merge "Store annotation during collation" into rvc-dev 2020-03-19 17:19:27 +00:00
Jeffrey Huang
f960305a31 StatsEvent annotations java autogen
Autogenerate annotations for atoms logged in java.

generated FrameworkStatsLog.java:
https://paste.googleplex.com/4789132691767296

Bug: 151102209
Test: m stats-log-api-gen
Test: m statslog-framework-java-gen
Change-Id: I50b01feaacfcb8ad5d0236432ceb0e97c8c907ab
2020-03-17 22:01:58 -07:00
Muhammad Qureshi
a31a6229dd Write annotations in native generated code.
Generated statslog.cpp: https://paste.googleplex.com/6345695252447232

Bug: 151101995
Test: m stats-log-api-gen
Test: m libstatslog
Change-Id: I745a9beb7be07a1890f811256c51627e6c5976dc
2020-03-17 21:57:35 -07:00
Muhammad Qureshi
b13a3214a1 Store annotation during collation
- Only collate atoms in the specified module.

- Replace signature_to_modules with signatureInfoMap. This maps each
signature to another map keyed by field number and whose values are vector of
annotations.

Bug: 151102006
Test: stats-log-api-gen-test
Change-Id: I25bbe4883c8f7f86a06d04d27cd425367b6d65a0
2020-03-17 20:19:04 -07:00
Jeffrey Huang
74fc4354b5 Remove apex_available from libstatslog
Test: m -j
Bug: 149781190
Change-Id: I765852b1719a3056c06a0965ea6a05a8e8410693
2020-03-10 15:14:27 -07:00
Muhammad Qureshi
d7944d6573 Merge "Remove libstatslog from libstatsmetadata" into rvc-dev 2020-03-06 08:04:46 +00:00
Muhammad Qureshi
513453c253 Merge "Support multiple module annotations per atom" into rvc-dev 2020-03-06 08:04:09 +00:00
Muhammad Qureshi
9e0f728fe4 Remove libstatslog from libstatsmetadata
Instead of relying on the constants in libstatslog, hardcode them in
atoms_info to get rid of the dependency on libstatslog

New generated atoms_info.cpp:
https://paste.googleplex.com/5779947622760448

Bug: 150417465
Test: m libstatsmetadata
Change-Id: I954c963f1883f889053b63d308c648548de71e56
2020-03-05 17:20:58 -08:00
Muhammad Qureshi
f8460f70d0 Support multiple module annotations per atom
Bug: 150864291
Test: m stats-log-api-gen-test &&
out/host/linux-x86/nativetest/stats-log-api-gen-test/stats-log-api-gen-test

Change-Id: Ibf8869078c6f1eae13e687f0196d6ed91ec35c64
2020-03-05 14:52:05 -08:00
Muhammad Qureshi
4341dbb253 Merge "Delete unused code in stats-log-api-gen" into rvc-dev 2020-03-05 21:41:17 +00:00
Muhammad Qureshi
4ef3c3bd26 Delete unused code in stats-log-api-gen
Bug: 145100015
Test: m
Change-Id: Idbbe35fbce0d751a5973128406596678e1ead5d3
2020-03-05 10:21:36 -08:00
Tej Singh
c88fbf1e1c Fail if there are repeated fields in atoms.proto
Repeated fields arent allowed outside of attribution node and key value
pairs, so we should fail to build if someone attempts to add one.

Bug: 149956370
Test: atest stats-log-api-gen-test
Change-Id: I0caa6fcf74dabe8e962360ecccfd3a87fcaa4909
Merged-In: I0caa6fcf74dabe8e962360ecccfd3a87fcaa4909
2020-03-05 01:06:57 -08:00
Tej Singh
eb7d8f4675 Make libstatssocket shared
Also make private versions of the libraries available to link statically
for tests. We should try to figure out a better long term solution.

Test: m
Test: bit statsd_test:*
Test: atest LibStatsPullTests
Test: bit libstatssocket_test:*
Test: atest GtsStatsdHostTestCases
Bug: 149340100

Change-Id: I05b91efab2a657aec75d436575aff4373f86ed3f
2020-02-27 11:38:41 -08:00
Tej Singh
d87589549f Turn off cpp aidlgen, link libstatssocket as share
Test: make
Change-Id: I052fa53544e47eff6d1d8f409b62569e3d59fd04
2020-02-18 18:32:56 -08:00
Jeffrey Huang
b41dacd813 Allow statsd to link against libstatslog
Bug: 145923087
Test: m -j
Change-Id: I44f8a24284b4ef8445ebb17edac72c33f72e1e39
2020-02-18 14:52:54 -08:00
TreeHugger Robot
d1c1a97b1b Merge "Turn on new socket protocol" 2020-02-06 05:38:36 +00:00
Muhammad Qureshi
deebbd9382 Turn on new socket protocol
Bug: 148909969
Bug: 143979391
Test: m
Test: fastboot flashall
Test: atest android.cts.statsd.atom.UidAtomTests#testLmkKillOccurred
Test: atest LibStatsPullTests
Test: atest GtsStatsdHostTestCases
Change-Id: I40bf643ac9e10e5b8caeb1d1b1a161d768e3bb99
2020-02-05 17:17:46 -08:00
Tej Singh
2bead545e7 Fix key value pairs logging on old schema
There was an extra level of lists caused by
FrameworkStatsLog.java. This fixes that.

Test: statsd_testdrive 83
Change-Id: I2afe6c739c786f886210f658c8ae8840774f8092
2020-02-05 17:17:16 -08:00
Howard Ro
1f0c52287b Merge "Update libstatssocket/pull API council feedback" 2020-02-05 18:28:12 +00:00
Howard Ro
6269383520 Turn back to the old socket schema
Bug: 148909969
Test: GTS tests pass after this change (they fail otherwise)
Change-Id: Iee8d6506c4fb2fbf3acef474f9ee13df1b764c41
2020-02-04 21:53:40 -08:00
Howard Ro
2688fe61c4 Merge "Turn on new socket schema" 2020-02-04 05:09:57 +00:00
Tej Singh
b26d044a75 Update libstatssocket/pull API council feedback
ALso, removes KeyValuePairs support from libstatssocket

Test: make
Test: atest libstatssocket_test
Test: atest statsd_test
Test: atest LibStatsPullTests
Test: statsd_testdrive 10068
Change-Id: I3a5d0e66f00cea5897e0db33cb769bf6ea67f320
2020-02-03 18:01:31 -08:00
Christine Tsai
f7e56a64e2 Merge "Allow default state, reset state, and nesting for binary states" 2020-02-03 22:46:06 +00:00
Muhammad Qureshi
af77b79ab6 Turn on new socket schema
This switches stats-log-api-gen to use StatsEvent.java and
stats_event.c/StatsLogCompat.cpp in generated StatsLog write methods.

Bug: 143979391
Test: m
Test: fastboot flashall
Test: atest CtsStatsdHostTestCases
Change-Id: I56f58d5545e51a0d7a34ec835110f5bc1b069d57
2020-02-03 02:22:48 +00:00
Automerger Merge Worker
338963acbe Merge "Generate compile-time Q-compatible StatsLog" am: d8a5039db0 am: f4eb2201ae am: 79e6c0d075
Change-Id: I21fed9646eeb8c936584490c252bc020bbe68107
2020-01-31 16:31:25 +00:00
Muhammad Qureshi
bb699b68c0 Generate compile-time Q-compatible StatsLog
Bug: 148388745
Test: m NetworkStack
Test: m NetworkStackNext
Change-Id: Ic37bb67ce625d0a583c424d4775e9f0fa132c418
2020-01-29 14:27:04 -08:00
Muhammad Qureshi
b6b3995f17 Add KeyValuePairs support for modules.
Add KeyValuePairs support to StatsLog autogen for modules.

Bug: 147674597
Test: m
Test: fastboot flashall
Test: adb logcat "*:S statsd:*" and inspect KeyValuePair atoms.
Test: statsd_testdrive 83
Change-Id: Ie40ea52fde494ec11da6b73f430c7a112711b55b
2020-01-28 16:34:22 -08:00
Muhammad Qureshi
b11e00d655 Add flag to stats-log-api-gen for WorkSource.
As part of getting rid of StatsLogInternal and moving every atom to be
logged from a module, this CL addresses 2 issues:
1. Add WorkSource logging to modules.
2. Gate WorkSource logging via a CLI flag to prevent WorkSource methods
from being included in modules that are Mainline modules and can't have
@hide calls.

Bug: 147541828
Test: m
Test: atest android.cts.statsd.atom.UidAtomTests

Change-Id: Icce2f6ed7298be5e5ceebfb05d87e6fa7cd6bdbb
Merged-In: Icce2f6ed7298be5e5ceebfb05d87e6fa7cd6bdbb
2020-01-23 11:39:38 -08:00
Muhammad Qureshi
17c204de33 Make writeKeyValuePairs accept Nullable args
API Review requested to remove unneeded allocations so pass null for
arguments that are empty when writing KeyValuePairs

Bug: 146383532
Test: m
Change-Id: Ic1788f65e8c7f837d6800d59c186519a82882bf7
Merged-In: Ic1788f65e8c7f837d6800d59c186519a82882bf7
(cherry picked from commit 996afa633c)
2020-01-23 11:32:07 -08:00
Muhammad Qureshi
0c8cdf54db Use preview_sdk_int for version code checks
Don't use Build.VERSION.CODENAME to check for R.
Use Build.VERSION.SDK_INT > Q || (Build.VERSION.SDK_INT == Q &&
        Build.VERSION.PREVIEW_SDK_INT > 0)

Bug: 146235828
Test: m DocumentsUIGoogle

Change-Id: I59e327d4e92d41a93b6bea145b8497fd4f85108c
Merged-In: I59e327d4e92d41a93b6bea145b8497fd4f85108c
(cherry picked from commit b6f32d9fe2)
2020-01-23 11:29:21 -08:00
Muhammad Qureshi
ad94425b15 Add flag to stats-log-api-gen for WorkSource.
As part of getting rid of StatsLogInternal and moving every atom to be
logged from a module, this CL addresses 2 issues:
1. Add WorkSource logging to modules.
2. Gate WorkSource logging via a CLI flag to prevent WorkSource methods
from being included in modules that are Mainline modules and can't have
@hide calls.

Bug: 147541828
Test: m
Test: atest android.cts.statsd.atom.UidAtomTests

Change-Id: Icce2f6ed7298be5e5ceebfb05d87e6fa7cd6bdbb
2020-01-17 08:36:52 -08:00