1. StatsdStats does not use the proto object in memory anymore.
2. lite_static -> lite
3. don't use sstream
Bug: 72129300
Test: statsd_test
Change-Id: I8a5adaf222d4d5034e8bf115215fb6dd5f042cac
+ A state change atom can have one exclusive state field, and any
number of primary key fields.
When there is primary key in the atom, it means the state belongs to the primary key.
For example,
message UidProcessStateChanged {
optional int32 uid = 1 [(stateFieldOption).option = PRIMARY];
optional android.app.ProcessStateEnum state = 2 [(stateFieldOption).option = EXCLUSIVE];
}
When there is no primary key fields in the atom, the state is global.
For example,
message ScreenStateChanged {
optional android.view.DisplayStateEnum state = 1 [(stateFieldOption).option = EXCLUSIVE];
}
+ The annotation is consumed by stats_log_api_gen to generate a static map from the state
atoms to its primary fields, and exclusive fields
+ stats_log.proto is splitted into 2 proto files, because statsd needs proto lite, and c++
lite proto library cannot properly ignore the field options which requires full proto.
This CL doesn't change any logic in the statsd yet. A separate CL will use the field option
information to correctly track the state.
Test: added unit tests in stats_log_api_gen_test. and statsd_test pases.
Change-Id: I9e8a979fe81ba60efd4d854bb7087ce4b2b147ec
Bug: 73264895
Test: statsd_test and locally built statsd with
LOCAL_CLANG:=true
LOCAL_SANITIZE:=address
Change-Id: Ifb8e04c5b4908446f553169846a3226db6e02f54
The key change is to revamp how we parse/store/match a log event, especially how we match repeated
field and attribution nodes, and how we construct dimensions and compare them.
+ We use a integer to encode the field of a log element. And also encode the FieldMatcher into an
integer and a bit mask. The log matching becomes 2 integer operations.
+ Dimension is stored as encoded field and value pair. Checking if 2 dimensions are equal is then
becoming checking if the underlying integers are equal. The integers are stored contiguously
in memory, so it's much faster than previous tree structure.
Start review from FieldValue.h
Test: statsd_test + new unit tests
Bug: 72659059
Change-Id: Iec8daeacdd3f39ab297c10ab9cd7b710a9c42e86
Sets DEBUG to false everywhere and replaces all ALOGD with VLOG so they
do not print with DEBUG false. Leaves all ALOGI, ALOGW and ALOGE as is.
Test: ran all CTS tests and checked "adb logcat -s statsd" to make sure
it wasn't spammy
Change-Id: Iaa8eb3a0a63723ffe40f94f2815f94df877fd432
1/ Avoid unnecessary field/dimension proto construction.
2/ use unordered_map for slicing.
3/ Use dimension fields to compare dimension keys.
Test: all statsd tests passed.
Change-Id: I2f74f78589b7f6ecd0803a2ead822b8d0399f334
+ We should benchmark the core functions in the critical path.
1. LogEvent parsing
2. Log matching
3. Event processing in various metrics
....
+ Using microbenchmark, we can quantify the optimization we do in the future.
---------------------------------------------------------
Benchmark Time CPU Iterations
---------------------------------------------------------
BM_LogEventCreation 4772 ns 4705 ns 145738
Test: make -j64 statsd_benchmark
Bug: 72242322
Change-Id: I43aa704ffdc7cd21e02ef9038eff66ca7022dbfb
+ Add log source whitelist in StatsdConfig
+ Some changes in UidMap API. Listener needs to be wp instead of sp.
+ Update dogfood app config to have log source
+ Increase the stats service thread pool size to 10 (9+1).
TODO: add unit tests(b/70805664). This unit test takes some time to write.
Test: statsd_test & manual
Change-Id: I129b1cc13db5114db7417580962bd7cc4438519d
bug fix: GaugeMetricProducer now works better with pulled events.
unit test also includes GaugeMetricProducer_test
Test: unit test
Change-Id: Ic60f09342d14cfb107be2130d445b323a56909e0
My previous understanding of the liblog api was wrong. Luckily it
is caught in time. Also added a memory profiling tool in another CL.
Test: statsd_test
adb shell dumpsys meminfo
adb shell cmd stats meminfo
Change-Id: I742af2a38e52e35edf0d4c77ccc35ede7019a73b
+ Changed how we construct LogEvent, now it's based on the context from log_msg
without making a copy of the list.
+ All stats logs now have the same event tag, the atom id is the first elem in the log.
Test: statsd_test
Change-Id: I4419380f2ee1c2b2155d427b9f2adb78883b337f
And other miscellaneous fixes.
+ clang-format
+ 2 bug fixes, one in dump-report command, one in ResourcePowerManagerPuller
Test: statsd_test
Change-Id: Ibd164d948ad62adcc529d813df1210781e38be47
Previously when fixing LogEvent missing strings because of pointer
lifetime issues, I had the constructor for LogEvent pass log_msg by
value. This fixes log_msg to pass by reference again so that we avoid
wasting memory.
Test: Checked statsd unit-tests still pass.
Change-Id: I94ac4d1bed19fec19d44e27fe94e07ab30383363
Note:
This is for value metric. The default operations is sum the diffs.
The test uses kernel wake lock, which also needs dimension by kernel
wake lock name.
The test is a bit cumbersome as it needs StatsCompanionService to do
the alarm, which is not exact alarm.
The internal state of a slice of bucket would look something like this:
4:ipc0000005e_727_android.hardwar
0 0
4:SensorService_wakelock
40 64
4:ipc0000005c_727_android.hardwar
...
Test: manual test on device.
Change-Id: I2ed0ac7d3c5fcba8b7611d46f38a38ffd8bdc92a
+ Started to use ProtoOutputStream in EventMetricProducer.
[TODO]: We need to auto-generate fieldIds for StatsLogReport, XXXMetricData, etc.
[TODO]: We need to add Enum type to liblog, otherwise we cannot reconstruct a proto containing
an enum
+ Some refactor in metric initialization code. There are still boiler plate code, because Metrics
are similar but with subtle differences.
Test: statsd_test
Change-Id: Id7e3212566249a8139b9680f04238c455d50c1b8
Previously, pulled data was returned as a string. We instead
return the data as an array of StatsLogEventWrapper, which encodes
using the binary-encoded format liblog uses. StatsD uses the same
parsing as for pushed events to convert these. This CL also fixes
the parsing of log_msg since the strings were previously emptied
before we had a chance to read the values.
Note that the cpp-aidl can't support List of Parcelable, so we
have to return the results as an array.
Test: Manual using the new command in StatsService to print results.
Also created a new unit-test by creating a dummy pull code of -1,
but this test is deleted since it required creating a fake output in
StatsCompanionService.
Change-Id: I1cfb9ea081a59292a60e934e8527adc40982ed80
Now we support following metrics:
<Duration> of [app holding a wake lock], while [*this app*] is [in background] [AND] [screen is off]
[Slice] the output by [app name, wake lock name], with bucket size [30sec]
+ Also added onDumpReport() api in MetricsManager, it can be called from client to fetch the data from
statsd
+ Also added command line tool to dump the StatsLogReport from all metrics for debugging.
+ Synced proto from google3. with a pending cl (cr/172359050)
TODO: We need to add tons of tests to test the Metrics. I will work on it after this CL so people
can be unblocked.
I locally test the duration metric with wake lock with an app that generates StatsLog events.
Test: statsd_test
and manual test, and run:
adb shell cmd stats dump-report
We have a default config, which contains a metrics to count PROCESS_START event sliced by
package name.
Change-Id: I4838cc6cf025c143b7e84f43040703a78121fd25
It stores all of the parsed fields in a single vector, and
provides accessor methods to get at fields by index with
their correct type.
Test: statsd_test
Change-Id: I4fa94e4ce52db3ac87f19b62f9c85398de6e8145
