Commit Graph

5563 Commits

Author SHA1 Message Date
Zhen Sun
e31edc21a1 Merge "Add atom of TvSettings and its enum" into rvc-dev 2020-03-31 23:24:21 +00:00
TreeHugger Robot
32f3bcfcd2 Merge "Log also the lang id model name that is involved in events" into rvc-dev 2020-03-31 20:04:47 +00:00
TreeHugger Robot
533a99f3b0 Merge "Guardrail for pull cooldown/timeout" into rvc-dev 2020-03-31 18:25:56 +00:00
Tony Mak
2c0049521b Log also the lang id model name that is involved in events
Possible values:
und_v0, und_v1

What is it:
The model version of the LangId model file.

Who determines this value?
The textclassifier team. The model version is burnt to the model file.

Test: m
Test: atest TextClassifierServiceTest

Bug: 152490822

Change-Id: I0058536ee4e5d8e9c14e100b61683c67af28981d
2020-03-31 17:12:17 +00:00
Tej Singh
abc1b8de87 Guardrail for pull cooldown/timeout
Enforce a 1 second minimum cooldown and a 10 second maximum timeout on
pullers.

Test: bit statsd_test:*
Test: atest LibStatsPullTests
Test: atest GtsStatsdHostTestCases
Bug: 152543610
Change-Id: Ida0047235a7e56d7b700f1a7ab5cfbc2a147caeb
2020-03-30 22:26:46 -07:00
Zhen Sun
0bec8d43a1 Add atom of TvSettings and its enum
The enums correspond to pages and entries of TvSettings. Some enums that
are related to unreleased features are not included in this CL but will
be added when this CL is cherrypicked into upstream non-public
TV-specific branches.

The atom definition is reviewed and approved by Metrics council.

Metrics council review bug: 151768952
Bug: 150979527
Test: the Android tree still builds normally

Change-Id: I4258312c19aa4f2f0cd178bc5b5dc53e52c6c737
Merged-In: I4258312c19aa4f2f0cd178bc5b5dc53e52c6c737
2020-03-30 23:39:19 +00:00
Mike Ma
b98050fc04 Fix a race condition on ReportHandler::mBatch
Hold mLock when accessing mBatch and mHandlerLooper in ReportHandler.

Fixes: 147326028
Test: Take an incident report. Verify no race condition.
Change-Id: I9d6da0067731f253532f60e5abb12dfb238b5411
2020-03-30 14:44:42 -07:00
Ryan Mitchell
fa0f02a91c Merge "Consider target path when verifyIdmap." into rvc-dev 2020-03-30 15:45:47 +00:00
Dichen Zhang
f24f832fd7 Merge "Fix command injection on screencap" into rvc-dev 2020-03-28 06:34:31 +00:00
Tej Singh
f4e17e6639 LibstatsPullTest, Testdrive updates for sandboxing
Test: atest LibStatsPullTests
Test: statsd_testdrive 10051 10004 10055
Test: adb shell cmd stats pull-source 10051 AID_STATSD
Test: adb shell cmd stats pull-source 10055 AID_GPU_SERVICE
Test: adb shell cmd stats pull-source 10004
Bug: 151642352
Change-Id: I3b104cb2f2427b4558d5175dfeb3bb2c80012d0e
2020-03-27 19:02:10 -07:00
Tej Singh
3be093b518 Uid Sandboxing of Pullers
Overall flow of implementation:
1. parsing the config in MetricsManager to store the uids per atom. It
follows the mAllowedLogSources logic very closely
2. MetricsManager register itself as a PullUidProvider with the
PullerManager.
3. Metrics pass the config key when pulling (for both registering
receivers and normal pulls) , and the puller manager gets
the allowed uids from the PullUidProvider for that config.
4. PullerManager keys receivers by <atomId, configKey> so that it can
look up the uids for that atom using the PullUidProvider as well.
5. Added shell subscriber support. Hardcode a default of AID_SYSTEM for
them and also allow packages per atom. This involved adding a second
interface to Pull that simply accepts the uids, since I didnt want to
make the ShellSubscriber a PullUidProvider as well.
6. Change adb shell cmd stats pull-source to allow users to specify a
package. Default to AID_SYSTEM as well.

Notes:
The feature is flagged off right now, since configs do not pass in the
desired package. Another approach could be to hardcode in the current
mapping, but that doesn't work for OEM pulled atoms.

Test: m statsd
Test: bit statsd_test:* with useUids = false
Test: bit statsd_test:* with useUids = true
Bug: 144099783
Bug: 151978258

Change-Id: I4a7481d7402a52b9beb4ea28b102803f9e50e79f
2020-03-27 18:07:24 -07:00
Chen Chen
0941d63aee Merge "Bluetooth Metrics: Add metric_id field for Bluetooth metrics to proto. The incremental id is neither a device id nor a pseudo name and therefore PII free." into rvc-dev 2020-03-28 00:54:26 +00:00
TreeHugger Robot
4efa78539d Merge "Fix atom id annotation parsing" into rvc-dev 2020-03-27 22:02:38 +00:00
Muhammad Qureshi
f2012da846 Fix atom id annotation parsing
Bug: 152633444
Test: m -j cts && m -j CtsStatsdHostTestCases && cts-tradefed run
cts-dev -m CtsStatsdHostTestCases -t
android.cts.statsd.atom.UidAtomTests#testAudioState
Test: bit statsd_test:*

Change-Id: I167012330e431b38b86f190ca6b5d4f90995172f
2020-03-27 12:38:00 -07:00
hg.choi
a3a68132b5 Consider target path when verifyIdmap.
Updating new version package with changing resource id,
Overlay idmap still retain same resource id and target path of prebuilt package.
Consequentially overlay can not apply to new update resource id.

As is : VerifyIdmap only verify uptodate of idmap.
To be : Verify change of target path.

Bug: 147794117
Bug: 150877400
Test: idmap2_tests

Change-Id: I2ed6b5ba49aeb1b24625866daefea56c9766baef
2020-03-26 12:35:58 -07:00
Maryam Karimzadehgan
4f9ab82c08 Proto changes and binary for adding more logs to the Back
Gesture. Design document:
go/logging-back-gesture.

Test: manual
Bug: 148104018
Change-Id: I63b745f30b7581306150d2d7ab1fe3cccfe5147f
2020-03-26 17:01:22 +00:00
Christine Tsai
0810d57f95 Merge "Update unit tests to use new socket schema" into rvc-dev 2020-03-26 16:42:44 +00:00
TreeHugger Robot
cfeb0d8323 Merge "Expand "extras" support in content tool." into rvc-dev 2020-03-26 06:29:59 +00:00
Jeff Sharkey
547bf5487d Expand "extras" support in content tool.
We recently added a new "Bundle extras" argument to insert(),
update(), delete(), and query(), so we should ensure that developers
working from the shell can populate these extras.

Uses binding logic that was already used for call() extras.  Fixes
parsing logic to support keys or values that have escaped colons,
and fix bug to always parse as requested data type.

Bug: 150982673
Test: adb shell content query
    --uri content://media/external/images/media/
    --extra android\\\\:query-arg-match-trashed:i:3
Change-Id: I40a9a94e85e175e298b01688d3b075a89b2e3954
2020-03-25 18:35:10 -06:00
Dichen Zhang
eab2d8c046 Fix command injection on screencap
There is a potential injection by using screencap in case of user handled parameters.
"dumpstate" command launches "screencap", when "-p" is argument is set. At that moment, content of "-o" parameter generates a path with ".png" extension to define "screencap" argument.
"dumpstate" is often run as a service with "root" privileged such as defined in "dumpstate.rc". For instance "bugreportz" call "ctl.start" property with "dumpstatez".
Launching "dumpstate" with "-p" option and a user input as "-o" would result in a root command execution. SE Linux might protect part of this attack.

Cherry-pick from ag/10651695 with fix ag/10700515

Bug: 123230379
Test: please see commands #4 and #5
Change-Id: Icd88cdf4af153e07addb4449cdb117b1a3c881d3
2020-03-26 00:09:21 +00:00
tsaichristine
d5800b208e Update unit tests to use new socket schema
I forgot to update Gauge and EventMetricProducer unit test files to use
the new socket schema.

The EventMetric tests were also incomplete so I added test code to
output the dump report and check its contents.

Test: bit statsd_test:*
Bug: 149590301 && 110561136

Change-Id: I685688285d9890cf7c5bd3ffc43f13cbbb8f93de
2020-03-25 16:36:18 -07:00
Jeffrey Huang
1bd7fe106a Merge changes I7127dd7d,I84da0cd9 into rvc-dev
* changes:
  Add mAlertToAnomalyTrackerMap to MetricsManager
  Add StatsMetadataList proto
2020-03-25 22:31:02 +00:00
Raff Tsai
9a54e23d8e Merge "Add SettingSnapshot atom" into rvc-dev 2020-03-25 20:42:07 +00:00
Muhammad Qureshi
6fe5fd6e2e Merge changes Iad76baea,I3db5f4ff into rvc-dev
* changes:
  Run clang-format in stats_log_api_gen
  Add truncate_timestamp annotation
2020-03-25 17:38:05 +00:00
TreeHugger Robot
8910bf95c6 Merge "Fix ValueMetricProducer_test on hwasan" into rvc-dev 2020-03-25 17:32:55 +00:00
Raff Tsai
ea4db183fe Add SettingSnapshot atom
Bug: 142941011
Test: m Settings
Change-Id: I135a8ee1c0f8527d539c3613425e7e4c082ec099
2020-03-25 17:08:26 +08:00
Tej Singh
547f46331c Fix ValueMetricProducer_test on hwasan
TestUseZeroDefaultBaseWithPullFailures was failing on hwasan builds
because the test had a reference to an Interval object that was
deleted. The test tried to get another Interval object and assign it to
the reference. However, what this really did is change the Interval
object that the reference referred to. Since that Interval object had
been deleted, hwasan failed the test.

This fixes the issue. Also clean up some usages of auto to make
the test more readable.

Test: bit statsd_test:* on blueline_hwasan
Bug: 152354576
Change-Id: I392740770715391c727d190b1ed8ba22cc01210f
2020-03-25 08:10:35 +00:00
Jeffrey Huang
38d702697d Add mAlertToAnomalyTrackerMap to MetricsManager
Bug: 148280505
Test: bit statsd_test:*
Change-Id: I7127dd7dac9f6c4f3d415534c339aaba28fa41f3
2020-03-24 17:22:32 -07:00
Muhammad Qureshi
9b995809fb Add truncate_timestamp annotation
Mark privacy-sensitive atoms with truncate_timestamp annotation.

Factor out annotation collation to a helper method.

Add truncate_timestamp annotation support in stats-log-api-gen.

Add writeAnnotation* calls in Java and native generated code for
atom id annotations.

TODO: remove kTruncatingTimestampAtoms from atoms_info.
TODO: use truncate_timestamp annotation inside statsd.

Bug: 151111680
Test: stats-log-api-gen-test
Test: m statslog-framework-java-gen
Test: m libstatsmetadata

Change-Id: I3db5f4ffbf959bd36c62f890cc88606912798d40
2020-03-24 16:49:36 -07:00
tsaichristine
63143251fe Update the cc_benchmark to use the new socket schema
Also removed unused CreateEvent helper functions in metric_util.cpp

Test: bit statsd_benchmark:*
Bug: 149794614
Change-Id: I14bcc254c44773eb42f749305beadaaf23a19b2b
2020-03-24 16:03:48 -07:00
Christine Tsai
4aa7af0a8f Merge "(Part 3) Use new socket schema with statsd tests" into rvc-dev 2020-03-24 20:55:26 +00:00
Mike Ma
06b3aaee39 Merge "Optimize memory usage in incidentd" into rvc-dev 2020-03-24 20:27:41 +00:00
Jeffrey Huang
8a617e0024 Add StatsMetadataList proto
Bug: 148280505
Test: m -j
Change-Id: I84da0cd9c79dc1073c40a439fe2acff2e3fbc7ec
2020-03-24 12:46:14 -07:00
junyulai
f889330c89 [SM01] Add rat type fields in data usage atoms for mobile data
Test: build
Bug: 129082217
Change-Id: Idfc048d49f98acb805f49046a0b3dc871e001142
Merged-In: Idfc048d49f98acb805f49046a0b3dc871e001142
(cherry picked from commit 071a06635b)
2020-03-24 04:28:48 +00:00
Mike Ma
892ccd9bd5 Optimize memory usage in incidentd
EncodedBuffer is used a lot in incidentd. EncodedBuffer uses malloc
internally to acquire memory. Frequently creating and destroying
EncodedBuffer creates memory fragmentation, leading to high memory
usage after taking an incident report.
Also fixes a few other places with lots of malloc/free operations.

This change:
* Creates a pool of EncodedBuffer in incidentd. The saving is
significant. It reduces EncodedBuffer creation from 3 per section to
3 per report.
* Replaces malloc with mmap inside EncodedBuffer. mmap is guaranteed
to be mem page aligned, so there will be no mem fragmentation after
destroying EncodedBuffer.
* Replaces new with mmap inside TombstoneSection
* Forks a process to execute LogSection, because liblog malloc & free
significant amount of memory

Result:
PSS before taking a report: 1295 KB
PSS after taking a report: 1336 KB

Bug: 150311553
Test: heapprofd
Change-Id: I83bd9c969b751c80b2f42747020799bd85d8aae6
2020-03-23 18:46:58 -07:00
TreeHugger Robot
14f33ae83e Merge "Native API Council asked that we remove the C++ class from the public header." into rvc-dev 2020-03-24 01:35:40 +00:00
tsaichristine
a3d2ed8cb2 (Part 3) Use new socket schema with statsd tests
Update last set of statsd tests to use new socket schema

Test: bit statsd_test:*
Bug: 149590301
Change-Id: I0fe2c219ad75813db54ff0cfbad50f55e29cb626
2020-03-23 12:23:26 -07:00
Greg Kaiser
f0f1d83624 ShellSubscriber: Avoid rollover bug
We initialize 'mToken' instead of leaving it uninitialized, making
it much less likely to rollover.  In additional, we change our
check to inequality, to handle rollover if that were to happen.

There is the (theoretical) possibility of exactly 2^32 other
requests being claimed between our claim and our check.  It's
assumed that is essentially impossible and not a concern.

Test: TreeHugger
Bug: 150619687
Change-Id: Iee303e05082a6b3b31ed546bd62d3afe67c771d8
2020-03-23 16:09:57 +00:00
Joe Onorato
084aa3cb17 Native API Council asked that we remove the C++ class from the public header.
The API is simple enough, so just reimplement everything using the
C API directly.

Bug: 148940365
Test: treehugger
Change-Id: I0a75744e975e8d3c2a557e533eacd03200388ddc
2020-03-21 18:33:28 -07:00
TreeHugger Robot
e13e948d6d Merge "Change sint to int in atoms.proto" into rvc-dev 2020-03-21 07:56:17 +00:00
TreeHugger Robot
1e377304cb Merge "Statsd update for native puller api feedback" into rvc-dev 2020-03-21 03:15:06 +00:00
Howard Ro
8e051743c5 Change sint to int in atoms.proto
sint32 is not supported in statsd

Bug: 151110217
Test: m statsd
Change-Id: Ic711cee06a15a957fd4e25c4c01176cc44598cb2
2020-03-20 15:07:20 -07:00
Muhammad Qureshi
4d57eccd1d Merge "Remove kBytesFieldAtoms from atoms_info" into rvc-dev 2020-03-20 20:34:35 +00:00
Tej Singh
73597dcb9a Statsd update for native puller api feedback
Update statsd to take in times in milliseconds instead of nanoseconds.

Also make appropriate updates for graphics stats, odpm, subsystem sleep
state, and LibStatsPullTests

Test: atest LibStatsPullTests
Test: bit statsd_test:*
Bug: 150788562
Change-Id: I593552d6c50bb4dcb89ca9cc1c737781653e7cc5
2020-03-20 11:16:18 -07:00
Muhammad Qureshi
f77913fbc6 Remove kBytesFieldAtoms from atoms_info
Bug: 150415347
Test: m
Test: bit statsd_test:*
Change-Id: I2e55b445121a39e3bf12164cd9123561e8b5bd4c
2020-03-20 10:55:46 -07:00
TreeHugger Robot
a799cdba86 Merge "Implement new perfd<->statsd ShellSubscriber comm." into rvc-dev 2020-03-20 17:35:58 +00:00
Christine Tsai
a9c3cc288c Merge "(Part 2.6 - Final one!) Use new socket schema in ValueMetricProducer_test" into rvc-dev 2020-03-20 16:37:11 +00:00
tsaichristine
e5405ba55a (Part 2.6 - Final one!) Use new socket schema in ValueMetricProducer_test
First CL at ag/10697747. Sending ValueMetricProducer_test as multiple CLs to retain diff and make
it easier to review.

Updates from TestInvalidBucketWhenMultipleBucketsSkipped to TestSlicedStateWithPrimaryField_WithDimensions

Test: bit statsd_test:*
Bug: 149590301
Change-Id: I682a19ea6fe39dd750ca396c9de479befa658f0d
2020-03-19 20:59:32 -07:00
Mike Ma
707263d99c Merge "TextDumpsysSection memory optimization" into rvc-dev 2020-03-20 03:52:54 +00:00
TreeHugger Robot
591bb7b1fe Merge "Enforce permission on native puller API" into rvc-dev 2020-03-19 21:12:52 +00:00