Commit Graph

49 Commits

Author SHA1 Message Date
Automerger Merge Worker
b00e24740b Merge changes from topic "flicker" am: db3931fe93 am: 3bc31d6b34
Change-Id: Idb5519009b60c378960893aa072ba9d21ebc8db2
2020-02-27 12:03:15 +00:00
Automerger Merge Worker
a96b1f5ad1 Ensure screenshot layer appears during rotation am: 6a087ec8ed am: f401edf498
Change-Id: Ief97c63ea36466c4cb042f8db3af4455000d15f8
2020-02-27 12:03:05 +00:00
Automerger Merge Worker
40ed98adcc New FlickerTests: Create initial flicker tests for Pip am: 992a155b61 am: 1e665c424e
Change-Id: I804d26250d1d4df1ce9342eed01f195bc13fd1a1
2020-02-27 12:03:01 +00:00
Automerger Merge Worker
2db43c5246 Merge "Print the checksum of the trace file on failure" am: ef04ccb466 am: 895568cfe1
Change-Id: Id455d3186726eb82d268700cfe646ec0f0e80259
2020-02-27 12:02:53 +00:00
Automerger Merge Worker
f401edf498 Ensure screenshot layer appears during rotation am: 6a087ec8ed
Change-Id: Ie2657049ce147e8c3b4a8955f6d575804fdf97b7
2020-02-27 11:49:25 +00:00
Automerger Merge Worker
1e665c424e New FlickerTests: Create initial flicker tests for Pip am: 992a155b61
Change-Id: I50d0d2294e297d062e4384b342e8577c66e62f02
2020-02-27 11:49:21 +00:00
Jorim Jaggi
db3931fe93 Merge changes from topic "flicker"
* changes:
  New flicker test: Add rotation to split screen test
  Ensure screenshot layer appears during rotation
  New FlickerTests: Create initial flicker tests for Pip
2020-02-27 11:39:03 +00:00
Nataniel Borges
68e0c874a3 New flicker test: Add rotation to split screen test
Currently the split screen test only starts the app in 0 degrees. As the other tests, start in both 0 and 90 degrees to ensure animations work correctly.

Test: atest FlickerTests:OpenAppToSplitScreenTest
Change-Id: Ied316d9c48b243728544ad4a972cb3e0fa81bec8
2020-02-25 12:55:20 +01:00
Nataniel Borges
6a087ec8ed Ensure screenshot layer appears during rotation
Add some additional checks on flicker tests to ensure that the screenshot layer ir correctly being displayed during the rotation animation.

Test: atest FlickerTests:ChangeAppRotationTest, atest FlickerTests:OpenAppColdTest and atest FlickerTests:OpenAppWarmTest
Change-Id: I2a6c74179c07eee86caae08f05430752aa4e084e
2020-02-25 12:55:04 +01:00
Nataniel Borges
992a155b61 New FlickerTests: Create initial flicker tests for Pip
Create an initial set of tests for pip. Initial tests are:
- All regions covered
- Status and navigation bar is always visible
- Pip window is shown, expanded, closed, and doesn't flicker
- Pip layer is shown, expanded, closed, and doesn't flicker
- Background is displayed behind Pip window

Test: atest FlickerTests:PipToHomeTest and atest FlickerTests:PipToAppTest
Change-Id: I16345de39a799744b97a5390495538d568693222
2020-02-25 12:54:43 +01:00
Nataniel Borges
0606f92d7d Print the checksum of the trace file on failure
When the flicker tests fail we log as well what is the checksum of the trace file which was analyzed to ensure that the file which was tested is the same available for debugging. Also, print the checksum information on the test failure message.

Test: atest FlickerTests, then check if the resulting folder contains a ".sha256" file for each WM/SF trace.
Change-Id: I448bf373e3b837c2e40317809d511a0f5236de1f
2020-02-10 11:06:57 +01:00
Automerger Merge Worker
01a6af0611 Merge "Reduce the number of rotations of flicker tests" am: b6d051c228 am: 77f788067d am: 425a97149a
Change-Id: Ie7551934d419d5c9ca102e0b5b86f28a7c8f2b97
2020-01-28 21:14:25 +00:00
Automerger Merge Worker
e1422db2b9 Merge "Disable flaky flicker tests" am: e56396645b am: 29ff225c90 am: 575f7ef7bb
Change-Id: I53636801cc06fe23eccb35e033770b504a166073
2020-01-28 21:13:34 +00:00
Jorim Jaggi
b6d051c228 Merge "Reduce the number of rotations of flicker tests" 2020-01-28 20:34:21 +00:00
Jorim Jaggi
e56396645b Merge "Disable flaky flicker tests" 2020-01-28 20:34:11 +00:00
Nataniel Borges
0f5ebe8048 Disable flaky flicker tests
These tests are still flaky. Disable them for now so that we can move the flicker tests to presubmit.

Test: atest FlickerTests
Change-Id: I811f32723cdbaece3aff7d6dbb100c0c5b2653fa
2020-01-28 16:58:36 +01:00
Nataniel Borges
e894f6a870 Reduce the number of rotations of flicker tests
To reduce the flicker test time, reduce the tests to consider only:
* Start test at 0 degrees
* Start test at 90 degrees
* Rotation tests from 0 to 90 degrees
* Rotation tests from 90 to 0 degrees

Also, execute each action a single time.

Test: atest FlickerTest
Change-Id: I5976cf950998a8150fd5a1b32f0f1432cd731ee1
2020-01-28 16:47:18 +01:00
Automerger Merge Worker
85ca06ebbf Merge "Disable seamless rotation on flicker tests" am: 35d3ce82d4 am: 776b515187 am: e56d164d1a
Change-Id: I35180b88fb0fbe6e624b183dd39a4247feb1c0cb
2020-01-27 13:37:22 +00:00
Nataniel Borges
239e4bffff Disable seamless rotation on flicker tests
Due to bug b/147659548 the seamless rotation is not working properly on the flicker tests. Disable the test until the bug is fixed.

Test: atest FlickerTests
Change-Id: Ib286cadde9019e12dbf7c673cb7f60b53a1687bf
2020-01-21 11:26:47 +01:00
Jorim Jaggi
1473439a19 Merge "Automatically clean up flicker test data" am: d915fc7025 am: ade366490a
am: 2d3934fcaa

Change-Id: I1a2be40c54e4025d4c160e2f6beaa61aa1eb51db
2019-12-09 06:32:34 -08:00
Nataniel Borges
9f2ad75a6e Automatically clean up flicker test data
Currently the flicker tests leave data on the /sdcard/flicker directory. We now clean up this data after the test.

The logs are still available in the atest results directory.

Test: atest FlickerTests
Change-Id: Ia83cbedff3104964e3d4be14a7c035f2193d36a8
2019-12-04 10:38:42 +01:00
Nataniel Borges
2447db9d17 Merge "Include janky runs on flicker rotation tests" am: 3341bbefea am: fcc1ef2d7d
am: e1367ec912

Change-Id: I4030670dd2f0068511dba0f4ccb4e49138b748d5
2019-11-26 08:48:29 -08:00
Treehugger Robot
3341bbefea Merge "Include janky runs on flicker rotation tests" 2019-11-26 16:29:45 +00:00
Nataniel Borges
8e65a898ec Include janky runs on flicker rotation tests
Sometimes the tests crash because all runs are janky. Our checks are so far robust to jank, so we will consider janky runs too

Test: atest FlickerTests
Change-Id: I20303d530e1a9cc266230530c6f7fa1cecd51020
2019-11-26 11:46:42 +01:00
Jorim Jaggi
670b181d0a Merge "Record videos of the flicker tests" am: 6d5457bbb8 am: fcc92a701e
am: b2803b2350

Change-Id: I383ce0f59d16181634b3d298bba758a5355bfd8e
2019-11-13 08:08:20 -08:00
Nataniel Borges
4512ac6fa8 Record videos of the flicker tests
After a change in the flickerlib the videos are no longer recorded by
default. The tests which need to record a video must signal it.

Test: atest FlickerTests, check if the video exists in /sdcard/flicker
Change-Id: I8e74411ca6bf654ec7c0afcacf1c0591610f50bb
2019-11-12 16:07:28 +01:00
Jorim Jaggi
40367bbd1b Also run flicker tests on cloud
Such that we can compare real device and cloud result

Test: Builds
Change-Id: I1f8cd0c7e6a2344a35c70e0222541058edf330ce
2019-11-07 17:45:44 +01:00
Vishnu Nair
ff8a65e941 Add FlickerTests to run only on crosshatch in postsubmit
These tests are timing dependent and require the DUT to have the same performance
characteristics as the real devices.

Test: cd frameworks/base/tests/FlickerTests && atest :postsubmit
Change-Id: Ie836aa0a5df1050733169c8bd16adaccbfdf5e3c
2019-10-16 15:09:46 +00:00
Treehugger Robot
f2a739d5f0 Merge changes from topic "flicker-rotation"
* changes:
  New FlickerTests: Run split screen tests at 0 and 90 degrees
  New FlickerTests: Start app warm and cold at both 0 and 90 degrees
  New FlickerTests: Start IME alongside the app (0 and 90 degrees)
  New FlickerTests: Start IME tests with app rotated by 90 degrees
2019-09-27 14:21:58 +00:00
Nataniel Borges
40c517afac New FlickerTests: Run split screen tests at 0 and 90 degrees
Test: atest ResizeSplitScreenTest
Change-Id: I44b0733f8abbb022f2a00072bbb398e6d29699c7
2019-09-26 17:46:47 +02:00
Nataniel Borges
97572097a4 Add secure settings to the test apps
When running the flicker tests on postsubmit the apps require the following permission: "android.permission.WRITE_SECURE_SETTINGS"

Test: atest FlickerTests
Change-Id: I6a2647fb6d2351617297b3f89ae3f3edc1aad981
2019-09-26 12:01:57 +02:00
Nataniel Borges
d051c3b767 New FlickerTests: Start app warm and cold at both 0 and 90 degrees
Test: atest OpenAppColdTest OpenAppWarmTest
Change-Id: I1ea7377a07938b583c537b3db2da7bd5a3fc0ae1
2019-09-25 16:12:18 +02:00
Nataniel Borges
4bfe816030 New FlickerTests: Start IME alongside the app (0 and 90 degrees)
Test: atest FlickerTests:CloseImeAutoOpenWindowToAppTest
Change-Id: I8ed60eb5e4653cdcf4d31402069be7da96c5d2bb
2019-09-25 16:11:14 +02:00
Nataniel Borges
89888bf124 New FlickerTests: Start IME tests with app rotated by 90 degrees
Test: atest FlickerTests:CloseImeWindowToAppTest FlickerTests:CloseImeWindowToHomeTest
Change-Id: Ib78e3ecf5629938b722e067027098b8c43a64771
2019-09-25 16:10:50 +02:00
Nataniel Borges
0e476aa881 Ignore flicker tests with open bugs
Some flicker tests are failing because of open bugs. Mark them as
ignored until the bugs are fixed to prevent test failures.

Test: atest FlickerTests
Change-Id: If9216bccc32d64cd32ff87fae785df200eae8b79
2019-09-18 17:18:22 +02:00
Nataniel Borges
eb63fdd6e5 Don't remove flicker test logs from device for easier debugging
Test: atest FlickerTests
Change-Id: Id8a7869967df2ccd2e905c61cc1a348ae06d4430
2019-09-18 17:17:35 +02:00
Nataniel Borges
f5a1ed72a8 Reduce delays on Flicker test rotation
Flicker tests currently wait 3 seconds until the screen rotation have
been complete. This long delay may hide some issues. Reduce delay to 1
second.

Test: atest FlickerTests
Change-Id: I1646086018fac52c0751ff70fff1f5082fee4685
2019-09-18 17:17:24 +02:00
Nataniel Borges
527ee2e145 Make OpenAppCold/Warm tests compatible with AOSP
Test: atest FlickerTests
Change-Id: Ib1071b125597e73a5c72e22e9107ae4d29f5cb7f
2019-09-18 17:17:03 +02:00
Nataniel Borges
f5c828cb48 Make Flicker Tests compatible with current AOSP
Current status:
atest FlickerTests: Passed: 132, Failed: 11, Ignored: 0

Test: atest FlickerTests
Change-Id: I581005fd293631630a3939eeaae485c1fd41ebb9
2019-09-18 12:45:46 +02:00
Winson Chung
0d4ac4ec1c Exposing flickerlib classes and layer tracing to sysui
- Allow recents component to enable/disable layer tracing
- Expose flickerlib classes for use with sysui/launcher tests
- Allow trace files to be moved to a path that the client can read
- Ensure layers are flattened with depth traversal to encode ordering

Bug: 140244969
Test: atest FlickerTests
Change-Id: I7af5699ff5b8a4bc62c1a8105c67b31bc45a2236
Merged-In: Ia0934ec7c2b9484fa6c85f5aa8d9b2e5e6f7dc0f
2019-09-16 18:08:13 +00:00
Colin Cross
e0bcbdb2bd Merge "Update frameworks tests for truth 0.42" into qt-dev-plus-aosp 2019-04-17 20:09:15 +00:00
Tadashi G. Takaoka
9f376950bd Rename cts-amwm-util to cts-wm-util
Test: Build FlickerTests
  atest -b FlickerTests
Bug: 129909356
Change-Id: I796bd8f52be21ccb7f770fb6c8dda9605451b10d
2019-04-17 02:36:22 +00:00
Colin Cross
b6cf22daf1 Update frameworks tests for truth 0.42
SubjectFactory has been replaced with Subject.Factory.

Bug: 130435731
Test: m checkbuild
Change-Id: I5ae4009aeeab54518c5169a5cd37948cb056058e
2019-04-12 14:34:58 -07:00
Julien Desprez
c82d3e9e3c Explicitly exclude @Ignore annotation
Some flicker class are marked Ignored causing some issue
with the AJUR runner. To avoid them, simply filter it.

Test: None
Bug: 129468528
Change-Id: I6442204eb6f89b4bf90a39ee821ebc4daebbe120
2019-03-28 08:50:03 -07:00
Hsin-Yi Chen
677a0025cb Replace GoogleDeviceSetup with DeviceSetup in FlickerTests
This commit replaces GoogleDeviceSetup which is not on AOSP.

Test: atest FlickerTests
Bug: 128877715
Change-Id: I2e4167dd7e4e5ef3352cb8fda2f753a0acf2f600
2019-03-19 05:31:40 +00:00
Sasha Smundak
89414e1ed5 Convert tests/**/Android.mk files to Android.bp, part II
See build/soong/README.md for more information.

Bug: 122332340
Test: treehugger
Change-Id: I886b6536a0ef1c8d21f15ec7ff9fdb9784d5b865
2019-03-12 09:37:32 -07:00
Brett Chabot
502ec7ae4b Migrate remainder of frameworks/base to androidx.test
See go/jetpack-test-android-migration

Exempt-From-Owner-Approval: automated package name refactoring

Test: m  m -j BroadcastRadioTests KeystoreTests mediaframeworktest ActivityManagerPerfTests AppLaunch AppLaunchWear BackgroundDexOptServiceIntegrationTests AppCompatibilityTest DynamicCodeLoggerIntegrationTests FlickerLibTest InternalTests PackageWatchdogTest RcsTests RollbackTestAppAv1 RollbackTestAppAv2 RollbackTestAppACrashingV2 RollbackTestAppBv1 RollbackTestAppBv2 RollbackTestAppASplitV1 RollbackTestAppASplitV2 RollbackTest ServiceCrashTest UsageStatsPerfTests UsbTests WindowAnimationJank
Change-Id: I32fe3297656eec6060da6c7e24582bcd5315fb16
2019-03-02 00:35:17 +00:00
Nataniel Borges
7b3ec11b8c Fix error message on Flicker's launchSplitActivity test
When the Flicker test is unable to launch a split activity it crashes
with a `NullPointerException`.

Change this error into a meaningful error message.

Test: Run `atest WindowManagerSmokeTest`
Change-Id: I1c3c06b3327b94ad6da1a1eabe147182202676a1
2019-01-30 17:29:13 -08:00
Vishnu Nair
8248b7c995 Move FlickerTests to frameworks/base/tests 2/2
platform_testing is moving to AOSP first development. FlickerTests may require access to
in development test apis and cannot be developed in AOSP.

Other minor changes:
- remove FlickerTests from APCT. FlickerTests will use TEST_MAPPING
- use @FlakyTest instead of @Ignore annotation to skip tests

Test: atest FlickerTests
Test: make -j32 continuous_instrumentation_tests
Test: cd frameworks/base/tests/FlickerTests/ && atest :postsubmit

Bug: 112044297
Change-Id: Icc3194ddd9b801034d8797bf2ec60ebff071612c
2018-08-01 10:13:36 -07:00