Commit Graph

277 Commits

Author SHA1 Message Date
Nataniel Borges
24512f91a6 Disable failing PIP tests while checking the issue
PIP is not automatically appareing when launching via intent after
the Bubbles tests, when launching independently it works fine.

Bug: 218604389
Test: atest WMShellFlickerTest
Change-Id: Ib5dd76476cbe5cc054e5d4964e6dd4a5cb0ae5cc
2022-02-11 14:01:41 +01:00
TreeHugger Robot
af9e9264f9 Merge "Fix PIP keyboard test" 2022-02-10 16:37:46 +00:00
Pablo Gamito
2c37d7a5e0 Merge "Remove fass assertions from tests" 2022-02-10 14:46:06 +00:00
Nataniel Borges
3bbed27849 Fix PIP keyboard test
Test was missing teardown block

Bug: 218604389
Test: atest WMShellFlickerTest
Change-Id: I2331a16f76b95d31b524c2b714b815a174fbcf04
2022-02-10 13:12:59 +01:00
TreeHugger Robot
d25198a25e Merge "Disable flaky tests" 2022-02-10 09:16:48 +00:00
Nataniel Borges
d86a80274a Merge "Run flicker tests 3x" 2022-02-10 07:32:47 +00:00
Nataniel Borges
77dff60b1b Disable flaky tests
Create specific class for shell transition when whole CUJ crashes

Bug: 209843622
Test: atest FlickerTests WMShellFlickerTests
Change-Id: I8d3d96af7303e6bd7bc4a540aa79b0f9ea3bf4a0
2022-02-09 19:02:17 +01:00
Nataniel Borges
c95bc4d69a Run flicker tests 3x
We run the flicker tests multiple times to detect CLs that intoruce flakiness early. Currently we run 5 executions of each CUJ. However, due to lab capacity and impact in presubmit times, this CL reduces the repetitions to 3

Bug: 209843622
Test: atest FlickerTests WMShellFlickerTests
Change-Id: I165937fb82953d62c028ffe96a948d0ca00dbbe2
2022-02-09 11:11:54 +01:00
Pablo Gamito
b85d6f1ad4 Remove fass assertions from tests
Assertions are very flaky, don't provide any immediately useful information in the current state and going to be replace soon anyways

Test: atest FlickerTests
Change-Id: Id8f7649b818237bc16051d808ef46ca875669faa
2022-02-08 17:10:10 +01:00
Riddle Hsu
5a68f5cd91 Add default theme for flicker test app
Otherwise its WidowManager.LayoutParams#windowAnimations is 0, then
it won't play animation (TransitionAnimation#loadAnimationAttr
returns null) so the the surface of closing app may be updated to
invisible at random timing that causes the test flaky.

Use darker gray as window background color to make it easier to
distinguish the visual result between home, starting window (gray)
and app content (orange).

Bug: 216297992
Bug: 213852103
Test: com.android.server.wm.flicker.launch.OpenAppColdTest
Change-Id: I3bb073dfa52484a63168b3aeb6e7f5f43bd02aac
2022-02-08 14:58:55 +00:00
Nataniel Borges
08136e8150 Merge "Run pip transition 5x" 2022-02-04 10:04:42 +00:00
Nataniel Borges
dd9373015a Run pip transition 5x
The test was previously running the transition 20 times incorrectly. This frequently led to OOM errors, since the system would store 80 instances of protos + flicker objects in memory while making assertions.

Intead run the default 5x

Bug: 216303412
Test: atest ExitPipWithSwipeDownTest
Change-Id: I20fe19239a857cfbc0818910814041323735692a
2022-02-03 15:35:20 +01:00
Nataniel Borges
ccc875cc97 Fetch traces async
Due to the large amount of traces on flicker, the tests take 15min+ to copy the files from the device to the host after the test execution. Instead, start copying the files asynchronously.

It is not possible to use the async mechanism from Tradefed's FilePullerLogCollector because it either: (1) pushes the files form the directory multiple times -- once per test -- creating a lot of duplicates; or (2) cleans up the directory without checking for new files, losing some files.

This CL creates an asynchronous file upload mechanism for flicker using coroutines

Bug: 216111397
Test: atest FlickerTests WMShellFlickerTests
Change-Id: I5bbae834cf3439950626823c6b665c733e27e517
2022-02-01 20:08:51 +01:00
Chris Li
6c76dc017f Merge "Move disabled shelltransit flicker tests to flaky" 2022-01-26 14:48:53 +00:00
Chris Li
63be3c89bb Move disabled shelltransit flicker tests to flaky
So that we can keep track of the status, and move it back to presubmit
after it becomes stable

Bug: 214452854
Test: pass existing
Change-Id: I60453789676cda9eb0892b601806d398af7bdc82
2022-01-26 18:27:49 +08:00
Chris Li
85174deb62 Enable Bubble flicker test with Shell transition
The failure should be fixed with ag/16678935

Bug: 213866869
Bug: 205288792
Bug: 214452854
Test: pass existing with shell transition enabled
Change-Id: I2f8696a14d5f2537858e92f05edc2ef7629e4afe
2022-01-26 14:07:14 +08:00
Chris Li
d4de0febd4 Merge "Enable flicker test group1 with Shell transition" 2022-01-25 02:20:55 +00:00
TreeHugger Robot
c5900ec571 Merge "Support refactored Region class" 2022-01-24 20:44:54 +00:00
Nataniel Borges
b03abbe07a Support refactored Region class
For use in KotlinJS (winscope) the Region class cannot have multiple constructors. JS doesn't support method or constructor overload.

This CL make the tests compatible with the newer class version

Bug: 216111397
Test: atest WMShellFlickerTests
Change-Id: Ifa5f8c6e0063185e2986019b3a27f24c93fe05d2
2022-01-24 18:53:22 +01:00
Nataniel Borges
ba142e0089 Demote flaky flicker tests from presubmit
The test flakiness is significantly slowing down presubmit

Bug: 209843622
Test: atest FlickerTests WMShellFlickerTests
Change-Id: I096e9f69e6de8ff73d527db533e76e8da44fdcb0
2022-01-22 18:23:25 +01:00
Treehugger Robot
c2b9af3ac8 Merge "Create owner test files for subpackages" am: 9e8cfbac2c am: 05bc30f21c am: a39bb54fdd am: 9e5539c793
Original change: https://android-review.googlesource.com/c/platform/frameworks/base/+/1952387

Change-Id: Idee1d783a52db16a7fb7aeadb11b69564e72aca5
2022-01-21 16:14:52 +00:00
Treehugger Robot
9e5539c793 Merge "Create owner test files for subpackages" am: 9e8cfbac2c am: 05bc30f21c am: a39bb54fdd
Original change: https://android-review.googlesource.com/c/platform/frameworks/base/+/1952387

Change-Id: I94c28df75a0af1ea24dcc388fee84c093268ffd0
2022-01-21 15:56:50 +00:00
Nataniel Borges
4e4babec1a Create owner test files for subpackages
Now failures can be automatically directed to the correct buganizer components and forwarded directly to the correct assignees, instead of being all created into the Winscope for further triage

Test: check owners file
Change-Id: I6876a702b4f46443328aab027f7fee994b2bb819
2022-01-20 10:21:40 +00:00
Chris Li
83d4efbbe3 Enable flicker test group1 with Shell transition
Except QuickSwitchBetweenTwoAppsForwardTest b/213867585

Bug: 214452854
Test: pass existing
Change-Id: Ic6d57b72d56906862a04685fa04117afe485c2b1
2022-01-18 16:15:17 +08:00
Nataniel Borges
77a4620a02 Update enter pip focus change test
Previously focus didn't change when entering PIP. On the latest versions it should transition to launcher.

Bug: 198124027
Test: atest WMShellFlickerTests
Change-Id: I323a455f23e912afd9eb6e43b365a289c5159331
2022-01-17 16:24:07 +01:00
Chris Li
c812398169 Merge "Enable flicker test group3 with Shell transition" 2022-01-17 09:08:35 +00:00
Chris Li
70873fd601 Enable flicker test group3 with Shell transition
Bug: 214452854
Test: pass existing
Change-Id: Ia1a10ebaef3a058526be943194e15bf5b5a50ca6
2022-01-14 14:14:11 +00:00
Chris Li
47d7776959 Enable flicker test group2 with Shell transition
Bug: 214452854
Test: pass existing
Change-Id: I85f62b56433562abdbc34c103cddbceff1ac5341
2022-01-14 01:00:29 +00:00
Chris Li
b258c0fd58 Merge "Fix ExitPipWithDismissButtonTest with Shell transition" 2022-01-11 02:30:08 +00:00
Chris Li
9131c6a128 Merge "Fix WallpaperWindowToken visible update with Shell transition" 2022-01-11 02:29:29 +00:00
Chris Li
2490b439d0 Fix ExitPipWithDismissButtonTest with Shell transition
Fix: 210965692
Test: atest WMShellFlickerTests:ExitPipWithDismissButtonTest
Change-Id: I828095f55e7400b661513b853372a3bef5466663
2022-01-09 22:59:36 +08:00
Chris Li
c8a3d88def Fix WallpaperWindowToken visible update with Shell transition
With Shell transition, when mVisibleRequested is changed, it will wait
until the transition finish to update the isVisible(). As a result, if
there is a visible change before the previous transition is finished,
it will be skipped.

For example, when we first open a fullscreen app and then immediately
enter pip:
1. TRANSIT_OPEN will call to hide wallpaper and set mVisibleRequest to
false, but not commit visible until transition animation is finished.
2. TRANSIT_PIP will call to show wallpaper, but if it is started before
the OPEN transition is finsihed, it will not update the wallpaper
mVisibleRequested, which will leave the wallpaper hidden.

Fix: 210965575
Bug: 210965692
Test: atest WMShellFlickerTests:ExpandPipOnDoubleClickTest
Change-Id: I2578aafd024a3c9a6ec96079d4dc6cb740c890d2
2022-01-09 19:35:16 +08:00
Nataniel Borges
dd1d5b61b1 Set cutout mode to shortEdges on flicker
Use short edges on the flicker tests so that the test apps cover the full screen in both raven and in flame devices. With default cutout, some assertions will need to be updated to consider the letterbox layers, as well as app windows not covering the entire display

Bug: 209935473
Test: atest FlickerTests WMShellFlickerTests
Change-Id: Iaeabe2033adc2890605bbef7bce139569f402c4b
2022-01-07 16:22:46 +01:00
Chris Li
85e4cc2e6e Fix SetRequestedOrientationWhilePinnedTest with Shell transition
Fix: 210965919
Test: atest WMShellFlickerTests:SetRequestedOrientationWhilePinnedTest
Change-Id: Ib59300e1d3af8fbcc02a5a85964a1d2ffdaaaf4f
2022-01-06 19:53:06 +08:00
Nataniel Borges
57d1c8b1aa Merge "Update tests to reflect region based assertion changes" 2022-01-04 09:25:31 +00:00
Chris Li
67f1f5abe6 Merge "Set PIP corner radius after rotation" 2021-12-26 07:59:37 +00:00
Chris Li
76e0ddc048 Set PIP corner radius after rotation
When there is screen rotation, we need to set the corner radius and
window crop in both start and finish transaction to avoid janky
animation.

Fix: 210965679
Test: atest WMShellFlickerTests:PipRotationTest
Change-Id: I03d1c3c4e39ea55dd4cc3f43b1170cfd6fe88cfa
2021-12-24 12:15:26 +08:00
Pablo Gamito
793153795a Update tests to reflect region based assertion changes
Bug: 204894659
Test: atest FlickerTests WMShellFlickerTests FlickerLibTest
Change-Id: Ib82953da0f52953046cf2d5f717c9dc11f221a93
2021-12-23 18:48:07 +01:00
Nataniel Borges
43eb55e5d1 Merge "Add support to newer devices on flicker tests" 2021-12-23 13:49:44 +00:00
Nataniel Borges
b876fb200d Add support to newer devices on flicker tests
Some of the assertions used in flicker worked in older devices but not on newer ones.

Disable these assertions and make flicker tests compatible with notch to run on newer devices

Bug: 206753786
Test: atest FlickerTests WMShellFlickerTests
Change-Id: Idf889506d767c4743938ee6727cd228b29873e66
2021-12-22 12:04:44 +01:00
Chris Li
862c4118d4 Set PIP corner radius when enter
Before, it didn't set the corner radius on the finish transition, so the
corner radius is set to 0 when finished.

Bug: 210965679
Test: atest WMShellFlickerTests:PipRotationTest
Change-Id: I7a3c478ef36a4bb730f2153d6b2487a41a469545
2021-12-20 15:13:45 +08:00
Nataniel Borges
306ef506b3 Disable failing shell transition tests
Some tests are failing when using shell transition.

Disable the tests that currently fail so that we can move some shell assertions into presubmit

Test: atest FlickerTests WMShellFlickerTests
Change-Id: If86fb6b053a384050aa533d2ebb45031c1acdf70
2021-12-14 14:04:44 +01:00
Nataniel Borges
12e300d402 Promote/demote flicker tests
Promote stable postsubmit to presubmit
Demote unstable postsubmit to flaky
Demote fass assertions to flaky

Test: atest FlickerTests WMShellFlickerTests
Change-Id: Ib652d9951794b7bf27ffcfd16f4f342bfbf2b2e2
2021-12-07 15:41:09 +01:00
Nataniel Borges
184d33e1ab Promote/demote flicker tests
Promote stable postsubmit to presubmit
Demote unstable postsubmit to flaky
Demote fass assertions to flaky

Test: atest FlickerTests WMShellFlickerTests
Change-Id: I48c4e26b05873a3305cd814b5cb4bedcae9def81
2021-12-01 18:49:05 +01:00
Ioana Moraru
b8f3f9d239 Enable FASS assertions for Pip Exit
Add a new test rule to ExitPipTest for comparing the new FASS
assertions with the old flicker testing. If the flicker tests from one category
(presubmit/postsumbit/flaky) are passing or failing, the corresponding
FASS test should have the same behaviour.

Bug: 201523242
Test: see the results in the dashboard go/wm-tests-flicker
Change-Id: I606e57d8780c0d7cb6191890b052501b7974b034
2021-11-29 10:29:46 +01:00
Nataniel Borges
3968a53df5 Promote/demote stable/flaky flicker tests
Also downgrade all FASS assertions to flaky

Promote:
- stable postsubmit to presubmit
- stable flaky to postsubmit

Demote:
- unstable postsubmit to flaky

Test: atest FlickerTests WMShellFlickerTests

Change-Id: I9e43398b8b00286ce74d4c41505124512bed7eeb
2021-11-24 18:14:34 +01:00
Ioana Moraru
699f3f7fea Enable FASS assertions for Pip Enter
Add a new test rule to EnterPipTest for comparing the new FASS
assertions with the old flicker testing. If the flicker tests from one category
(presubmit/postsumbit/flaky) are passing or failing, the corresponding
FASS test should have the same behaviour.

Bug: 201523242
Test: see the results in the dashboard go/wm-tests-flicker
Change-Id: I884437876f5bef6a58b4f7bacfbad0fa435e0256
2021-11-17 16:00:02 +01:00
Nataniel Borges
eeb9d427ce Merge "Disable tests not compatible with current shell transitions" 2021-11-17 13:59:35 +00:00
Nataniel Borges
7fdad8f97a Disable tests not compatible with current shell transitions
Not all tests are working when wm shell is enabled. However, it is important to put the ones that are working in presubmit/postsubmit to prevent regressions

Test: atest FlickerTests WMShellFlickerTests with shell transitions on
Bug: 206753786
Bug: 206669574
Bug: 206086894
Bug: 204574221
Bug: 204570898
Bug: 206085788
Bug: 206090480
Bug: 206094140
Bug: 206101151

Change-Id: I3e7f6c8c044221c403a0a4030eb7806f2c8b63a5
2021-11-17 12:23:45 +01:00
Nataniel Borges
9aafa92630 Stop rebooting device after Flicker tests
Because of the configuration file split we no longer need to reboot the device at the end of flicker tests - this will speed up the time required to get the final results from tests.

Test: atest FlickerTests WMShellFlickerTests
Bug: 206633034
Change-Id: I76285ca64aa7a8913088e625999d5756a2e662a0
2021-11-16 19:18:11 +01:00