Commit Graph

27 Commits

Author SHA1 Message Date
Steve Block
cda9448206 Fix FsUtils.readDataFromUrl() to do the network request on a background thread
This is required now that doing network requests on the main thread
throws NetworkOnMainThreadException.

Also updates TestsListPreloaderThread to create the FileFilter lazily
and removes a superfluous FileFilter param from the Summarizer
constructor.

Change-Id: If15fddac934cd8be611f6693b25837657abbddc3
2010-11-09 12:33:28 +00:00
Steve Block
24652bc958 Fix DumpRenderTree2 to correctly handle crashed tests
Change-Id: I894987d5b8f24ea31893aefe3cb6450c2736aa90
2010-10-18 20:20:43 +01:00
Steve Block
f460dd4219 Fix a bug with the expected result path in DumpRenderTree2
If we fail to read from any of the possible expected result paths, we should
not record an expected result path.

Change-Id: Ica96b791c14a7755154be4ec9480be4d13f47a19
2010-09-29 18:51:42 +01:00
Maksymilian Osowski
394e0fb849 Updated Summarizer to use SQLite DB.
It avoids OOM issues by dumping the memory to the DB every
few hundred tests and then reading them in chunks when
generating summaries, etc.

Change-Id: I5f63d960d456f17e9f2189230a7ae0ca5607ce51
2010-09-17 16:35:46 +01:00
Maksymilian Osowski
fadb0de33d Changed Summarizer to dump generated html code to file every few hundred tests to avoid OOM errors.
Change-Id: Id0917d45ffab2e47d67d17d629f342add5292d4e
2010-09-17 16:14:39 +01:00
Maksymilian Osowski
9893d96d12 Updated ManagerService to call Summarizer.summarize() in a separate thread.
This could potentially take a long time and result in ANR.

Change-Id: Ib8b246e627e841217b68c955d92c97801e93d923
2010-09-17 15:11:15 +01:00
Steve Block
8a6def0247 Update DumpRenderTree2 to handle failing tests where no meaningful diff is available
Currently, DumpRenderTree2 asigns the following result codes
{PASS, FAIL_RESULT_DIFFERS, FAIL_NO_EXPECTED_RESULT, FAIL_TIMED_OUT,
FAIL_CRASHED}

This is not strictly correct, as a test may fail for any of three
reasons - crashing, timing out, or the the result not being as
expected. Therefore, it's possible for a test to fail even if the
result is as expected.

This patch updates AbstractResult to handle each of the three reasons
for failure separately. We then test all three to determine if the
test has passed.

This allows us to correctly report whether or not the result differs
from expected for a failing test.

Change-Id: I7adcfe72c4dd0bd3de2e1b868d9807be6eb5bddf
2010-09-17 13:21:22 +01:00
Steve Block
c0847c55ea Remove superfluous CSS from DumpRenderTree2's Summarizer
Change-Id: Ifc3764ec3cf7bd5ee412abaf45112eed22083456
2010-09-17 12:30:21 +01:00
Maksymilian Osowski
75aa0d3828 Changed Summarizer to add header before printing the path in summary.txt and details.html
Change-Id: I6413e1871ede077793d49b7d4732f90d0aea352f
2010-09-14 18:13:07 +01:00
Steve Block
ab87db4ad8 Update DumpRenderTree2 to only calculate the WebKit revision once per test run
This is obtained by a HTTP request to ThirdPartyProject.prop

Change-Id: I34e310cf52fbcef24bb2a76e64aab5263b3052f1
2010-09-13 17:32:16 +01:00
Steve Block
7438b53487 Update DumpRenderTree2 text results output to better match HTML version
Always print number of crashes and line things up nicely.

Change-Id: I79a72056ba5ce007fdd4154f1518b9535b595317
2010-09-13 14:28:46 +01:00
Steve Block
0e2bae14b4 Update DumpRenderTree2 to reflect removal of WEBKIT_MERGE_REVISION
We now use ThirdPartyProject.prop
See https://android-git.corp.google.com/g/#change,64831

Change-Id: Ib52fdd93ac08fb19ec8b7f539eb084c0b668e01b
2010-09-07 22:05:21 +01:00
Maksymilian Osowski
2ca8acdb56 Adds webkit revision to the summary and details.
The revision number is taken from external/webkit/WEBKIT_MERGE_REVISION and is served to the java code by apache server. To be able to do that, the new alias directive had to be
added to the run_apache2.py script.

Bug: 2889572
Change-Id: Ie3d147e4d8ea9edd0144b819152121563b8bd759
2010-09-02 17:27:35 +01:00
Maksymilian Osowski
01c1f9d9e9 The results now show which expected result was used.
Bug: 2889857
Change-Id: If2b7e6e6a22ddb5db6c6a50a358e8f263914e6c0
2010-09-02 17:18:48 +01:00
Maksymilian Osowski
1b034781f4 Shows the details when all the tests finish and fixes the orientation issue.
The tests no longer restart when orientation changes.

Bug: 2904456
Change-Id: Ie1ee463dbd63fb1a2ad2e9c28bf5989002f22215
2010-09-02 14:49:04 +01:00
Maksymilian Osowski
6ae5ce4b40 Updated Summarizer to report tests under all 4 possible categories now.
Categories: ((un)expected passes/failures).
Also, FileFilter uses FAIL instead of IGNORE_RESULTS now.

Bug: 2899965
Change-Id: I06835b3d5cea84cbc92225a940bf7361cf832f59
2010-09-02 12:53:26 +01:00
Maksymilian Osowski
603b70dd8b Updated the Summarizer to link to the source of the tests.
Change-Id: I90018898c627b99f5a49c301e87593b52cd18ce4
Bug-Id: 2889573
2010-09-01 12:31:33 +01:00
Maksymilian Osowski
2e64bb7afd Added more info to the Summarizer.
Bug: 2889572
Change-Id: If02dbdf90e8b8717370b5586cc72be235c93db79
2010-08-12 14:32:30 +01:00
Maksymilian Osowski
5fb9ff420d Added a basic version of python script and supporting code in DumpRenderTree.
Change-Id: Ic60ef9b89f74a3a36a4c31765f99c8de08dce911
2010-08-11 12:20:30 +01:00
Maksymilian Osowski
ea46f2595a Resets Summarizer after everything's finished.
Bug: 2903641
Change-Id: I7cecd454fe726a1527999259d37bd51ea2785116
2010-08-10 18:28:52 +01:00
Maksymilian Osowski
3df4eab3de Added generating text summary to Summarizer.
Change-Id: I8768403b0e4edd46132018c9da6de54b218fb594
2010-08-09 18:15:33 +01:00
Maksymilian Osowski
7e2a39b598 Added detection of missing EventSender functions.
Change-Id: I5268a66089d5e29a9ae5e496d05f3c79465771f5
2010-08-06 10:11:12 +01:00
Maksymilian Osowski
4a9e3a6dc3 Added detecting unimplemented layoutTestController functions and green squares.
Change-Id: Iaa7bbdb83a074a874a210f7975c6c3b731e0a924
2010-08-03 15:35:07 +01:00
Maksymilian Osowski
6d0dae6a65 Upgraded DRT2 to generate nice HTML summaries of the tests.
Change-Id: Ibe4ee3af30fe20aeab2f3fed9c4c5693c9a0f268
2010-07-29 17:17:36 +01:00
Maksymilian Osowski
5f0ccd76a8 Moved practically all of the prerefactoring functionality to the new design.
Renamed LayoutTestsRunner activity to TestsListActivity. It will be responsible for restrating the Executer after the crash. Now it only starts
it.
Renamed LayoutTestsRunnerThread to TestsListPreloaderThread. It only preloads tests now.
LayoutTest class is no longer needed, its functionality is in LayoutTestsExecuter.
Most of the functionality from LayoutTestsRunnerThread is now in ManagerService.

Change-Id: I08924d949ceb9f8816888bc8e795256d0542fa99
2010-07-27 16:15:35 +01:00
Maksymilian Osowski
4ee7f4b194 LayoutTest class with supporting classes (AbstractResult, TextResult).
These classes are responsible for actually running the tests, checking the results, generating visual diffs, etc.

Change-Id: I9b7f554409db1c97ac086c456db3aea3b993b5c6
2010-07-21 15:20:47 +01:00
Maksymilian Osowski
3c8ccb3845 Added the LayoutTestsRunner class that is responsible for running the tests. Also, added some methods to FileFilter.
It preloads the tests from the given path, runs them and asks for dumps and diffs. It will also prepare summaries in the future. It delegates
most of the work of actually running the individual tests to LayoutTest class and AbstractResult (and its subclasses in the future).

Change-Id: I483bf26a380b539e4769e61b4a09fa270ab0e8e9
2010-07-16 14:32:12 +01:00