am fe089334: Merge "Doc Change: cherry-pick from master. ->Testing Procedures documents testing_eclipse.html, testing_otheride." into froyo

Merge commit 'fe08933450f4be42c9b3eac900c64f23857ecd17' into froyo-plus-aosp

* commit 'fe08933450f4be42c9b3eac900c64f23857ecd17':
  Doc Change: cherry-pick from master. ->Testing Procedures documents testing_eclipse.html, testing_otheride.
This commit is contained in:
Dirk Dougherty
2010-05-12 07:49:00 -07:00
committed by Android Git Automerger
7 changed files with 1194 additions and 62 deletions

View File

@@ -0,0 +1,37 @@
page.title=Testing Overview
@jd:body
<p>
Android includes powerful tools for setting up and running test applications.
Whether you are working in Eclipse with ADT or working from the command line, these tools
help you set up and run your tests within an emulator or the device you are targeting.
The documents listed below explain how to work with the tools in your development environment.
</p>
<p>
If you aren't yet familiar with the Android testing framework, please read the topic
<a href="{@docRoot}guide/topics/testing/testing_android.html">Testing and Instrumentation</a>
before you get started.
For a step-by-step introduction to Android testing, try the <a
href="{@docRoot}resources/tutorials/testing/helloandroid_test.html">Hello, Testing</a>
tutorial, which introduces basic testing concepts and procedures.
For a more advanced tutorial, try <a
href="{@docRoot}resources/tutorials/testing/activity_test.html">Activity Testing</a>,
which guides you through a more complex testing scenario.
</p>
<dl>
<dt><a href="testing_eclipse.html">Testing in Eclipse, with ADT</a></dt>
<dd>
The ADT plugin lets you quickly set up and manage test projects directly in
the Eclipse UI. Once you have written your tests, you can build and run them and
then see the results in the Eclipse JUnit view. You can also use the SDK command-line
tools to execute your tests if needed.
</dd>
<dt><a href="testing_otheride.html">Testing in Other IDEs</a></dt>
<dd>
The SDK command-line tools provide the same capabilities as the ADT plugin. You can
use them to set up and manage test projects, build your test application,
run your tests, and see the results. You use
the <code>android</code> tool to create and manage test projects, the Ant build system
to compile them, and the <code>adb</code> tool to install and run them.
</dd>
</dl>

View File

@@ -0,0 +1,370 @@
page.title=Testing In Eclipse, with ADT
@jd:body
<div id="qv-wrapper">
<div id="qv">
<h2>In this document</h2>
<ol>
<li><a href="#CreateTestProjectEclipse">Creating a Test Project</a></li>
<li><a href="#CreateTestAppEclipse">Creating a Test Application</a></li>
<li><a href="#RunTestEclipse">Running Tests</a></li>
</ol>
</div>
</div>
<p>
This topic explains how create and run tests of Android applications in Eclipse with ADT.
with the basic processes for creating and running applications with ADT, as described in
<a href="{@docRoot}guide/developing/eclipse-adt.html">Developing In Eclipse, with ADT</a>.
Before you read this topic, you should read about how to create a Android application with the
basic processes for creating and running applications with ADT, as described in
<a href="{@docRoot}guide/developing/eclipse-adt.html">Developing In Eclipse, with ADT</a>.
You may also want to read
<a href="{@docRoot}guide/topics/testing/testing_android.html">Testing and Instrumentation</a>,
which provides an overview of the Android testing framework.
</p>
<p>
ADT provides several features that help you set up and manage your testing environment
effectively:
</p>
<ul>
<li>
It lets you quickly create a test project and link it to the application under test.
When it creates the test project, it automatically inserts the necessary
<code>&lt;instrumentation&gt;</code> element in the test application's manifest file.
</li>
<li>
It lets you quickly import the classes of the application under test, so that your
tests can inspect them.
</li>
<li>
It lets you create run configurations for your test application and include in
them flags that are passed to the Android testing framework.
</li>
<li>
It lets you run your test application without leaving Eclipse. ADT builds both the
application under test and the test application automatically, installs them if
necessary to your device or emulator, runs the test application, and displays the
results in a separate window in Eclipse.
</li>
</ul>
<p>
If you are not developing in Eclipse or you want to learn how to create and run tests from the
command line, see
<a href="{@docRoot}guide/developing/testing/testing_otheride.html">Testing in Other IDEs</a>.
</p>
<h2 id="CreateTestProjectEclipse">Creating a Test Project</h2>
<p>
To set up a test environment for your Android application, you must first create a separate
application project that holds the test code. The new project follows the directory structure
used for any Android application. It includes the same types of content and files, such as
source code, resources, a manifest file, and so forth. The test application you
create is connected to the application under test by an
<a href="{@docRoot}guide/topics/manifest/instrumentation-element.html">
<code>&lt;instrumentation&gt;</code></a> element in its manifest file.
</p>
<p>
The <strong>New Android Test Project</strong> dialog makes it easy for you to generate a
new test project that has the proper structure, including the
<code>&lt;instrumentation&gt;</code> element in the manifest file. You can use the New Android
Test Project dialog to generate the test project at any time. The dialog appears just after you
create a new Android main application project, but you can also run it to create a test project
for a project that you created previously.
</p>
<p>
To create a test project in Eclipse with ADT:
</p>
<ol>
<li>
In Eclipse, select <strong>File &gt; New &gt; Other</strong>. This
opens the Select a Wizard dialog.
</li>
<li>
In the dialog, in the Wizards drop-down list,
find the entry for Android, then click the toggle to the left. Select
Android Test Project, then at the bottom
of the dialog click Next. The New Android Test Project wizard appears.
</li>
<li>
Enter a project name. You may use any name, but you may want to
associate the name with the project name for your Application. One
way to do this is to take the Application's project name, append the
string "Test" to it, and then use this as the test case project name.
</li>
<li>
In the Test Target panel, set
An Existing Android Project, click
Browse, then select your Android application from
the list. You now see that the wizard has completed the Test
Target Package, Application Name, and
Package Name fields for you (the latter two are in
the Properties panel).
</li>
<li>
In the Build Target panel, select the Android SDK
platform that you will use to test your application. Make this the same as the
build target of the application under test.
</li>
<li>
Click Finish to complete the wizard. If
Finish is disabled, look
for error messages at the top of the wizard dialog, and then fix
any problems.
</li>
</ol>
<p>
</p>
<h2 id="CreateTestAppEclipse">Creating a Test Application</h2>
<p>
Once you have created a test project, you populate it with a test
Android application. This application does not require an {@link android.app.Activity Activity},
although you can define one if you wish. Although your test application can
combine Activities, Android test class extensions, JUnit extensions, or
ordinary classes, you should extend one of the Android test classes or JUnit classes,
because these provide the best testing features.
</p>
<p>
Test applications do not have an Android GUI. Instead, when you run the application in
Eclipse with ADT, its results appear in the JUnit view. If you run
your tests with {@link android.test.InstrumentationTestRunner InstrumentationTestRunner} (or a related test runner),
then it will run all the methods in each class. You can modify this behavior
by using the {@link junit.framework.TestSuite TestSuite} class.
</p>
<p>
To create a test application, start with one of Android's test classes in the Java package {@link android.test android.test}.
These extend the JUnit {@link junit.framework.TestCase TestCase} class. With a few exceptions, the Android test classes
also provide instrumentation for testing.
</p>
<p>
For test classes that extend {@link junit.framework.TestCase TestCase}, you probably want to override
the <code>setUp()</code> and <code>tearDown()</code> methods:
</p>
<ul>
<li>
<code>setUp()</code>: This method is invoked before any of the test methods in the class.
Use it to set up the environment for the test. You can use <code>setUp()</code>
to instantiate a new <code>Intent</code> object with the action <code>ACTION_MAIN</code>. You can
then use this intent to start the Activity under test.
<p class="note"><strong>Note:</strong> If you override this method, call
<code>super.setUp()</code> as the first statement in your code.
</p>
</li>
<li>
<code>tearDown()</code>: This method is invoked after all the test methods in the class. Use
it to do garbage collection and re-setting before moving on to the next set of tests.
<p class="note"><strong>Note:</strong> If you override this method, you must call
<code>super.tearDown()</code> as the <em>last</em> statement in your code.</p>
</li>
</ul>
<p>
Another useful convention is to add the method <code>testPreConditions()</code> to your test
class. Use this method to test that the application under test is initialized correctly. If this
test fails, you know that that the initial conditions were in error. When this happens, further test
results are suspect, regardless of whether or not the tests succeeded.
</p>
<p>
The Resources tab contains an <a href="{@docRoot}resources/tutorials/testing/activity_test.html">Activity Testing</a>
tutorial with more information about creating test classes and methods.
</p>
<h2 id="RunTestEclipse">Running Tests</h2>
<div class="sidebox-wrapper">
<div class="sidebox">
<h2>Running tests from the command line</h2>
<p>
If you've created your tests in Eclipse, you can still run your tests and test
suites by using command-line tools included with the Android SDK. You may want to
do this, for example, if you have a large number of tests to run, if you have a
large test case, or if you want a fine level of control over which tests are run at
a particular time.
</p>
<p>
To run tests created in Eclipse with ADT with command-line tools, you must first
install additional files into the test project using the <code>android</code> tool's
"create test-project" option. To see how to do this, read the section
<a href="{@docRoot}guide/developing/testing/testing_otheride.html#CreateProject">
Creating a test project</a> in the topic
<a href="{@docRoot}guide/developing/testing/testing_otheride.html">Testing in Other
IDEs</a>.
</p>
</div>
</div>
<p>
When you run a test application in Eclipse with ADT, the output appears in
an Eclipse view panel. You can run the entire test application, one class, or one
method of a class. To do this, Eclipse runs the <code>adb</code> command for running a test application, and
displays the output, so there is no difference between running tests inside Eclipse and running them from the command line.
</p>
<p>
As with any other application, to run a test application in Eclipse with ADT you must either attach a device to your
computer or use the Android emulator. If you use the emulator, you must have an Android Virtual Device (AVD) that uses
the same target
</p>
<p>
To run a test in Eclipse, you have two choices:</p>
<ol>
<li>
Run a test just as you run an application, by selecting
<strong>Run As... &gt; Android JUnit Test</strong> from the project's context menu or
from the main menu's <strong>Run</strong> item.
</li>
<li>
Create an Eclipse run configuration for your test project. This is useful if you want multiple test suites, each consisting of selected tests from the project. To run
a test suite, you run the test configuration.
<p>
Creating and running test configurations is described in the next section.
</p>
</li>
</ol>
<p>To create and run a test suite using a run configuration:</p>
<ol>
<li>
In the Package Explorer, select the test
project, then from the main menu, select
<strong>Run &gt; Run Configurations...</strong>. The
Run Configurations dialog appears.
</li>
<li>
In the left-hand pane, find the
Android JUnit Test entry.
In the right-hand pane, click the Test tab.
The Name: text box
shows the name of your project. The
Test class: dropdown box shows one your project's classes
test classes in your project.
</li>
<li>
To run one test class, click Run a single test, then enter your project
name in the Project: text box and the class name in the
Test class: text box.
<p>
To run all the test classes,
click Run all tests in the selected project or package,
then enter the project or package name in the text box.
</p>
</li>
<li>
Now click the Target tab.
<ul>
<li>
Optional: If you are using the emulator, click
Automatic, then in the Android Virtual Device (AVD)
selection table, select an existing AVD.
</li>
<li>
In the Emulator Launch Parameters pane, set the
Android emulator flags you want to use. These are documented in the topic
<a href="{@docRoot}guide/developing/tools/emulator.html#startup-options">Emulator Startup Options</a>.
</li>
</ul>
<li>
Click the Common tab. In the
Save As pane, click Local to save
this run configuration locally, or click Shared to
save it to another project.
</li>
<li>
Optional: Add the configuration to the Run toolbar and the <strong>Favorites</strong>
menu: in the Display in Favorites pane
click the checkbox next to Run.
</li>
<li>
Optional: To add this configuration to the <strong>Debug</strong> menu and toolbar, click
the checkbox next to Debug.
</li>
<li>
To save your settings, click Close.<br/>
<p class="note"><strong>Note:</strong> Although you can run the test immediately by
clicking Run, you should save the test first and then
run it by selecting it from the Eclipse standard toolbar.</p>
</li>
<li>
On the Eclipse standard toolbar, click the down arrow next to the
green Run arrow. This displays a menu of saved Run and Debug
configurations.
</li>
<li>
Select the test run configuration you just created.
</li>
<li>
The progress of your test appears in the Console view.
You should see the following messages, among others:
<ul>
<li>
<code>Performing Android.test.InstrumentationTestRunner JUnit launch</code><br>
The class name that proceeds "JUnit" depends on the Android instrumentation
class you have chosen.
</li>
<li>
If you are using an emulator and you have not yet started it, then you will see
the message:
<p>
<code>Automatic Target Mode: launching new emulator with compatible
AVD <em>avdname</em></code><br>(where <em>avdname</em> is the name of
the AVD you are using.)
</p>
</li>
<li>
If you have not already installed your test application, then you will see
the message:
<p>
<code>Uploading <em>testclass</em>.apk onto device '<em>device-id</em>'</code><br>
where <em>testclass</em> is the name of your unit test class and <em>device-id</em>
is the name and port for your test device or emulator, followed by the message <code>Installing <em>testclass</em>.apk</code>
</p>
</li>
<li>
<code>Launching instrumentation Android.test.InstrumentationTestRunner on device <em>device-id</em></code>.<br>
This indicates that Android's Instrumentation system is now testing your code. Again, the
instrumentation class name depends on the Android instrumentation class you have chosen.
</li>
<li>
<code>Test run complete</code>.<br> When you see this, your unit tests have finished.
</li>
</ul>
</ol>
<p>
The test results appear in the JUnit view. This is divided into an upper summary pane,
and a lower stack trace pane.
</p>
<p>
The upper pane contains test information. In the pane's header, you see the following
information:
</p>
<ul>
<li>
Total time elapsed for the test application (labeled Finished after <em>x</em> seconds).
</li>
<li>
Number of runs (Runs:) - the number of tests in the entire test class.
</li>
<li>
Number of errors (Errors:) - the number of program errors and exceptions encountered
during the test run.
</li>
<li>
Number of failures (Failures:) - the number of test failures encountered during the test
run. This is the number of assertion failures. A test can fail even if the program does
not encounter an error.
</li>
<li>
A progress bar. The progress bar extends from left to right as the tests run. If all the
tests succeed, the bar remains green. If a test fails, the bar turns from green to red.
</li>
</ul>
<p>
The body of the upper pane contains the details of the test run. For each test case class
that was run, you see a line with the class name. To look at the results for the individual
test methods in that class, you click the left arrow to expand the line. You now see a
line for each test method in the class, and to its right the time it took to run.
If you double-click the method name, Eclipse opens the test class source in an editor view
pane and moves the focus to the first line of the test method.
</p>
<p>
The lower pane is for stack traces. If you highlight a failed test in the upper pane, the
lower pane contains a stack trace for the test. If a line corresponds to a point in your
test code, you can double-click it to display the code in an editor view pane, with the
line highlighted. For a successful test, the lower pane is empty.
</p>

View File

@@ -0,0 +1,683 @@
page.title=Testing In Other IDEs
@jd:body
<div id="qv-wrapper">
<div id="qv">
<h2>In this document</h2>
<ol>
<li>
<a href="#CreateTestProjectCommand">Working with Test Projects</a>
<ol>
<li>
<a href="#CreateTestProject">Creating a test project</a>
</li>
<li>
<a href="#UpdateTestProject">Updating a test project</a>
</li>
</ol>
</li>
<li>
<a href="#CreateTestApp">Creating a Test Application</a>
</li>
<li>
<a href="#RunTestsCommand">Running Tests</a>
<ol>
<li>
<a href="#RunTestsAnt">Quick build and run with Ant</a>
</li>
<li>
<a href="#RunTestsDevice">Running tests on a device or emulator</a>
</li>
</ol>
</li>
<li>
<a href="#AMSyntax">Using the Instrument Command</a>
<ol>
<li>
<a href="#AMOptionsSyntax">Instrument options</a>
</li>
<li>
<a href="#RunTestExamples">Instrument examples</a>
</li>
</ol>
</li>
</ol>
<h2>See Also</h2>
<ol>
<li>
<a
href="{@docRoot}guide/topics/testing/testing_android.html">Testing and Instrumentation</a>
</li>
<li>
<a href="{@docRoot}resources/tutorials/testing/activity_test.html">Activity Testing</a>
</li>
<li>
<a href="{@docRoot}guide/developing/tools/adb.html">Android Debug Bridge</a>
</li>
</ol>
</div>
</div>
<p>
This document describes how to create and run tests directly from the command line.
You can use the techniques described here if you are developing in an IDE other than Eclipse
or if you prefer to work from the command line. This document assumes that you already know how
to create a Android application in your programming environment. Before you start this
document, you should read the document <a
href="{@docRoot}guide/topics/testing/testing_android.html">Testing and Instrumentation</a>,
which provides an overview of Android testing.
</p>
<p>
If you are developing in Eclipse with ADT, you can set up and run your tests
directly in Eclipse. For more information, please read <a
href="{@docRoot}guide/developing/testing/testing_eclipse.html">Testing&nbsp;in&nbsp;Eclipse,&nbsp;with&nbsp;ADT</a>.
</p>
<h2 id="CreateTestProjectCommand">Working with Test Projects</h2>
<p>
You use the <code>android</code> tool to create test projects.
You also use <code>android</code> to convert existing test code into an Android test project,
or to add the <code>run-tests</code> Ant target to an existing Android test project.
These operations are described in more detail in the section <a
href="#UpdateTestProject">Updating a test project</a>.
The <code>run-tests</code> target is described in <a
href="#RunTestsAnt">Quick build and run with Ant</a>.
</p>
<h3 id="CreateTestProject">Creating a test project</h3>
<p>
To create a test project with the <code>android</code> tool, enter:
<pre>android create test-project -m &lt;main_path&gt; -n &lt;project_name&gt; -p &lt;test_path&gt;</pre>
<p>
You must supply all the flags. The following table explains them in detail:
</p>
<table>
<tr>
<th>Flag</th>
<th>Value</th>
<th>Description</th>
<tr>
<td><code>-m, --main</code></td>
<td>
Path to the project of the application under test, relative to the test application
directory.
</td>
<td>
For example, if the application under test is in <code>source/HelloAndroid</code>, and you
want to create the test project in <code>source/HelloAndroidTest</code>, then the value of
<code>--main</code> should be <code>../HelloAndroid</code>.
</td>
<tr>
<td><code>-n, --name</code></td>
<td>Name that you want to give the test project.</td>
<td>&nbsp;</td>
</tr>
<tr>
<td><code>-p, --path</code></td>
<td>Directory in which you want to create the new test project.</td>
<td>
The <code>android</code> tool creates the test project files and directory structure in this
directory. If the directory does not exist, <code>android</code> creates it.
</td>
</tr>
</table>
<p>
If the operation is successful, <code>android</code> lists to STDOUT the names of the files
and directories it has created.
</p>
<p>
This creates a new test project with the appropriate directories and build files. The directory
structure and build file contents are identical to those in a regular Android application
project. They are described in detail in the topic
<a href="{@docRoot}guide/developing/other-ide.html">Developing In Other IDEs</a>.
</p>
<p>
The operation also creates an <code>AndroidManifest.xml</code> file with instrumentation
information. When you run the test, Android uses this information to load the application you
are testing and control it with instrumentation.
</p>
<p>
For example, suppose you create the <a
href="{@docRoot}resources/tutorials/hello-world.html">Hello, World</a> tutorial application
in the directory <code>~/source/HelloAndroid</code>. In the tutorial, this application uses the
package name <code>com.example.helloandroid</code> and the activity name
<code>HelloAndroid</code>. You can to create the test for this in
<code>~/source/HelloAndroidTest</code>. To do so, you enter:
</p>
<pre>
$ cd ~/source
$ android create test-project -m ../HelloAndroid -n HelloAndroidTest -p HelloAndroidTest
</pre>
<p>
This creates a directory called <code>~/src/HelloAndroidTest</code>. In the new directory you
see the file <code>AndroidManifest.xml</code>. This file contains the following
instrumentation-related elements and attributes:
</p>
<ul>
<li>
<code>&lt;application&gt;</code>: to contain the
<code>&lt;uses-library&gt;</code> element.
</li>
<li>
<code>&lt;uses-library android:name=&quot;android.test.runner&quot;</code>:
specifies this testing application uses the <code>android.test.runner</code> library.
</li>
<li>
<code>&lt;instrumentation&gt;</code>: contains attributes that control Android
instrumentation. The attributes are:
<ul>
<li>
<code>android:name=&quot;android.test.InstrumentationTestRunner&quot;</code>:
{@link android.test.InstrumentationTestRunner} runs test cases. It extends both
JUnit test case runner classes and Android instrumentation classes.
</li>
<li>
<code>android:targetPackage=&quot;com.example.helloandroid&quot;</code>: specifies
that the tests in HelloAndroidTest should be run against the application with the
<em>Android</em> package name <code>com.example.helloandroid</code>. This is the
package name of the <a
href="{@docRoot}resources/tutorials/hello-world.html">Hello, World</a>
tutorial application.
</li>
<li>
<code>android:label=&quot;Tests for .HelloAndroid&quot;</code>: specifies a
user-readable label for the instrumentation class. By default,
the <code>android</code> tool gives it the value &quot;Tests for &quot; plus
the name of the main Activity of the application under test.
</li>
</ul>
</li>
</ul>
<h3 id="UpdateTestProject">Updating a test project</h3>
<p>
You use the <code>android</code> tool when you need to change the path to the
project of the application under test. If you are changing an existing test project created in
Eclipse with ADT so that you can also build and run it from the command line, you must use the
"create" operation. See the section <a href="#CreateTestProject">Creating a test project</a>.
</p>
<p class="note">
<strong>Note:</strong> If you change the Android package name of the application under test,
you must <em>manually</em> change the value of the <code>&lt;android:targetPackage&gt;</code>
attribute within the <code>AndroidManifest.xml</code> file of the test application.
Running <code>android update test-project</code> does not do this.
</p>
<p>
To update a test project with the <code>android</code> tool, enter:
</p>
<pre>android update-test-project -m &lt;main_path&gt; -p &lt;test_path&gt;</pre>
<table>
<tr>
<th>Flag</th>
<th>Value</th>
<th>Description</th>
</tr>
<tr>
<td><code>-m, --main</code></td>
<td>The path to the project of the application under test, relative to the test project</td>
<td>
For example, if the application under test is in <code>source/HelloAndroid</code>, and
the test project is in <code>source/HelloAndroidTest</code>, then the value for
<code>--main</code> is <code>../HelloAndroid</code>.
</td>
</tr>
<tr>
<td><code>-p, --path</code></td>
<td>The of the test project.</td>
<td>
For example, if the test project is in <code>source/HelloAndroidTest</code>, then the
value for <code>--path</code> is <code>HelloAndroidTest</code>.
</td>
</tr>
</table>
<p>
If the operation is successful, <code>android</code> lists to STDOUT the names of the files
and directories it has created.
</p>
<h2 id="CreateTestApp">Creating a Test Application</h2>
<p>
Once you have created a test project, you populate it with a test application.
The application does not require an {@link android.app.Activity Activity},
although you can define one if you wish. Although your test application can
combine Activities, Android test class extensions, JUnit extensions, or
ordinary classes, you should extend one of the Android test classes or JUnit classes,
because these provide the best testing features.
</p>
<p>
If you run your tests with {@link android.test.InstrumentationTestRunner}
(or a related test runner), then it will run all the methods in each class. You can modify
this behavior by using the {@link junit.framework.TestSuite TestSuite} class.
</p>
<p>
To create a test application, start with one of Android's test classes in the Java package
{@link android.test android.test}. These extend the JUnit
{@link junit.framework.TestCase TestCase} class. With a few exceptions, the Android test
classes also provide instrumentation for testing.
</p>
<p>
For test classes that extend {@link junit.framework.TestCase TestCase}, you probably want to
override the <code>setUp()</code> and <code>tearDown()</code> methods:
</p>
<ul>
<li>
<code>setUp()</code>: This method is invoked before any of the test methods in the class.
Use it to set up the environment for the test. You can use <code>setUp()</code>
to instantiate a new <code>Intent</code> object with the action <code>ACTION_MAIN</code>.
You can then use this intent to start the Activity under test.
<p class="note">
<strong>Note:</strong> If you override this method, call
<code>super.setUp()</code> as the first statement in your code.
</p>
</li>
<li>
<code>tearDown()</code>: This method is invoked after all the test methods in the class. Use
it to do garbage collection and re-setting before moving on to the next set of tests.
<p class="note"><strong>Note:</strong> If you override this method, you must call
<code>super.tearDown()</code> as the <em>last</em> statement in your code.</p>
</li>
</ul>
<p>
Another useful convention is to add the method <code>testPreConditions()</code> to your test
class. Use this method to test that the application under test is initialized correctly. If this
test fails, you know that that the initial conditions were in error. When this happens, further
test results are suspect, regardless of whether or not the tests succeeded.
</p>
<p>
To learn more about creating test applications, see the topic <a
href="{@docRoot}guide/topics/testing/testing_android.html">Testing and Instrumentation</a>,
which provides an overview of Android testing. If you prefer to follow a tutorial,
try the <a href="{@docRoot}resources/tutorials/testing/activity_test.html">Activity Testing</a>
tutorial, which leads you through the creation of tests for an actual Android application.
</p>
<h2 id="RunTestsCommand">Running Tests</h2>
<p>
If you are not developing in Eclipse with ADT, you need to run tests from the command line.
You can do this either with Ant or with the {@link android.app.ActivityManager ActivityManager}
command line interface.
</p>
<p>
You can also run tests from the command line even if you are using Eclipse with ADT to develop
them. To do this, you need to create the proper files and directory structure in the test
project, using the <code>android</code> tool with the option <code>create test-project</code>.
This is described in the section <a
href="#CreateTestProjectCommand">Working with Test Projects</a>.
</p>
<h3 id="RunTestsAnt">Quick build and run with Ant</h3>
<p>
You can use Ant to run all the tests in your test project, using the target
<code>run-tests</code>, which is created automatically when you create a test project with
the <code>android</code> tool.
</p>
<p>
This target re-builds your main project and test project if necessary, installs the test
application to the current AVD or device, and then runs all the test classes in the test
application. The results are directed to <code>STDOUT</code>.
</p>
<p>
You can update an existing test project to use this feature. To do this, use the
<code>android</code> tool with the <code>update test-project</code> option. This is described
in the section <a href="#UpdateTestProject">Updating a test project</a>.
<h3 id="RunTestsDevice">Running tests on a device or emulator</h3>
<p>
When you run tests from the command line with the ActivityManager (<code>am</code>)
command-line tool, you get more options for choosing the tests to run than with any other
method. You can select individual test methods, filter tests according to their annotation, or
specify testing options. Since the test run is controlled entirely from a command line, you can
customize your testing with shell scripts in various ways.
</p>
<p>
You run the <code>am</code> tool on an Android device or emulator using the
<a href="{@docRoot}guide/developing/tools/adb.html">Android Debug Bridge</a>
(<code>adb</code>) shell. When you do this, you use the ActivityManager
<code>instrument</code> option to run your test application using an Android test runner
(usually {@link android.test.InstrumentationTestRunner}). You set <code>am</code>
options with command-line flags.
</p>
<p>
To run a test with <code>am</code>:
</p>
<ol>
<li>
If necessary, re-build your main application and test application.
</li>
<li>
Install your test application and main application Android package files
(<code>.apk</code> files) to your current Android device or emulator</li>
<li>
At the command line, enter:
<pre>
$ adb shell am instrument -w &lt;test_package_name&gt;/&lt;runner_class&gt;
</pre>
<p>
where <code>&lt;test_package_name&gt;</code> is the Android package name of your test
application, and <code>&lt;runner_class&gt;</code> is the name of the Android test runner
class you are using. The Android package name is the value of the <code>package</code>
attribute of the <code>manifest</code> element in the manifest file
(<code>AndroidManifest.xml</code>) of your test application. The Android test runner
class is usually <code>InstrumentationTestRunner</code>.
</p>
<p>Your test results appear in <code>STDOUT</code>.</p>
</li>
</ol>
<p>
This operation starts an <code>adb</code> shell, then runs <code>am instrument</code> in it
with the specified parameters. This particular form of the command will run all of the tests
in your test application. You can control this behavior with flags that you pass to
<code>am instrument</code>. These flags are described in the next section.
</p>
<h2 id="AMSyntax">Using the Instrument Command</h2>
<p>
The general syntax of the <code>am instrument</code> command is:
</p>
<pre>
am instrument [flags] &lt;test_package&gt;/&lt;runner_class&gt;
</pre>
<p>
The main input parameters to <code>am instrument</code> are described in the following table:
</p>
<table>
<tr>
<th>
Parameter
</th>
<th>
Value
</th>
<th>
Description
</th>
</tr>
<tr>
<td>
<code>&lt;test_package&gt;</code>
</td>
<td>
The Android package name of the test application.
</td>
<td>
The value of the <code>package</code> attribute of the <code>manifest</code>
element in the test application's manifest file.
</td>
</tr>
<tr>
<td>
<code>&lt;runner_class&gt;</code>
</td>
<td>
The class name of the instrumented test runner you are using.
</td>
<td>
This is usually {@link android.test.InstrumentationTestRunner}.
</td>
</tr>
</table>
<p>
The flags for <code>am instrument</code> are described in the following table:
</p>
<table>
<tr>
<th>
Flag
</th>
<th>
Value
</th>
<th>
Description
</th>
</tr>
<tr>
<td>
<code>-w</code>
</td>
<td>
(none)
</td>
<td>
Forces <code>am instrument</code> to wait until the instrumentation terminates
before terminating itself. The net effect is to keep the shell open until the tests
have finished. This flag is not required, but if you do not use it, you will not
see the results of your tests.
</td>
</tr>
<tr>
<td>
<code>-r</code>
</td>
<td>
(none)
</td>
<td>
Outputs results in raw format. Use this flag when you want to collect
performance measurements, so that they are not formatted as test results. This flag is
designed for use with the flag <code>-e perf true</code> (documented in the section
<a href="#AMOptionsSyntax">Instrument options</a>).
</td>
</tr>
<tr>
<td>
<code>-e</code>
</td>
<td>
&lt;test_options&gt;
</td>
<td>
Provides testing options , in the form of key-value pairs. The
<code>am instrument</code> tool passes these to the specified instrumentation class
via its <code>onCreate()</code> method. You can specify multiple occurrences of
<code>-e &lt;test_options</code>. The keys and values are described in the next table.
<p>
The only instrumentation class that understands these key-value pairs is
<code>InstrumentationTestRunner</code> (or a subclass). Using them with
any other class has no effect.
</p>
</td>
</tr>
</table>
<h3 id="AMOptionsSyntax">Instrument options</h3>
<p>
The <code>am instrument</code> tool passes testing options to
<code>InstrumentationTestRunner</code> or a subclass in the form of key-value pairs,
using the <code>-e</code> flag, with this syntax:
</p>
<pre>
-e &lt;key&gt; &lt;value&gt;
</pre>
<p>
Where applicable, a &lt;key&gt; may have multiple values separated by a comma (,).
For example, this invocation of <code>InstrumentationTestRunner</code> provides multiple
values for the <code>package</code> key:
<pre>
$ adb shell am instrument -w -e package com.android.test.package1,com.android.test.package2 com.android.test/android.test.InstrumentationTestRunner
</pre>
<p>
The following table describes the key-value pairs and their result. Please review the
<strong>Usage Notes</strong> following the table.
</p>
<table>
<tr>
<th>Key</th>
<th>Value</th>
<th>Description</th>
</tr>
<tr>
<td>
<code>package</code>
</td>
<td>
&lt;Java_package_name&gt;
</td>
<td>
The fully-qualified <em>Java</em> package name for one of the packages in the test
application. Any test case class that uses this package name is executed. Notice that this
is not an <em>Android</em> package name; a test application has a single Android package
name but may have several Java packages within it.
</td>
</tr>
<tr>
<td rowspan="2"><code>class</code></td>
<td>&lt;class_name&gt;</td>
<td>
The fully-qualified Java class name for one of the test case classes. Only this test case
class is executed.
</td>
</tr>
<tr>
<td>&lt;class_name&gt;<strong>#</strong>method name</td>
<td>
A fully-qualified test case class name, and one of its methods. Only this method is
executed. Note the hash mark (#) between the class name and the method name.
</td>
</tr>
<tr>
<td><code>func</code></td>
<td><code>true</code></td>
<td>
Runs all test classes that extend {@link android.test.InstrumentationTestCase}.
</td>
</tr>
<tr>
<td><code>unit</code></td>
<td><code>true</code></td>
<td>
Runs all test classes that do <em>not</em> extend either
{@link android.test.InstrumentationTestCase} or {@link android.test.PerformanceTestCase}.
</td>
</tr>
<tr>
<td><code>size</code></td>
<td>[<code>small</code> | <code>medium</code> | <code>large</code>]
</td>
<td>
Runs a test method annotated by size. The annotations are <code>@SmallTest</code>,
<code>@MediumTest</code>, and <code>@LargeTest</code>.
</td>
</tr>
<tr>
<td><code>perf</code></td>
<td><code>true</code></td>
<td>
Runs all test classes that implement {@link android.test.PerformanceTestCase}.
When you use this option, also specify the <code>-r</code> flag for
<code>am instrument</code>, so that the output is kept in raw format and not
re-formatted as test results.
</td>
</tr>
<tr>
<td><code>debug</code></td>
<td><code>true</code></td>
<td>
Runs tests in debug mode.
</td>
</tr>
<tr>
<td><code>log</code></td>
<td><code>true</code></td>
<td>
Loads and logs all specified tests, but does not run them. The test
information appears in <code>STDOUT</code>. Use this to verify combinations of other filters
and test specifications.
</td>
</tr>
<tr>
<td><code>emma</code></td>
<td><code>true</code></td>
<td>
Runs an EMMA code coverage analysis and writes the output to <code>/data//coverage.ec</code>
on the device. To override the file location, use the <code>coverageFile</code> key that
is described in the following entry.
<p class="note">
<strong>Note:</strong> This option requires an EMMA-instrumented build of the test
application, which you can generate with the <code>coverage</code> target.
</p>
</td>
</tr>
<tr>
<td><code>coverageFile</code></td>
<td><code>&lt;filename&gt;</code></td>
<td>
Overrides the default location of the EMMA coverage file on the device. Specify this
value as a path and filename in UNIX format. The default filename is described in the
entry for the <code>emma</code> key.
</td>
</tr>
</table>
<strong><code>-e</code> Flag Usage Notes</strong>
<ul>
<li>
<code>am instrument</code> invokes
{@link android.test.InstrumentationTestRunner#onCreate(Bundle)}
with a {@link android.os.Bundle} containing the key-value pairs.
</li>
<li>
The <code>package</code> key takes precedence over the <code>class</code> key. If you
specifiy a package, and then separately specify a class within that package, Android
will run all the tests in the package and ignore the <code>class</code> key.
</li>
<li>
The <code>func</code> key and <code>unit</code> key are mutually exclusive.
</li>
</ul>
<h3 id="RunTestExamples">Instrument examples</h3>
<p>
Here are some examples of using <code>am instrument</code> to run tests. They are based on
the following structure:</p>
<ul>
<li>
The test application has the Android package name <code>com.android.demo.app.tests</code>
</li>
<li>
There are three test classes:
<ul>
<li>
<code>UnitTests</code>, which contains the methods
<code>testPermissions</code> and <code>testSaveState</code>.
</li>
<li>
<code>FunctionTests</code>, which contains the methods
<code>testCamera</code>, <code>testXVGA</code>, and <code>testHardKeyboard</code>.
</li>
<li>
<code>IntegrationTests</code>,
which contains the method <code>testActivityProvider</code>.
</li>
</ul>
</li>
<li>
The test runner is {@link android.test.InstrumentationTestRunner}.
</li>
</ul>
<h4>Running the Entire Test Application</h4>
<p>
To run all of the test classes in the test application, enter:
</p>
<pre>
$ adb shell am instrument -w com.android.demo.app.tests/android.test.InstrumentationTestRunner
</pre>
<h4>Running All Tests in a Test Case Class</h4>
<p>
To run all of the tests in the class <code>UnitTests</code>, enter:
</p>
<pre>
$ adb shell am instrument -w \
-e class com.android.demo.app.tests.UnitTests \
com.android.demo.app.tests/android.test.InstrumentationTestRunner
</pre>
<p>
<code>am instrument</code> gets the value of the <code>-e</code> flag, detects the
<code>class</code> keyword, and runs all the methods in the <code>UnitTests</code> class.
</p>
<h4>Selecting a Subset of Tests</h4>
<p>
To run all of the tests in <code>UnitTests</code>, and the <code>testCamera</code> method in
<code>FunctionTests</code>, enter:
</p>
<pre>
$ adb shell am instrument -w \
-e class com.android.demo.app.tests.UnitTests,com.android.demo.app.tests.FunctionTests#testCamera \
com.android.demo.app.tests/android.test.InstrumentationTestRunner
</pre>
<p>
You can find more examples of the command in the documentation for
{@link android.test.InstrumentationTestRunner}.
</p>

View File

@@ -274,6 +274,25 @@
<li><a href="<?cs var:toroot ?>guide/developing/debug-tasks.html"> <li><a href="<?cs var:toroot ?>guide/developing/debug-tasks.html">
<span class="en">Debugging Tasks</span> <span class="en">Debugging Tasks</span>
</a></li> </a></li>
<li class="toggle-list">
<div>
<a href="<?cs var:toroot ?>guide/developing/testing/index.html">
<span class="en">Testing</span>
</a> <span class="new">new!</span>
</div>
<ul>
<li>
<a href="<?cs var:toroot ?>guide/developing/testing/testing_eclipse.html">
<span class="en">Testing in Eclipse, with ADT</span>
</a> <span class="new">new!</span>
</li>
<li>
<a href="<?cs var:toroot ?>guide/developing/testing/testing_otheride.html">
<span class="en">Testing in Other IDEs</span>
</a> <span class="new">new!</span>
</li>
</ul>
</li>
<li class="toggle-list"> <li class="toggle-list">
<div><a href="<?cs var:toroot ?>guide/developing/tools/index.html"> <div><a href="<?cs var:toroot ?>guide/developing/tools/index.html">
<span class="en">Tools</span> <span class="en">Tools</span>

View File

@@ -51,10 +51,6 @@ name specified in the <code><a href="{@docRoot}guide/topics/manifest/manifest-el
There is no default. The name must be specified. There is no default. The name must be specified.
</p></dd> </p></dd>
<!-- ##api level indication## -->
<dt>introduced in:</dt>
<dd>API Level 1</dd>
<dt><a name="trgt"></a>{@code android:targetPackage}</dt> <dt><a name="trgt"></a>{@code android:targetPackage}</dt>
<dd>The application that the Instrumentation object will run against. <dd>The application that the Instrumentation object will run against.
An application is identified by the package name assigned in its manifest An application is identified by the package name assigned in its manifest
@@ -62,4 +58,8 @@ file by the <code><a href="{@docRoot}guide/topics/manifest/manifest-element.html
</dl></dd> </dl></dd>
<!-- ##api level indication## -->
<dt>introduced in:</dt>
<dd>API Level 1</dd>
</dl> </dl>

View File

@@ -1,5 +1,6 @@
page.title=Testing and Instrumentation page.title=Testing and Instrumentation
@jd:body @jd:body
<div id="qv-wrapper"> <div id="qv-wrapper">
<div id="qv"> <div id="qv">
<h2>In this document</h2> <h2>In this document</h2>
@@ -72,70 +73,92 @@ page.title=Testing and Instrumentation
<h2>See Also</h2> <h2>See Also</h2>
<ol> <ol>
<li> <li>
<a href="{@docRoot}guide/developing/testing_eclipse.html">Testing in Eclipse, with ADT</a> <a href="{@docRoot}guide/developing/testing/testing_eclipse.html">Testing in Eclipse, with ADT</a>
</li> </li>
<li> <li>
<a href="{@docRoot}guide/developing/testing_otheride.html">Testing in Other IDEs</a> <a href="{@docRoot}guide/developing/testing/testing_otheride.html">Testing in Other IDEs</a>
</li> </li>
</ol> </ol>
</div> </div>
</div> </div>
<p>
Android includes a powerful set of testing tools that extend the industry-standard JUnit test framework with features specific to the Android environment. Although you can <p>Android includes a powerful set of testing tools that extend the
test an Android application with JUnit, the Android tools allow you to write much more sophisticated tests for every aspect of your application, both at the unit and at the framework level. industry-standard JUnit test framework with features specific to the Android
</p> environment. Although you can test an Android application with JUnit, the
<p> Android tools allow you to write much more sophisticated tests for every aspect
Key features of the Android testing environment include: of your application, both at the unit and framework levels.</p>
</p>
<p>Key features of the Android testing environment include:</p>
<ul> <ul>
<li>Android extensions to the JUnit framework that provide access to Android system objects.</li> <li>Android extensions to the JUnit framework that provide access to Android
<li>An instrumentation framework that lets tests control and examine the application.</li> system objects.</li>
<li>An instrumentation framework that lets tests control and examine the
application.</li>
<li>Mock versions of commonly-used Android system objects.</li> <li>Mock versions of commonly-used Android system objects.</li>
<li>Tools for running single tests or test suites, with or without instrumentation.</li> <li>Tools for running single tests or test suites, with or without
<li>Support for managing tests and test projects in the ADT Plugin for Eclipse and at the command line.</li> instrumentation.</li>
<li>Support for managing tests and test projects in the ADT Plugin for Eclipse
and at the command line.</li>
</ul> </ul>
<p>
This document is an overview of the Android testing environment and the way you use it. The document assumes you have a basic knowledge of <p>This document is an overview of the Android testing environment and the way
Android application programming and JUnit testing methodology. you use it. The document assumes you have a basic knowledge of Android
</p> application programming and JUnit testing methodology.</p>
<h2 id="Overview">Overview</h2> <h2 id="Overview">Overview</h2>
<p>
At the heart of the Android testing environment is an instrumentation framework that your test application uses to precisely control the application under test. With instrumentation, you can <p> At the heart of the Android testing environment is an instrumentation
set up mock system objects such as Contexts before the main application starts, control your application at various points of its lifecycle, send UI events to the application, and framework that your test application uses to precisely control the application
examine the application's state during its execution. The instrumentation framework accomplishes this by running both the main application and the test application in the same process. under test. With instrumentation, you can set up mock system objects such as
</p> Contexts before the main application starts, control your application at various
<p> points of its lifecycle, send UI events to the application, and examine the
Your test application's manifest file links it to the application under test. The the &lt;instrumentation&gt; attribute in the manifest file points to the application under test application's state during its execution. The instrumentation framework
and also tells Android how to run the test application. This is described in more detail in the section <a href="#InstrumentationTestRunner">Instrumentation Test Runner</a>. accomplishes this by running both the main application and the test application
</p> in the same process. </p>
<p>
The following diagram summarizes the Android testing environment: <p>Your test application is linked to the application under test by means of an
</p> <a
href="{@docRoot}guide/topics/manifest/instrumentation-element.html"><code>&lt;instrumentation&gt;</code></a>
element in the test application's manifest file. The attributes of the element
specify the package name of the application under test and also tell Android how
to run the test application. Instrumentation is described in more detail in the
section <a href="#InstrumentationTestRunner">Instrumentation Test
Runner</a>.</p>
<p>The following diagram summarizes the Android testing environment:</p>
<img src="{@docRoot}images/testing/android_test_framework.png"/> <img src="{@docRoot}images/testing/android_test_framework.png"/>
<!-- Something about test tools in Eclipse and in other IDEs -->
<p> <p>In Android, test applications are themselves Android applications, so you
In Android, test applications are themselves Android applications, so you write them in much the same way as the application you are testing. The SDK tools write them in much the same way as the application you are testing. The SDK
help you create a main application project and its test project at the same time. You can run Android tests within Eclipse with ADT or from the command line. tools help you create a main application project and its test project at the same
Eclipse with ADT provides an extensive set of tools for creating tests, running them, and viewing their results. You can also use the <code>adb</code> tool to time. You can run Android tests within Eclipse with ADT or from the command
run tests, or use a built-in Ant target. line. Eclipse with ADT provides an extensive set of tools for creating tests,
</p> running them, and viewing their results. You can also use the <code>adb</code>
<p> tool to run tests, or use a built-in Ant target.</p>
To learn how to set up and run tests in Eclipse, please refer to <a href="{@docRoot}guide/developing/testing_eclipse.html">Testing in Eclipse, with ADT</a>.
If you're not working in Eclipse, refer to <a href="{@docRoot}guide/developing/testing_otheride.html">Testing in Other IDEs</a>. <p>To learn how to set up and run tests in Eclipse, please refer to <a
</p> href="{@docRoot}guide/developing/testing/testing_eclipse.html">Testing in
<p> Eclipse, with ADT</a>. If you're not working in Eclipse, refer to <a
If you want a step-by-step introduction to Android testing, try one of the testing tutorials: href="{@docRoot}guide/developing/testing/testing_otheride.html">Testing in Other
</p> IDEs</a>.</p>
<ul>
<li> <p>If you want a step-by-step introduction to Android testing, try one of the
The <a href="{@docRoot}resources/tutorials/testing/helloandroid_test.html">Hello, Testing</a> tutorial introduces basic testing concepts and procedures in the testing tutorials:</p>
context of the Hello, World application.
</li> <ul>
<li> <li>The <a
The <a href="{@docRoot}resources/tutorials/testing/activity_test.html">Activity Testing</a> tutorial is an excellent follow-up to the Hello, Testing tutorial. href="{@docRoot}resources/tutorials/testing/helloandroid_test.html">Hello,
It guides you through a more complex testing scenario that you develop against a more realistic application. Testing</a> tutorial introduces basic testing concepts and procedures in the
</li> context of the Hello, World application.</li>
</ul> <li>The <a
href="{@docRoot}resources/tutorials/testing/activity_test.html">Activity
Testing</a> tutorial is an excellent follow-up to the Hello, Testing tutorial.
It guides you through a more complex testing scenario that you develop against a
more realistic application.</li>
</ul>
<h2 id="TestAPI">The Testing API</h2> <h2 id="TestAPI">The Testing API</h2>
<p> <p>
For writing tests and test applications in the Java programming language, Android provides a For writing tests and test applications in the Java programming language, Android provides a
@@ -277,15 +300,15 @@ The following diagram summarizes the Android testing environment:
</p> </p>
<p> <p>
The tools and procedures you use with testing depend on the development environment you are using. If you use Eclipse, then the ADT plug in for Eclipse provides tools that The tools and procedures you use with testing depend on the development environment you are using. If you use Eclipse, then the ADT plug in for Eclipse provides tools that
allow you to develop and run tests entirely within Eclipse. This is documented in the topic <a href="{@docRoot}guide/developing/testing_eclipse.html">Testing in Eclipse, with ADT</a>. allow you to develop and run tests entirely within Eclipse. This is documented in the topic <a href="{@docRoot}guide/developing/testing/testing_eclipse.html">Testing in Eclipse, with ADT</a>.
If you use another development environment, then you use Android's command-line tools, as documented in the topic <a href="{@docRoot}guide/developing/testing_otheride.html">Testing in Other IDEs</a>. If you use another development environment, then you use Android's command-line tools, as documented in the topic <a href="{@docRoot}guide/developing/testing/testing_otheride.html">Testing in Other IDEs</a>.
</p> </p>
<h3 id="TestProjects">Working with test projects</h3> <h3 id="TestProjects">Working with test projects</h3>
<p> <p>
To start testing an Android application, you create a test project for it using Android tools. The tools create the project directory and the files and subdirectories needed. To start testing an Android application, you create a test project for it using Android tools. The tools create the project directory and the files and subdirectories needed.
The tools also create a manifest file that links the application in the test project to the application under test. The procedure for creating a test project in Eclipse with The tools also create a manifest file that links the application in the test project to the application under test. The procedure for creating a test project in Eclipse with
ADT is documented in <a href="{@docRoot}guide/developing/testing_eclipse.html">Testing in Eclipse, with ADT</a>. The procedure for creating a test project for use with development ADT is documented in <a href="{@docRoot}guide/developing/testing/testing_eclipse.html">Testing in Eclipse, with ADT</a>. The procedure for creating a test project for use with development
tools other than Eclipse is documented in <a href="{@docRoot}guide/developing/testing_otheride.html">Testing in Other IDEs</a>. tools other than Eclipse is documented in <a href="{@docRoot}guide/developing/testing/testing_otheride.html">Testing in Other IDEs</a>.
</p> </p>
<h3 id="TestClasses">Working with test case classes</h3> <h3 id="TestClasses">Working with test case classes</h3>
<p> <p>

View File

@@ -86,7 +86,7 @@ page.title=Hello, Testing
</p> </p>
<p> <p>
If you aren't developing in Eclipse with ADT or you would like to run tests directly from the If you aren't developing in Eclipse with ADT or you would like to run tests directly from the
command line, please see the topic <a href="{@docRoot}guide/developing/testing_otheride.html">Testing in Other IDEs</a> command line, please see the topic <a href="{@docRoot}guide/developing/testing/testing_otheride.html">Testing in Other IDEs</a>
for instructions. for instructions.
</p> </p>
<h2 id="CreateTestProject">Creating the Test Project</h2> <h2 id="CreateTestProject">Creating the Test Project</h2>