Commit Graph

72 Commits

Author SHA1 Message Date
Remi NGUYEN VAN
6686257924 Add --no-hidden-api-checks to runtest
New test tools are adding --no-hidden-api-checks when tests have
hidden-api-checks=false in their AndroidTest.xml, however this is not
supported by runtest. As only runtest is available in AOSP, this leaves
no option to make tests of internal classes pass.

Add an option to disable the check in runtest so we can still run tests in
AOSP.

Test: runtest frameworks-net still fails
Test: runtest --no-hidden-api-checks frameworks-net now passes
Change-Id: I9be9674d0bf2588ff509121273379f08d4cb3aea
2018-06-06 10:49:08 +09:00
Dan Willemsen
7d5963c0af Merge "runtest: Fix mmm equivalent"
am: 0ba0482cc5

Change-Id: I0b25811ec3371b3aca29650e2577cc789786c512
2016-08-15 21:17:40 +00:00
Dan Willemsen
c794996b46 runtest: Fix mmm equivalent
This was installing all modules defined in Soong, since we always load
the Soong-generated Makefile for mm/mmm. So use MODULES-IN-* instead of
all_modules.

Also switch to the new GET-INSTALL-PATH-IN-* arguments.

Bug: 30191725
Change-Id: I8a660e24c9933f3f77eb23efd71f8e374a248576
2016-08-11 17:38:23 -07:00
Stuart Scott
ed4c4e800d Merge "Remove CTS tests from runtest.py" 2015-09-28 16:39:19 +00:00
Ying Wang
7baf5da7b9 am ab5b0e57: am f2e32c0c: am e4cded63: Merge "Substitue "/" with "-" in the MODULES-IN phony goal names."
* commit 'ab5b0e5786f42d9fdd6c46637f5da21b88d34f67':
  Substitue "/" with "-" in the MODULES-IN phony goal names.
2015-09-25 00:14:57 +00:00
Ying Wang
f2e32c0c9d am e4cded63: Merge "Substitue "/" with "-" in the MODULES-IN phony goal names."
* commit 'e4cded631dae5fb121891f3ff9a9d72075f21b0e':
  Substitue "/" with "-" in the MODULES-IN phony goal names.
2015-09-24 23:59:26 +00:00
Ying Wang
21d83c6817 Substitue "/" with "-" in the MODULES-IN phony goal names.
Bug: 24309760
Change-Id: I2372d0be0a811b11f3000646a2b7804e93852a84
2015-09-24 16:35:36 -07:00
Ying Wang
32bf15ea7e am 7eca6be6: am f797dc2b: am 944edb56: Merge "Switch to the new mmma implementation using make goals."
* commit '7eca6be6c26f3ba74f4bdf76c17fba33b02dce2a':
  Switch to the new mmma implementation using make goals.
2015-09-24 04:17:35 +00:00
Ying Wang
f797dc2b92 am 944edb56: Merge "Switch to the new mmma implementation using make goals."
* commit '944edb562b67bdf8e052bb292629afd365086df1':
  Switch to the new mmma implementation using make goals.
2015-09-24 02:15:42 +00:00
Ying Wang
1b06fea486 Switch to the new mmma implementation using make goals.
Bug: 24309760
Change-Id: Ia79bbdcd396bd5cadf3deac27cb121b9a816d465
2015-09-23 16:56:25 -07:00
Stuart Scott
be975e3911 Remove CTS tests from runtest.py
bug:24006796
bug:21762834
Change-Id: Ic321ac8871d85cbc0370e09a7eda3826da55b260
2015-09-15 10:38:44 -07:00
Xiaohui Chen
fc784a4e31 runtest: add --install-filter parameter
--install-filter parameter is a regular expression that will be used
to match the compiled apks to determine if it needs to be installed.
This is very helpful when there are many apks in a test directory,
but you only care about one during development.
e.g. fw/base/core/tests/coretests

Change-Id: I6a6a04640e549be4d90c9c659e557e7870af6771
2015-08-25 12:45:59 -07:00
Xiaohui Chen
85586e0e03 runtest: introduce --user parameter
Currently runtest script installs and then runs test apks with default
"adb install" and "adb shell am instrument" behavior.  They are actually
different,  "adb install" uses USER_ALL, "am instrument" uses
USER_CURRENT.  This cl introduces a way to specify exactly which user a
test should run with in a multiple user environment.  This also provides
a work around for b/22976637 for now.

Bug: 19913735
Bug: 22976637
Change-Id: I65e2b2c79768734a8bc9261f8c4693c9f89016b1
2015-08-24 22:11:59 -07:00
Santos Cordon
4e0ad8f810 Add flag to grant permissions when running tests.
This change automatically adds -g to all adb install commands
for runtest. It also allows users to:
1) Specify '-k' as a command line arg to avoid granting permissions.
2) Declare a test suite as 'requiring permissions' within test_defs

Also fixes a potential bug where continuous="false" would translate
to True in the python code.

Bug: 21327566
Change-Id: I62d14e8106174861397b3fd8caa3e567e3538713
2015-05-21 15:46:59 -07:00
Ruben Brunk
e5aa0f3fa8 runtest: Remove unused CtsTestStubs dependency.
Change-Id: Icee3004731ab5d37e8eb2caba76b19b0a5f1b1b7
2014-10-02 13:08:24 -07:00
Jean-Luc Brouillet
3bc1e46683 Fix directory where CtsTestStubs.apk is found.
Empirically, it's now found in a subdirectory named CtsTestStubs.

Change-Id: Iae50c37ac7655f0b32cc2cc5a1b50371d1397b54
2014-07-21 16:09:31 -07:00
Brett Chabot
8b538bd847 Temporarily turn off runtest's disabling of dalvik verifier
Bug: 14088982
Change-Id: Iad2610fde7c9bd7c52a25fc1a1f5cddff6bc43e4
2014-04-22 11:46:00 -07:00
Igor Murashkin
8d703533ba runtest: add --build-install-only/-i flag to skip test execution
Change-Id: I3033b442e33a69442b881f0a5a7b9e2299d35d54
2014-01-24 16:03:11 -08:00
Igor Murashkin
a0afc8cfb2 runtest: Update to work with emulator and to build dependencies
* Combine dalvik.vm.dexopt-flags with existing to disable verification
* Always do a runtime reset (stop;start) instead of rebooting device
* Build dependencies (mma) if regular build fails (mm)

Change-Id: I5e9fcebeea052c3314d1f4146b074637c329c0a0
2014-01-24 16:03:06 -08:00
Brett Chabot
5f5928cc42 Find install artifacts using GET-INSTALL-PATH.
Change-Id: I7eb3229885df814ab25eb445f40963859b163bf1
2013-08-20 17:07:52 -07:00
JP Abgrall
f38107c03f testrunner: don't hardcode "out"
"out" can actually be $OUT_DIR.
"out/host/arch" is actually $ANDROID_HOST_OUT.

Bug: 7183057
Change-Id: I91ed7fdd57da92288f0529ba60cef1520b6d31d3
2013-07-11 17:39:16 -07:00
Brett Chabot
e607d3a8a4 Attempt to fix runtest for native tests.
Create dest dir before pushing
Use correct exec path data/nativetest/test/test

Change-Id: Ib2555c0e98ea5e13fdf6bf24697bd1b82d74b681
2013-05-16 23:06:25 -07:00
Brett Chabot
a75e8a374a am 834218e7: am e1bb1d3b: Merge "Make coverage work without test defs." into jb-mr1-dev
* commit '834218e7fb82ddb86375682c2fba3bd24074a427':
  Make coverage work without test defs.
2012-09-24 00:48:50 -07:00
Brett Chabot
e1bb1d3be9 Merge "Make coverage work without test defs." into jb-mr1-dev 2012-09-21 12:33:55 -07:00
Brett Chabot
3dd6d288f5 am f5e9dcd8: am 42c14447: Merge "Fix install of full build packages." into jb-mr1-dev
* commit 'f5e9dcd82939e5077d003bbcbfd13752debf857b':
  Fix install of full build packages.
2012-09-20 09:29:01 -07:00
Brett Chabot
8ac51186a6 Make coverage work without test defs.
Change-Id: I946df038e97dc5c2f40a4610d4076e13ab6bde37
2012-09-20 08:06:05 -07:00
Geremy Condra
5df6da94db Merge "Fixed runtest to work with python != 2.4" 2012-09-20 00:19:44 -07:00
Brett Chabot
b0b8c7816d Fix install of full build packages.
Change-Id: I8296304b2297d3ac746920f5f2f70096d055a46c
2012-09-19 08:32:56 -07:00
Brett Chabot
81c475e4ac Fix log syntax.
And add better error message when instrumentation test is not
present.

Change-Id: I381421e59a3b667a5c3090ccd49e38fe99d680c5
2012-09-13 09:47:25 -07:00
Geremy Condra
01844b15d9 Fixed runtest to work with python != 2.4
Change-Id: I6104adb5a6549eb4eaa3f2096633a49e4743ce50
2012-09-11 15:24:25 -07:00
Brett Chabot
7454171f17 Coverage and unbundling changes for runtest.
Stop using 'adb sync' in runtest. It was unreliable even when working
with full platform builds, and doesn't work at all for unbundled apps.
Instead, use output from build command to find produced artifacts and
use 'adb install' where possible.
However, note that this approach won't sync previously built artifacts
to device.

Also adjust to build system support for code coverage. Its no longer
required to build libcore to get code coverage.

Change-Id: I9c5d37897c9570d2d29db3ec82f5c53e60a8f485
2012-08-31 18:55:15 -07:00
Mike Lockwood
825502444a Merge commit '12db436d227ad9b97cb3d13812e1175cf9be8ed6'
Increase default runtest -j to 16.

Change-Id: I17adc7c585559ffbd6612b6bf9276cb3c5131f83
2012-03-21 15:29:52 -07:00
Brian Muramatsu
83f4573b4a Fixes for runtest
- No need to include cts/CtsBuild.mk, because that is included by the
  core build system.

- Include the tradefed-prebuilt.jar on the classpath when running
  host side tests.

- Update some of test_defs that have changed. Remove suites that
  have been deleted.

Change-Id: Ie2845db7f209ad4ccb881d2ad47b9415c9f518a4
2012-03-08 11:50:17 -08:00
Brett Chabot
25dfd79b2e Fix runtest local.prop handling.
Change-Id: Iec8d942fc11d46bdd54f55485aedb4b0021afee3
2012-02-17 15:58:01 -08:00
Brett Chabot
12db436d22 Increase default runtest -j to 16.
Change-Id: I4f98a630a7ad75557e462a122b83dd3c36cf6ab6
2012-01-17 16:04:53 -08:00
Brian Muramatsu
95cbc9e1ec Fix runtest for CTS Tests
The CTS packages use CTS-specific Makefiles rules like
BUILD_CTS_PACKAGE instead of BUILD_PACKAGE. Update runtest
to include CtsBuild.mk which defines all of these.

Change-Id: I6ea6c77b4f2ab113126319141e5120e016055c9d
2012-01-10 12:06:12 -08:00
Brett Chabot
0e5c937ff7 Merge "Fix runtest build step." 2011-06-08 13:56:42 -07:00
Brett Chabot
546a3285ce Fix runtest build step.
Update make syntax to new 'mmm' logic.

Bug 4556556

Change-Id: Ie1fd5f6714ee7ef731b063751e0ddd674bc31021
2011-06-07 19:21:11 -07:00
Brett Chabot
cdfaae1a34 Make runtest reboot and root more robust.
Add a couple sleep statements to allow time for device to go offline.

Change-Id: I3a5fffc948f6c7b67bcdec4526990a6becd2d02f
2011-06-07 10:16:02 -07:00
Brett Chabot
3c9cefc70f Remove runtest hard dependency on test_defs.xml files.
Previously runtest would fail hard if development/test_runner/test_defs.xml
file was not present, even if it wasn't needed (ie running in --path mode).

This commit skips test_defs parsing if --path is specified, and only parses
development/test_runner/test_defs.xml if its present.

Bug 4556556

Change-Id: Ief2dbbdab5c7e82e5db64414392d6e2a564e815d
2011-06-07 08:27:36 -07:00
Brett Chabot
b1eb5d2434 resolved conflicts for merge of fa6e5585 to master
Change-Id: I967289246c3472a30f4623581226cbc3054f0b56
2010-06-15 11:18:42 -07:00
Brett Chabot
ccae47d0c0 Workaround for package-private framework tests in runtest.
Checks if the test to run is a framework test based on path, and if so
disables the davlik verifier by writing to /data/local.prop and
rebooting device.

Change-Id: Ibf94fb921662b5ddf6136dd12360d5de83a3a284
2010-06-14 15:24:42 -07:00
Guang Zhu
863870c168 code coverage improvement (DO NOT MERGE)
* automatically detect if emma is on device
* build emma into libcore if necessary
* stop runtime before adb sync and restart afterwards

Change-Id: Id5087fdd6c0345f2b9b76658a5083b9260af061e
2010-05-18 10:54:47 -07:00
Guang Zhu
8aff3602e7 code coverage improvement
* automatically detect if emma is on device
* build emma into libcore if necessary
* stop runtime before adb sync and restart afterwards

Change-Id: I6f1beacdd266310c481351165a054dca8f8657b2
2010-04-20 16:24:27 -07:00
Brett Chabot
8dc9eb8ba6 Runtest support for tests that need 'make' not 'mmm' to build.
Change-Id: I0508db2578826249f9bf4756d5adc6f95e841231
2010-04-16 14:44:41 -07:00
Brett Chabot
2e16fbc36b Fix grammar typo in runtest not-annotation help.
Change-Id: I5f5a81d16715f968ed9b265398d2a9e483c23a89
2010-02-23 12:28:27 -08:00
Brett Chabot
c061154208 Add "annotation" and "not-annotation" args to runtest.
Bug 2239240

Change-Id: I49bbee4b74e8b0a51b6e6144818a7e9c08b1b5c2
2010-02-23 12:17:15 -08:00
Brett Chabot
4a5d9f1ba7 Replace runtest --cts option with more generic --suite option.
This will allow you to run a the set of test_defs with attribute suite="foo"
like this:
  runtest --suite foo

Bug 2401250
2010-02-19 09:42:17 -08:00
Brett Chabot
920e9fefab Make runtest more fault tolerant on test run failures.
Adjust so when running multiple suites, the whole run is not aborted on a
single suite failure.
2010-01-21 17:33:05 -08:00
Brett Chabot
59b4778dd6 Add runtest --path.
This supports specifying a file system path to test(s) to run. The path can be
a java test class file, a java package directory, or a parent directory
containing many tests.

This change allows users to run tests independently from test_defs.xml.

BUG 2133198
2009-10-26 12:22:46 -07:00