Commit Graph

20 Commits

Author SHA1 Message Date
Patrick Rohr
533af6d72d TrackRecord: add a method to return callbacks since last poll
This change adds a backtrace() method which returns a list of events
that were received since the last time a user called poll() on a
ReadHead. This is particularly useful for logging observed events while
a poll was in progress that did not end up matching an event.

Test: atest EthernetManagerTest
Change-Id: If019cf9eb5e3e9268c5e6b74edbd8f49959cc71c
2023-08-09 05:26:36 -07:00
Remi NGUYEN VAN
7035b4418d Merge "Add an annotation for @SkipMainlinePresubmit" into main 2023-07-27 02:13:22 +00:00
Remi NGUYEN VAN
5e4900dbbe Add an annotation for @SkipMainlinePresubmit
This will typically be used in mainline-presubmit TEST_MAPPING
configuration to exclude some tests.

This is necessary as some tests are flaky in mainline configurations
(when running on older devices), and they need to be disabled to meet
SLO requirements, but they are not flaky on presubmit using the latest
platform builds.

Test: m
Change-Id: Ia532d6d3f9833ddec613d79c0eb517d20a1c529a
2023-07-26 13:37:47 +09:00
Mike Yu
8f4debc3f9 Add DnsResolverModuleTest annotation
This annotation can be used to mark a test case that requires
the latest resolv module. Tests that don't run with the latest
resolv module (for example CtsNetTestCasesLatestSdk) can exclude
the testcases being flaky due to some known issues in old resolv
module.

Bug: 279846955
Test: TreeHugger
Change-Id: Ie19eed1c4aa17645c4eec45493e7999027a01205
2023-06-26 07:40:18 +00:00
Xiao Ma
cff263aa10 Add NetworkStackModuleTest annotation.
This annotation can be used to exclude the testcases which requires the
latest NetworkStack module in the CTS/MTS suite.

Bug: 283200648
Test: atest CtsTetheringTestLatestSdk
Change-Id: Idaffed93af077c2998081142af7b3bfa311dcd90
2023-06-01 13:34:01 +09:00
Remi NGUYEN VAN
bcdc972c86 Add packet filters for IPv6 UDP
This is useful to test mDNS.

Bug: 266151066
Test: atest NsdManagerTest (with related change)
Change-Id: I790da3f3be5277f2480600cfbbaeac86c306f77d
2023-01-23 18:20:44 +09:00
Chalard Jean
a19f85124e Add Function3
...and rename ExceptionUtils to FunctionalUtils as the old name
would no longer be appropriate.

Test: FrameworksNetTests
Change-Id: I2affd69fb84d7f250b4a45497eec6c052bf6ec50
2022-08-16 18:53:17 +09:00
Remi NGUYEN VAN
a79814aade Fix handling of Throwable in tryTest/catch
Throwable could be used as a type argument to catch(), but it would
possibly never be caught, because the method only catches Exception.

Notably, junit assertions fail with AssertionError, which is not a
subclass of Exception. Due to that, tests or utilities like
DeviceConfigRule that have failing assertions in catch() would not
run the subsequent cleanup steps.

Bug: 210377950
Test: atest CleanupTest
Change-Id: I54e2922cb466f077ba4d219f8c3c6f885316296c
2022-06-21 12:52:10 +09:00
Chalard Jean
997300b9d1 Support multiple cleanup blocks in tryTest{}
This is useful for some tests that need to make sure a few
things all happen, but some of them may throw an exception.

In vanilla Java you'd say

try {
  ...
} finally {
  try {
    ...
  } finally {
    try {
      ...
    } finally {
      ...
    }
  }
}

With this patch, you can pass a list of blocks to clean.
Kotlin :

tryTest {
  ...
} cleanupStep {
  ...
} cleanupStep {
  ...
} cleanup {
  ...
}

Java :
tryAndCleanup(() -> {
  ...
  }, () -> {
  ...
  }, () -> {
  ...
  })

This keeps the semantics of tryTest{} of throwing any
exception that was thrown in tryTest{} and adding the
exception in the cleanup steps as suppressed.

Test: new tests for this
Change-Id: Ie26b802cb4893e13a70646aa0b7887701dee1ec6
2021-12-06 17:57:29 +09:00
Chalard Jean
fee2cab5d6 Correct a comment
Test: TH
Change-Id: I08ac8f0f40b9e3732564da73057cd3d22c06c79b
2021-11-25 19:55:59 +09:00
Chalard Jean
28348e43ae Have tryTest{} work with throwables
Test: NetworkStaticLibTests
Change-Id: I9ee52e7d34866c82e169bbecbca3f928bfd79993
2021-11-25 16:26:42 +09:00
Chalard Jean
2918ada958 Rename ExceptionCleanupBlock into TryExpr
This is a lot more inline with standard compiling theory

Test: NetworkStaticLibTests
Change-Id: I32c8926cb08a30b4a87bc6867ce194483a7f0c3a
2021-11-25 16:26:42 +09:00
Chalard Jean
8f0a33054b Add catch{} to tryTest{}
Usage is
tryTest {
  ...
}.catch<Exception> {
  it
} cleanup {
  ...
}

Test: New tests for this
Change-Id: Idb7d9bd33034921c8f212288179b34a175e866f4
2021-11-25 16:26:42 +09:00
Chalard Jean
b6fdf63095 tryTest to return its last evaluated expression
This is the same thing try{} does, and allows to lift
return out of tryTest.

This allows syntaxes like :
fun foo() = try {
    "Foo";
  } cleanup {
    doSomeCleanup()
  }
}

val network = try {
    registerNetworkCallback
    callback.getNetwork()
  } cleanup {
    unregisterNetworkCallback
  }
}

Note: bypassing ktlint because of b/185077240

Test: FrameworksNetTests
Change-Id: Ib8f6fde7ccfd62fdcb3c1e3b7b03909ed94d4b23
2021-11-25 16:26:42 +09:00
Chalard Jean
7df6f89c37 Disallow non-local returns in tryTest{}
Non-local returns will prevent the execution of the cleanup{}
block, which is too much of a footgun to allow. See the bug
for details of how it happens.

There doesn't seem to be a way to keep the nice syntax, allow
non-local returns and still guarantee execution of the cleanup
block in all cases. Thus, forbid non-local returns. Users can
still use return@tryTest to accomplish almost the same thing,
and the next patch will also let tryTest{} return its last
evaluated value, fixing remaining cases.

E.g.
tryTest {
  foo()
  if (condition) return result
  bar()
} cleanup {
  doCleanup()
}

can always be written

return tryTest {
  foo()
  if (condition) return@tryTest result
  bar()
} cleanup {
  doCleanup()
}

...and it's a rare case, so the additional syntax is acceptable.

Test: NetworkStaticLibTests
Bug: 207358921
Change-Id: I40443acf7e4d86813641adc877e27fb2334d0daf
2021-11-24 17:08:54 +09:00
Chalard Jean
bfbcacef6d Make sure no suppressed expression is present when they shouldn't
In fact the test already passes, because addSuppressed contains
an explicit test that if this === argument, then it doesn't add
it. But otherwise that's a bug in tryCleanup

Test: NetworkStaticLibTests
Change-Id: I202790bbe8d82445c5affdd9076561c2c6ea9b59
2021-11-24 17:07:28 +09:00
Chalard Jean
5872cb8dbd Add a new utility to improve stack traces from automatic testing
A common pattern in tests is to use try-finally to make sure
cleanup is executed. This is necessary in CTS in particular to
make sure the test does not leave the device in a bad state.

The problem with using try-finally in this manner is that any
exception thrown in the finally{} block will override any thrown
in the try{} block. If the exception in finally{} is caused by
the one from try{}, then the stack trace reported by the tools
is the consequence and not the cause, making it difficult to
interpret the stack trace of automated tests.

So today, code has to make sure that either the code in
finally{} can't throw, or can't be affected by any code in
try{}, and both of these are really difficult to ensure in
presence in finally{} of code the tester does not control.

What we'd want ideally is a structure like try-finally, that
guaratees the code in finally{} is executed in all cases,
but that bubbles up the exception from try{} if any, and
will still bubble up any exception from finally{} if the
try{} block hasn't thrown.

That's what this new tool does.

Usage from Kotlin is like try-finally :
tryTest {
  testing code
} cleanup {
  cleanup code
}

Usage from Java can't be made as nice, but it's relatively okay :
testAndCleanup(() -> {
  testing code
}, () -> {
  cleanup code
});

Bugs listed below are some tests that have been affected by
this issue and have unhelpful traces.

Test: new test for this code
Bug: 198586720
Bug: 198998862
Change-Id: I54b30a7d53772feeade99274b6120a79707ad1c9
2021-09-17 12:21:09 +09:00
Remi NGUYEN VAN
843740592c Add the ConnectivityModuleTest annotation
The annotation will be used to mark tests cases that are part of MTS
modules used by multiple modules, to indicate that the test case should
only be run if the MTS suite should cover an updated Connectivity
(tethering) module.

This allows tests to verify new behavior in the Connectivity module,
without failing if that module is not updated.

Bug: 196755836
Test: mts-tradefed run mts-network with exclude-annotation in the config
Change-Id: I05d2200c7ab68042e4e48029e045f164c26b8037
2021-08-16 18:09:41 +09:00
junyulai
4a73a9f08f [FUI26.1] Generalize assertSameElements
Test: atest NetworkStaticLibTests
Bug: 174123988
Change-Id: I72a33a3c6ba849666e9f892745be3175aeb34b0e
2021-05-14 11:06:18 +08:00
Remi NGUYEN VAN
be2a07fce3 Move net-test-utils to its own directory
Having the test targets in a different directory allows setting
visibility rules for tests only, which is necessary for access to
targets that should not be used for device builds.

Bug: 182859030
Test: m
Change-Id: Iaf426cf339a97833acf80c941db692329c6e2dcb
2021-03-30 16:52:09 +09:00