Commit Graph

60 Commits

Author SHA1 Message Date
Ivan Lozano
26aa1c30d2 cargo2android: Improve package version logic
If there are multiple packages present in the workspace, attempt
to determine what the current package is by checking the emitted
crate_name value against the package name. If this still fails, a new
`--no-pkg-vers` flag is added to skip this behavior.

Bug: 196803495
Test: New logic finds package if package.name is used for crate_name
Test: New flag prevents emitting the pkg version.
Change-Id: I152e94a0520f09f37622a6cec340ae35842d56ea
2021-08-16 11:37:28 -04:00
Ivan Lozano
cc660f1d75 cargo2android.py: Support cargo_pkg_version flag.
This parses a module's Cargo.toml file for package version number,
setting it to cargo_pkg_version. Also sets cargo_env_compat true by
default.

Bug: 171011485
Test: generated Android.bp includes cargo_env_compat, cargo_pkg_version.
Change-Id: Ibb8021633719650e2021b90806374a5d3c6a161a
2021-08-12 13:04:25 -04:00
Joel Galenson
e4f5388cf3 Let cargo2android.py generate data properties for tests.
Some tests load data from files.  These files thus need to be copied
to the correct location on the host/device so that the tests can
access them.  This adds an option to cargo2android.py so that it can
generate these data properties.

Bug: 171710847
Test: Use for a crate.
Change-Id: I14a1ad8aca4625cc5f09807d19ffd7499fcf3689
2021-07-19 12:18:21 -07:00
Ivan Lozano
9192086c9c cargo2android.py:vendor and vendor_ramdisk flags.
Adds flags to emit vendor_available and vendor_ramdisk_available flags
to generated modules.

Bug: 194078853
Test: Calling script with new flags generates intended properties.
Change-Id: I4f7a6870b4f210b6627054c98962f50e98bc3e03
2021-07-19 15:18:11 +00:00
Joel Galenson
14b71a757b Merge "When merging tests, prefer non-blocklisted ones." 2021-07-16 17:27:59 +00:00
Jooyung Han
a427c9b64f cargo2android: rename libxml to libxml_rust
Bug: 193808832
Test: m
Change-Id: Ib3acfdda03197e8b773cbec500cadcc94d1a52a3
2021-07-16 08:53:14 +09:00
Joel Galenson
57fa23a5db When merging tests, prefer non-blocklisted ones.
The test blocklist was failing when multiple tests were merged into
one.  To fix this, when merging tests we prefer ones that are not
blocklisted.

Fixes: 193790055
Test: Run on the previously failing crate and another.
Change-Id: I53d1179bebef824ce80ac4f977015b04a26f8607
2021-07-15 10:53:24 -07:00
Joel Galenson
1353136072 Merge "Rename rust_defaults that contain tests." 2021-07-14 16:14:48 +00:00
Joel Galenson
d37d7e6d2c Rename rust_defaults that contain tests.
This patch ensures that rust_defaults for tests have _tests in the
name.  This can make the Android.bp file a bit more readable.

Test: Run on a crate and build it.
Change-Id: I5d034c456052b892bd01c3c041b2572bc9364692
2021-07-13 09:03:01 -07:00
Joel Galenson
12467e54a7 Let cargo2android ensure some libraries are whole_static_libs.
Test: Use on a crate.
Change-Id: I1d6a3bdb47141d5d20af3c118fd7a05bef223a48
2021-07-13 08:55:17 -07:00
Treehugger Robot
5b65b9117d Merge "cargo2android.py: remove TEST_MAPPING generation" 2021-07-02 15:49:52 +00:00
Thiébaud Weksteen
198e93f61b cargo2android.py: remove TEST_MAPPING generation
Do not generate TEST_MAPPING when running cargo2android.py. Still accept
the command line argument so current users/wrappers continue working.

Bug: 192348260
Test: Run cargo2android.py manually on ahash
Change-Id: I1975119f8fdd3ae850649b105678dec92fa67518
2021-07-02 16:30:31 +02:00
Luke Huang
a1371afd29 Add name mapping for base64
Test: TH
Bug: 155855709
Change-Id: I1e411e2f831ede6f48822352774f6e44db5d409d
2021-06-29 18:57:33 +08:00
Joel Galenson
5664f2af26 Allow adding blocks of texts to the Android.bp file.
You can either add the blocks to the toplevel of the file or into the
main module.  The blocks are given in separate files.

Test: Use both for some crates.
Change-Id: I74204867d4e348c83c7adc2a833be166956bfb7e
2021-06-14 13:44:58 -07:00
Joel Galenson
cb5f2f0bd7 Add a lib blocklist and an option to force rust_library_rlib.
Test: Use in some crates.
Change-Id: I5d098ce9b0f1fc37c5d8408524c0755869657992
2021-06-08 15:08:29 -07:00
Joel Galenson
3d6d1e78b5 Allow blocklisting cfgs.
Test: Use it for a crate.
Change-Id: I72e383ce58f64c7e545db3049376feea8ac006d1
2021-06-07 15:00:24 -07:00
Joel Galenson
f6b3c91d21 Allow blocklisting tests.
This allows you to give cargo2android.py a list of test files to ignore.

Test: Use on a couple crates.
Change-Id: Id9422c496de6693029754f32d9805f1cb92d33cb
2021-06-03 16:00:54 -07:00
Joel Galenson
97e414af12 Allow blocklisting dependencies.
A few crates have patches that manually remove a dependency that
cargo2android.py adds.  This option can allow us to remove those
patches.

Test: Use when running on a few crates.
Change-Id: Ie8337789ab232fb726ccfe361df489acfdd31b20
2021-06-02 15:49:04 -07:00
Joel Galenson
7e8247e2eb Abort cargo2android.py if the patch fails.
This allows the external_updater to understand that an error occurred.

The pre-patched version of the file will still exist.

Test: Run when a patch fails.
Change-Id: I59cc12eeaa751b5e3e2956060f58981d27875748
2021-05-24 15:31:08 -07:00
Joel Galenson
6bf54e3abb When cargo errors, print an error about cargo.out.
Currently if cargo produces an error it's not obvious how to debug it.
Add a message telling the user to look at the cargo.out file, which
has its output.

Test: Ran on crates with and without errors
Change-Id: Ib681a8e87f60467c7425aed5f7e5277a0603bcc8
2021-05-17 10:54:50 -07:00
Joel Galenson
367360cdaf Do not write out the --no-test-mapping option.
This option is only intended to be temporary, so if it is used it
should not "propagate" itself by being added to the Android.bp header
or the config file.

Test: Do not see it in the header or config file.
Change-Id: Ibb18c76bdab26aa47b38ac75ea3c410ddc612856
2021-04-29 14:33:13 -07:00
Joel Galenson
d9c4de6fe3 Let cargo2android.py generate apex_available and min_sdk_version.
This commit adds an --apex-available option to cargo2android.py.  It
takes in multiple strings that are included in the generated block.

It also adds --min-sdk-version.

Test: Use flags.
Change-Id: I319a32fc5e8be11015ba4484b0ab1665ee457326
2021-04-26 11:34:15 -07:00
Joel Galenson
0fbdafe016 Allow cargo2android.py to use a config file instead of command-line options.
The command-line options to c2a are already long and will only get
longer as new arguments are added.  This allows it to load them from a
config file.  For simplicity of implementation, the config file
supplements the command-line arguments and uses the same format they
do, although it ideally will replace them over time.

This also adds an option to dump the current set of command-line
arguments to a config file and exit.  This is intended to ease
migration over to the config file: for a given crate, we can call c2a
with its command-line arguments plus this new flag to generate the
config file and then use just that instead.

Test: Dump and load some config files.
Change-Id: I63b29dd20bcff5d5832dbd380d7c6eb273547ed0
2021-04-26 11:33:52 -07:00
Thiébaud Weksteen
a5a728b4c8 cargo2android: Use cfgs property
Test: Run cargo2android on multiple crates, verify Android.bp
Bug: 183727250
Change-Id: I087ffb0de33db7a09735d2e9c668d991c9f5bb49
2021-04-08 14:23:49 +02:00
Treehugger Robot
ad8e4d2a7e Merge "NFC: Refactor a string so we can add a comment to it." 2021-04-07 22:15:50 +00:00
Joel Galenson
4d82acfeee Merge "Add a cargo2android flag to disable generating TEST_MAPPING." 2021-04-07 21:41:15 +00:00
Joel Galenson
3f42f80970 NFC: Refactor a string so we can add a comment to it.
Test: Run cargo2android.py
Change-Id: Id833bd3292a4cd9f8c348aee9725e0924d826d97
2021-04-07 12:42:17 -07:00
Joel Galenson
cf4ebb00a6 Add a cargo2android flag to disable generating TEST_MAPPING.
Generating a TEST_MAPPING file uses Bazel, which can be quite slow
(multiple minutes).  Add a flag to allow skipping it.  This is off by
default as it ideally will only be used for debugging.

Test: Run all four configurations of --tests and this flag.
Change-Id: I08bde29ea298fcf677addd86f6d953b6695e147f
2021-04-07 08:39:51 -07:00
Joel Galenson
d9d13b8309 Fix cargo2android's license block preservation
The code that preserves license blocks currently also preserves
genrules if they come before a rust_ rule, causing them to be
duplicated.  Let's fix that.

Test: Upgrade some crates
Change-Id: Ie4f62cd04c2f1dd0de788db0ca5e77502fb3b4c8
2021-04-07 08:20:58 -07:00
Haibo Huang
0f72c95e70 [cargo2android.py] Reserve all lines before the first rust_* rule
Change-Id: Ic3cffcc7f34d878b7cc627f10361d42271b12f1e
2021-03-19 15:17:42 -07:00
Joel Galenson
56446748cf Have cargo2android put warning not to modify Android.bp files
Test: Run cargo2android
Test: Run upgrader script
Change-Id: Idd9fc9b8ad7a7c4fb3da954178d0221c95132fd3
2021-02-18 09:36:53 -08:00
Jeff Vander Stoep
1b24dc3f71 update_crate_tests: fail gracefully
Allow this script to accept a path to the crate being updated.
This allows it to be run separately from cargo2android.

Also, fail gracefully on error. TEST_MAPPING update failures should
not block crate updates.

Test: tools/external_updater/updater.sh update rust/crates/libc
Bug: 179132533
Change-Id: I3a4229f479ab3d2793df16b470f0a0632b9ee495
2021-02-03 21:42:40 +01:00
Jeff Vander Stoep
cec8bac2cb Cargo2Android: add rdep tests to TEST_MAPPING
This is important because:
1. Some crates do not have their own tests enabled in presubmit.
2. As much as possible, we try to stick with one version of each
   crate. This often results in using different versions of
   dependencies than a crate has specified in its Cargo.toml.

Ensuring that a crate's tests continue to pass when its dependencies
are upgraded improves our confidence that the update is safe.

The underlying implementation uses the new Bazel queryview
to query modules and reverse dependencies.

Bug: 168167373
Test: Run cargo2android.py on a number of crates including rusqlite,
scopeguard, lock_api.

Change-Id: Id24f2d3267cf8d5e0369ece2442f8971d4ab1343
2021-01-15 20:56:59 +01:00
Treehugger Robot
169b7827ed Merge "Cleanup cargo2android.py." 2021-01-13 20:47:40 +00:00
Joel Galenson
4319a7e8a5 Cleanup cargo2android.py.
Followup to aosp/1548356 to remove the now unused host parameter to add_test.

Test: Manually run script and verify the output.
Change-Id: Id00e56a2dc5523a44dc3cb473702c1ae8785d3db
2021-01-13 08:36:31 -08:00
Treehugger Robot
a727e45813 Merge "Do not generate TEST_MAPPINGs for host tests" 2021-01-13 16:20:27 +00:00
Joel Galenson
e261a15598 Do not generate TEST_MAPPINGs for host tests
Instead of generating TEST_MAPPING entries for host tests, we add a
test_options section to its Android.bp entry.

Fixes: 176097100
Test: Manually run cargo2android on a few crates and verify the output
Change-Id: Ia57a53dad6910410c7e7aab6d592fba79720f998
2021-01-12 11:31:53 -08:00
Ivan Lozano
0c057ade57 rust: Use rust_ffi for Rust generated C libraries.
We no longer have the rust_library_shared and rust_library_static module
types. They've been renamed to rust_ffi_* instead. This brings
cargo2android.py up to date.

Bug: 175155132
Test: cargo2android's Android.bp for libquiche contains rust_ffi modules
Change-Id: I01bb8b4e56c41d36481ffd363f96aecb770d7cb8
2020-12-15 10:41:26 -05:00
Victor Hsieh
21bea79f5a cargo2android: map fuse crate in crosvm to libfuse_rust
Test: run cargo2android in external/crosvm/fuse and build
Bug: 174797066
Change-Id: I32d092818164d422e263c10f8bf5b7b50971a515
2020-12-04 11:00:46 -08:00
Treehugger Robot
7534ce72da Merge "cargo2android: reorder TEST_MAPPING statements" 2020-11-05 19:16:05 +00:00
Jeff Vander Stoep
d11be480f3 cargo2android: reorder TEST_MAPPING statements
"host: true" entry should go before "name:" to match the output of
json writers which sort by alphabetical order. This will help and
avoid unnecesary changes when adding new tests to TEST_MAPPING - e.g.
aosp/1488728

Bug: 168167373
Test: cargo2android.py --run --tests
verify output

Change-Id: Id346a7f8a2cd910115e7fcc448c589363209bee9
2020-11-05 16:13:36 +01:00
Chih-Hung Hsieh
60140756b8 Give a warning to use --copy-out
* Suggest --copy-out if it is not used, cargo output files are found,
  and env!("CARGO_OUT") is found in .rs files.
* Expect to see this warning for libsqlite3-sys and grpcio-sys,
  which fit to the use case of --copy-out but they have locally
  defined .bp files and non-original output files.
* Fix one pylint warning of extra long line.

Bug: 172299436
Test: run cargo with same flags for all rust/crates/*
Test: run cargo without --copy-out for anyhow, protobuf, clang-sys, etc.
Change-Id: Ic346c8e1146b1bb77e63c4cb12947f202b0f458e
2020-11-03 15:51:52 -08:00
Chih-Hung Hsieh
ec8846be07 Add --patch and --ignore-cargo-errors options
* The given patch file will be applied to ./Android.bp.
* Use --ignore-cargo-errors when it is impossible to
  use the patch file to fix the error messages in Android.bp.

Bug: 172093078
Test: apply on external/rust/crates/*, some with this flag
Change-Id: Ibca2fd4c63e46a32f32c61afedbba960a684f5d7
2020-11-02 21:55:39 +00:00
Chih-Hung Hsieh
610a894a49 Do not use Cargo.lock during cargo build.
* Save Cargo.lock before calling cargo, and restore it after the run.
* Use the --use-cargo-lock flag if the latest dependent crates failed
  to build the current crate. In this case, the published Cargo.lock
  will be used, but the generated .bp might not work with the other
  latest dependent crates in external/rust/crates.
* Fix one gpylint warning.

Bug: 172016745
Test: run through all external/rust/crates/*
Change-Id: I1874a706ac15b74c1dd5a606983b107284d9b28a
2020-11-02 21:49:31 +00:00
Chih-Hung Hsieh
e2342bac4b Add a --copy-out flag for build.rs output files.
* When --copy-out is used:
  * copy build.rs output files to ./out
  * add a genrule module to copy ./out/* files to its output dir
  * add this copy-out genrule module into the srcs list
  This makes include! with $OUT_DIR path work without local patch.
  This only works for the root directory (package).
* Upgrade to python3 for some required shutil and glob functions.
* Unified dump_srcs_list for defaults and non-defaults modules.
* dump_android_property_list output only one line for a single item list.

Bug: 171659849
Test: regenerate all .bp files in rust/crates/*
Change-Id: Ia7dde0ccede2bcc068f23a046e85304c6f50b0b0
2020-10-25 18:06:04 -07:00
Chih-Hung Hsieh
e1b7bb667b Use x86_64-unknown-linux-gnu for host cargo build
* Now a host cargo build uses the same --target flag as a default
  device build.
* When --device is given without --no-host or other device flag,
  we can use one cargo build to get the required rustc flags for
  both host and device builds.
* We still need merge_host_device because cargo build --test could
  recompile a library and create duplicated calls to rustc.
* Fix a gpylint warning missing-function-docstring.
* This change also detected a bug in
  external/rust/crates/structopt-derive/Android.bp
  The cargo build --tests for device was optimized out
  and the device test module was missing in Android.bp.

Bug: 169872957
Test: regenerate all .bp files in external/rust/crates
Change-Id: Ieabdc5a99534a60e4d459c9a966f7a867104c785
2020-10-01 16:59:04 -07:00
Chih-Hung Hsieh
6c13b72f15 Ignore -C codegen-units flag
Test: regenerate .bp files in external/rust/crates/*
Change-Id: I023b0b035d50db9eb7c7462d21beba9b95078378
2020-09-11 21:27:37 -07:00
Chih-Hung Hsieh
63459ed34e Ignore embed-bitcode flag
Bug: 166331593
Test: regenerate .bp files in external/rust/crates
Change-Id: I3051a4d2d4e79b108f44bbed3fb0c51c86b81baf
2020-08-26 11:54:25 -07:00
Andrew Walbran
e51f104b42 libbase from latest crosvm also needs to be renamed.
Bug: 158290206
Test: ran cargo2android.py on latest crosvm
Change-Id: Id6cef5e3af72f7bd8defbc4ed61aa4b5b3729d6b
2020-08-11 16:42:48 +01:00
Chih-Hung Hsieh
07119865a0 Add options to work in a subdirectory.
* Problem to solve:
  When cargo2android.py is run under external/crosvm,
  it does not generate .bp file in some subdirectories like kvm.
* Use --add_workspace flag to append [workspace] in Cargo.toml
  temporarily to generate correct relative source file paths.
  Otherwise, the path will be based on parent/root package
  such as external/crosvm.
* Use --global_defaults=default_name flag to insert a global
  default module name like "crosvm_defaults" in every module.
* Use --no-subdir flag to generate .bp file in one directory,
  and skip all changes to subdirectories.
* Sort the option names so they show up in order with --help.
* Use relative path for local dependent packages.
* Example: run in external/crosvm with flags;
    --run --tests --dependencies --no-subdir
  fix-up external/crosvm/Android.bp file,
  and then run in each subdirectory with flags:
    --run --tests --dependencies --add_workspace
    --global_defaults=crosvm_defaults
* Add rename mapping:
    libminijail ==> libminijail_rust

Bug: 161716839
Test: regen .bp files in external/crosvm subdirectories
Test: make && atest -m -c --include-subdirs external/crosvm
Change-Id: I0c08d358cc2f88f66e99b59032613d2a5b4ea5eb
2020-07-29 15:23:03 -07:00