Commit Graph

350 Commits

Author SHA1 Message Date
Chih-Hung Hsieh
e2342bac4b Add a --copy-out flag for build.rs output files.
* When --copy-out is used:
  * copy build.rs output files to ./out
  * add a genrule module to copy ./out/* files to its output dir
  * add this copy-out genrule module into the srcs list
  This makes include! with $OUT_DIR path work without local patch.
  This only works for the root directory (package).
* Upgrade to python3 for some required shutil and glob functions.
* Unified dump_srcs_list for defaults and non-defaults modules.
* dump_android_property_list output only one line for a single item list.

Bug: 171659849
Test: regenerate all .bp files in rust/crates/*
Change-Id: Ia7dde0ccede2bcc068f23a046e85304c6f50b0b0
2020-10-25 18:06:04 -07:00
Chih-Hung Hsieh
03f14e4b4f Better heuristic to find license types
* Handle lower case file names like LICENSE-Apache.
* Use "license" key value in Cargo.toml if LICENSE-* files are missing.
* Recognize license type: ISC
* Improve readability of a sequence of if-statements.

Bug: 170254513
Test: get_rust_pkg.py --add3prf time-macros-impl
Test: run add3prf.py in current rust/crates/* projects
Change-Id: I9642f52090b20922302513a94fa97d41277f9762
2020-10-19 23:15:50 -07:00
Chih-Hung Hsieh
3d24aed831 Add add3prf.py and call it from get_rust_pky.py
* add3prf.py can be called in existing rust package directories
  to normalize METADATA, OWNERS, and other LICENSE* files.
* Add option in get_rust_pkg.py to call add3prt.py and add
  required files for 3rd party reviews.

Bug: 170254513
Test: call add3prf.py in external/rust/crates/*
Test: get_rust_pkg.py --add3prf quote remain paste
Change-Id: I70ed5942847716af2ca740a91e7ecde50543b7a0
2020-10-13 22:29:07 -07:00
Chih-Hung Hsieh
e1b7bb667b Use x86_64-unknown-linux-gnu for host cargo build
* Now a host cargo build uses the same --target flag as a default
  device build.
* When --device is given without --no-host or other device flag,
  we can use one cargo build to get the required rustc flags for
  both host and device builds.
* We still need merge_host_device because cargo build --test could
  recompile a library and create duplicated calls to rustc.
* Fix a gpylint warning missing-function-docstring.
* This change also detected a bug in
  external/rust/crates/structopt-derive/Android.bp
  The cargo build --tests for device was optimized out
  and the device test module was missing in Android.bp.

Bug: 169872957
Test: regenerate all .bp files in external/rust/crates
Change-Id: Ieabdc5a99534a60e4d459c9a966f7a867104c785
2020-10-01 16:59:04 -07:00
Haibo Huang
987436c90b Add source map for binaries from RBE
Binaries built from RBE are using a source path under /b/f/w. Also map that to local source path.

Test: lldbclient.py -r /system/bin/ls
Change-Id: I02a90a9c827189a14b2d83485ed2abda68f9e084
2020-09-23 04:03:01 +00:00
Haibo Huang
6a4636a9ef Make lldb default for gdbclient.py
Adds a symlink lldbclient.py that pointing to gdbclient.py

Bug: 168187078
Change-Id: I1831875812d22bb6a264b8396bc9d3b8ed7712e3
2020-09-15 18:55:52 +00:00
Chih-Hung Hsieh
6c13b72f15 Ignore -C codegen-units flag
Test: regenerate .bp files in external/rust/crates/*
Change-Id: I023b0b035d50db9eb7c7462d21beba9b95078378
2020-09-11 21:27:37 -07:00
Treehugger Robot
8240673629 Merge "gdbclient.py: Try ro.build.product also." 2020-09-11 06:13:38 +00:00
Junichi Uekawa
6612c929d7 gdbclient.py: Try ro.build.product also.
ro.product.name can be individual device name and not the lunch target
name.

466e289dc8 removed this check but we
still need this.

Bug: 166071019
Test: gdbclient.py -n vold  # on eve

Change-Id: I60fc20e6cc02ca6dd01fef609c7f60e76c580876
2020-09-09 00:23:49 +00:00
Xin Li
e3b57410da Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)
Bug: 166295507
Merged-In: I37a77cdaf905fe0bf58ad8eebcdff36873aec71a
Change-Id: Idac9218a0e169df9d02d9d8d376a6816315a4b28
2020-08-27 10:16:04 -07:00
Chih-Hung Hsieh
63459ed34e Ignore embed-bitcode flag
Bug: 166331593
Test: regenerate .bp files in external/rust/crates
Change-Id: I3051a4d2d4e79b108f44bbed3fb0c51c86b81baf
2020-08-26 11:54:25 -07:00
Andrew Walbran
e51f104b42 libbase from latest crosvm also needs to be renamed.
Bug: 158290206
Test: ran cargo2android.py on latest crosvm
Change-Id: Id6cef5e3af72f7bd8defbc4ed61aa4b5b3729d6b
2020-08-11 16:42:48 +01:00
Chih-Hung Hsieh
07119865a0 Add options to work in a subdirectory.
* Problem to solve:
  When cargo2android.py is run under external/crosvm,
  it does not generate .bp file in some subdirectories like kvm.
* Use --add_workspace flag to append [workspace] in Cargo.toml
  temporarily to generate correct relative source file paths.
  Otherwise, the path will be based on parent/root package
  such as external/crosvm.
* Use --global_defaults=default_name flag to insert a global
  default module name like "crosvm_defaults" in every module.
* Use --no-subdir flag to generate .bp file in one directory,
  and skip all changes to subdirectories.
* Sort the option names so they show up in order with --help.
* Use relative path for local dependent packages.
* Example: run in external/crosvm with flags;
    --run --tests --dependencies --no-subdir
  fix-up external/crosvm/Android.bp file,
  and then run in each subdirectory with flags:
    --run --tests --dependencies --add_workspace
    --global_defaults=crosvm_defaults
* Add rename mapping:
    libminijail ==> libminijail_rust

Bug: 161716839
Test: regen .bp files in external/crosvm subdirectories
Test: make && atest -m -c --include-subdirs external/crosvm
Change-Id: I0c08d358cc2f88f66e99b59032613d2a5b4ea5eb
2020-07-29 15:23:03 -07:00
Chih-Hung Hsieh
776f6a198c Handle -Cflag syntax and use prebuilt cargo.
* Accept both "-C flag" and "-Cflag" syntax.
  * Newer cargo uses "-Cembed-bitcode=no".
  * Some -C flags are filtered out, not used in .bp file,
    because they are meaningless in .bp file build system.
  * Remaining -C flags are passed to rustc in .bp files,
    which was missing before this change.
* Look up cargo in the prebuilts directory.
  * Now limit this script to run only on linux.
  * Try to find cargo/rust version from
    build/soong/rust/config/global.go first.
  * Otherwise, use the latest (largest) version in prebuilt.
* Add a new --cargo_bin flag to use any user-selected cargo.
* Add the selected cargo directory to PATH before calling cargo,
  so it can find rustc in the same directory.

Bug: 161825397
Bug: 161927172
Test: regenerate and check .bp files in external/rust/crates
Change-Id: Ica46f536c2b37b62238d1245ced59685deebad33
2020-07-23 16:04:58 -07:00
Haibo Huang
e4d8bfdfa1 Uses lldb wrapper if exists for gdbclient.py
Bug: 161853714
Change-Id: I3c19e06195569d46889d4c967cdab1fc5c7e39b8
2020-07-22 16:48:40 -07:00
Chih-Hung Hsieh
f7eff158a9 Generate TEST_MAPPING for test modules in .bp file
* Share srcs in rust_defaults if len(srcs) is 1.

Test: regenerate .bp and TEST_MAPPING files in external/rust/crates
Test: make
Test: atest -c  --include-subdirs external/rust/crates
Bug: 161259631
Change-Id: I22a1d567ccabcd945e4cac870049ac3ce7a3049a
2020-07-17 16:20:22 -07:00
Chih-hung Hsieh
7b8b3ce128 Merge "Add EXCLUDED_CRATES and better default names" am: 1604140fca am: 4ca8702740
Original change: https://android-review.googlesource.com/c/platform/development/+/1362349

Change-Id: Icb82ed4794fed021e26343d806f5cc50db877431
2020-07-15 19:35:08 +00:00
Chih-Hung Hsieh
e02dce1683 Add EXCLUDED_CRATES and better default names
* protobuf_bin_gen_rust_do_not_use is in EXCLUDED_CRATES
* build_default_name returns a short and readable name
  for a rust_defaults module, from the following choices:
    (1) root_pkg + '_defaults',
    (2) root_pkg + '_defaults_' + crate_name
    (3) root_pkg + '_defaults_' + main_src_basename_path
    (4) root_pkg + '_defaults_' + a_positive_sequence_number

Test: regen external/rust/crates/*/Android.bp
Change-Id: I6aa16a8a6c7ea05744a31a9d4ff0c4524ef91fea
2020-07-14 18:50:15 -07:00
Chih-hung Hsieh
1a75277c5a Merge changes I8c70f568,I96c43cff am: 9dadf3459f am: 99b8956df6
Original change: https://android-review.googlesource.com/c/platform/development/+/1361109

Change-Id: I7bbb40b87e0b4edd33169a2827f00e157a93f293
2020-07-13 18:09:51 +00:00
Chih-Hung Hsieh
3725e085ec More options and rename maps; one file per rust_test
* Add a --no-host option.
* Add a --host-first-multilib option.
* Add special renaming for protoc_gen_rust.
* Dump a rust_defaults module to be shared by rust_test modules.
  * Every test source file is in a rust_test module.
  * Add RENAME_DEFAULTS_MAP for altered rust_defaults names.
* Suppress old error message on --extern proc_macro.

Test: regenerate Android.bp files and check differences
Bug: 161090277
Bug: 161089524
Change-Id: I8c70f568facf58d282ea2a2e72b60c8a2972b27d
2020-07-12 23:06:57 -07:00
Chih-Hung Hsieh
35ca4bcf21 Use rustlibs and not deny_warnings.
* The old deny_warnings is replaced with a comment to remind the users.

Test: regenerate Android.bp files and check differences
Bug: 161001041
Bug: 161001220
Change-Id: I96c43cff83edcb7b50e6beee972c9157d20c8dc9
2020-07-10 17:40:36 -07:00
Roland Levillain
7607e1c318 Merge "Rename native code coverage paths environment variable in acov." am: e00048a28b am: 9f1eb641e5
Original change: https://android-review.googlesource.com/c/platform/development/+/1329073

Change-Id: I794aee351972bb4470863671c76c51d5be932db6
2020-06-15 10:37:09 +00:00
Roland Levillain
e00048a28b Merge "Rename native code coverage paths environment variable in acov." 2020-06-15 10:12:09 +00:00
Roland Levillain
0fb7707b2d Rename native code coverage paths environment variable in acov.
Rename `COVERAGE_PATHS` as `NATIVE_COVERAGE_PATHS`.

Test: n/a
Bug: 158212027
Change-Id: Ifbf4bed29880cd169aaca63733420f260cf6551d
2020-06-10 12:53:49 +01:00
Chih-hung Hsieh
727f16f7bf Merge "Include command-line arguments in comment." am: f7bf9abb58 am: 8b34a8a348
Original change: https://android-review.googlesource.com/c/platform/development/+/1327013

Change-Id: I424202b7e79a05d89e4897e72214342090f6eb3e
2020-06-09 21:51:49 +00:00
Andrew Walbran
80e90be1b7 Include command-line arguments in comment.
This makes it clear exactly how a build file was generated and how to
reproduce it.

Bug: 158290206
Change-Id: Icf28a66bf0da708a3fcd0fe3d49893191fb1a8f1
2020-06-09 14:33:39 +01:00
Treehugger Robot
682fb39f4c Merge "Allow passing arguments with -- in addition to -." am: 6999b11924 am: 316661da78
Original change: https://android-review.googlesource.com/c/platform/development/+/1324434

Change-Id: I5bed3af520a0225e7d21c6fffe51322ed9dc30b5
2020-06-05 00:04:42 +00:00
Joel Galenson
70da55bef5 Allow passing arguments with -- in addition to -.
This allows the Unix-like command-line argument --show.  We keep the
existing -show.

Test: get_rust_pkg.py -show foo
Test: get_rust_pkg.py --show foo
Change-Id: I5906e0c59ef6f9e4ad33b0a9645a1fc4d9d9e1aa
2020-06-04 08:49:46 -07:00
Haibo Huang
3052e34b3c Merge "Updates gdbclient.py for lldb" am: b5727d3e0e am: 36981ede3b
Change-Id: Ifbd280408477aacabf6c87cb07a5f10b31b1d65e
2020-05-13 21:51:05 +00:00
Haibo Huang
07e17073c5 Updates gdbclient.py for lldb
1. Sets up source mapping so that people can run from other directories.
2. Removes PYTHONHOME. It is embedded into the binary.

Change-Id: Ic7a7f1483dc645d890b7e0f3dcbb0ebb169794ac
2020-05-12 16:51:29 -07:00
Treehugger Robot
2496a5da3b Merge "Miscellaneous coding style and output improvements" am: 856b7fc7d1 am: 137218a565
Change-Id: I648a68ff30eee6a919e412fe1a12f0b171a60c3b
2020-05-12 20:22:50 +00:00
Treehugger Robot
856b7fc7d1 Merge "Miscellaneous coding style and output improvements" 2020-05-12 19:49:12 +00:00
Chih-hung Hsieh
71eb527474 Merge "Ensure unique name for non-merged modules." am: 489013b6c0 am: 8d973bd2f8
Change-Id: I9113a3834136fb7c1f28a4df9c7d02f06ee757c9
2020-05-12 18:40:53 +00:00
Chih-Hung Hsieh
185052a369 Ensure unique name for non-merged modules.
* Do not emit relative_install_path;
  use default unique module name.
* Copy errors in cargo output to Android.bp.
  Some Rust packages are incomplete at crates.io
  and fail to run through cargo test.
* Apply pyformat.

Bug: 156014698
Test: in external/rust/crates/*, cargo2android.py --run --tests
Change-Id: Ide77ca34e47048a8bc03b18eebbe92fb17666784
2020-05-08 12:31:04 -07:00
Chih-Hung Hsieh
2f22d1cb80 Miscellaneous coding style and output improvements
* Call print with flush=True to get immediate output to piped files.
* Simplify code with functools.reduce or for-loops.
* Clarify error handling and messages.
* Skip output of empty dependent lists.
* Suppress output of repeated dependent lists.
* Move "revisit" logic from recursive add_non_dev_dependencies to
  a while loop in add_indirect_build_deps.
  This is less efficient, but more complete and easier to understand.
* Align table captions to something like:
    build_deps[k] = # of non-dev-dependent packages of pkg[k]
      dev_deps[k] = # of all dependent packages of pkg[k]
all_build_deps[k] = # of non-dev-dependent ... of pkg[1] to pkg[k]
  all_dev_deps[k] = # of all dependent ... of pkg[1] to pkg[k]

Test: get_rust_pkg.py -h
Test: get_rust_pkg.py -v -show remain syn quote
Test: get_rust_pkg.py -v -show glob libloading
Change-Id: I6171b6be8806bcdd1129f52e7e5ccf7b274e2e8c
2020-05-06 18:22:17 -07:00
Chih-hung Hsieh
02861af0a4 Merge "Show number of dependent packages" am: 7dc3f534a0 am: 15fbb9c2ea
Change-Id: I030a3a9cf015e566090657ae9d057d6641f43682
2020-05-06 17:46:43 +00:00
Chih-Hung Hsieh
783dfd4c64 Show number of dependent packages
* With flag "-show" and a list of package names,
  call crates.io api to count dependent packages.
  * The packages are NOT fetched.
  * Optional dependent packages are not counted.
  * Indirect non-dev dependent packages are counted.
  * Finally all non-dev dependent packages are also
    included in the report.
  * The report is a table like this:

  build_deps[k] = # of non-dev-dependent packages of pkg[k]
  dev_deps[k] = # of all dependent packages of pkg[k]
  all_build_deps[k] = # of non-dev-dependent ... of pkg[1] to pkg[k]
  all_dev_deps[k] = # of all dependent ... of pkg[1] to pkg[k]
    k pkg         build_deps   dev_deps all_build_deps   all_dev_deps
    1 unicode-xid          0          0              0              0
    2 proc-macro2          1          3              0              1
    3 syn                  2        107              0            104
    4 quote                2         18              0            107
    5 remain               4         18              0            107

  Packages are sorted topologically by the non-dev-dependencies.
  The column all_build_deps should be 0 if the topological sort
  did not find cyclic dependency.

* Simplify find_dl_path with a crates.io api call to get max_version.
* Handle urllib.error.HTTPError exception
* Formatted by pyformat.

Test: get_rust_pkg.py -h
Test: get_rust_pkg.py -show syn remain quote
Test: get_rust_pkg.py -v -show remain syn quote

Change-Id: Ib4ece8dd391558fb5fd866b153f30cd2ef59f6b3
2020-05-05 17:13:34 -07:00
Chih-hung Hsieh
babf3b6db0 Merge "Accept and fetch multiple packages" am: 066b315b2c am: 0bb869f863
Change-Id: Iac261b4ef886350548aca1ff1fb15200fbd71235
2020-04-28 17:58:00 +00:00
Chih-Hung Hsieh
23a7406de1 Accept and fetch multiple packages
* Duplicated package names are skipped.
* A package name can be a simple base name like 'syn',
  or with version number like 'syn-1.0.5'.

Test: get_rust_pkg.py -h
Test: get_rust_pkg.py pkg1 pkg2 pkg3
Test: get_rust_pkg.py -o tmp -v pkg1 pkg2 pkg1
Change-Id: I5757f20c9a7cb244dc4b06113e28fcf4585c0703
2020-04-24 13:24:14 -07:00
Chih-hung Hsieh
d0af31058e Merge "Accept multiple --crate-type and fix for new cargo." am: 8833757764 am: c81d221b0c
Change-Id: Icc61ecc41977b84b3f2a089a26a985856b185a60
2020-04-07 19:40:05 +00:00
Chih-Hung Hsieh
8a1a2300ed Accept multiple --crate-type and fix for new cargo.
* Change crate_type string to crate_types list.
  In dump_android_module, iterate over crate_types and
  call dump_one_android_module for each crate type.
* In decide_one_module_type, add mapping of Rust crate types:
        lib => rlib         (no change)
        rlib => rlib        (new)
        dylib => dylib      (new)
        cdylib => shared    (changed)
        staticlib => static (new)
  Maybe we should change and map 'lib' to 'lib',
  but before that, crate_types should not contain
  both 'lib' and 'rlib'.
* Accept new cargo output format, which can
  call rustc with the main source file name after
  other flags, not always after the "--crate-name" flag.
* Work around duplicated test module name problem;
  call self.runner.claim_module_name for every named test module.

Bug: 153118477
Test: fetch quiche-0.3.0, bindgen-0.53.2 and other Rust crates
Test: cargo2android.py --run --vv --device --tests
Change-Id: Id65c3929b5c593df73f26f1510ad7e89194b209e
2020-04-07 10:49:08 -07:00
Treehugger Robot
4aed385cbb Merge "Skip yanked packages and use newest version number." am: f50e489c40 am: 42a7aeb008
Change-Id: Icca8b90c38dacd8f2774f60fc1dd4299484ec8a4
2020-04-02 21:58:42 +00:00
Chih-Hung Hsieh
7c6362a96b Skip yanked packages and use newest version number.
* crates.io keeps yanked packages with the "yanked" attribute.
  For example, log-0.4.9 and log-0.4.10 were yanked so
  "get_rust_pkg log" should skip them.
* Compare version numbers before the publish id number.
  For example, lazy_static-1.1.1 was published after lazy_static-1.4.0,
  but "get_rust_pkg lazy_static" should fetch 1.4.0.

Test: get_rust_pkg.py -v log  # get log.0.4.8
Test: get_rust_pkg.py -v log-0.4.10  # get log.0.4.10
Test: get_rust_pkg.py -v lazy_static  # get lazy_static-1.4.0
Change-Id: Id005a91b75c0885285a76136bd6de19d0dafa5c1
2020-04-02 10:23:48 -07:00
Chih-hung Hsieh
6d03b47f67 Merge "Pass --no-default-features to cargo build" am: ebfb3a9927 am: 7a56a6c23e
Change-Id: Icf56a490dae40ed98a8a0b2d29a22b773bd5b164
2020-03-31 19:02:34 +00:00
Chih-Hung Hsieh
6c8d52f6eb Pass --no-default-features to cargo build
* When --features is specified,
  pass --no-default-features to cargo.
* When a dependent Rust package is imported to AOSP,
  if not all "default" features are needed, run
  cargo2android.py with empty or a list of specific features.

Test: cargo2android.py --run # use default features
Test: cargo2android.py --run --features=''  # no default
Test: cargo2android.py --run --features='default,std'
Change-Id: I854f1ecb7d7466490ff34e229ec14a07c1273c6d
2020-03-31 10:08:06 -07:00
Chih-hung Hsieh
ae62411fe4 Merge "add get_rust_pkg.py" am: e95677343a am: b415c21248
Change-Id: Ia088e70176b474c0a9af0114d975d675b32f7e92
2020-03-30 21:13:44 +00:00
Chih-Hung Hsieh
8a3e910fee add get_rust_pkg.py
Test: get_rust_pkg.py syn
Test: get_rust_pkg.py -v -o /tmp bitflags-1.2.0
Change-Id: I540b757a9dab56d4216015513d649b23063fb901
2020-03-27 12:41:29 -07:00
Automerger Merge Worker
7b95be8430 Merge "Update acov script to handle new coverage file locations." am: 834c8442cb am: 0b6aee253a
Change-Id: I6b4c2b75f047af0ec285094fe007d28e93f0c7d3
2020-03-12 18:24:07 +00:00
Oliver Nguyen
f7421ae9db Update acov script to handle new coverage file locations.
Test: Build with native coverage and run acov, check report contents.
Change-Id: I47afd35216cfcc7df7da2378cccdf8dba23bd8e2
2020-03-11 10:54:07 -07:00
Automerger Merge Worker
c726424b42 Merge "Add support for xxx.so!xxx.so lines." am: 197cd3d795 am: dc6db11f99 am: 094e559462
Change-Id: I55765fa70a454a849f53d59a255f225f7f49aa23
2020-02-01 04:53:20 +00:00