The original idea of using module specific soong variable was for the modules when they start off as optional module (before transition to mandatory). However, practically at build time, it can remain optional. For now, we will keep self.last_optional_release as the condition and the module can have their specific variable (for enabling/disabling prebuilts) by setting last_optional_release.
Bug: 301183608
Test: atest mainline_modules_sdks_test --host
packages/modules/common/build/mainline_modules_sdks.sh
Change-Id: I8af5d3dd8733d244c2ade1d9b5118308b3c00622
Add list of all the sdk supported modules in sdk-modules.txt. This file
should generate for all builds (even when only non-sdk modules build is
requested).
The file only contains modules which are requested by the build, and
supports sdk.
Bug: 242316893
Test: Generate sdk-modules.txt in the dist directory root.
Test: atest --host mainline_modules_sdks_test --no-bazel-mode
Change-Id: Ic8aa3c63209717e1076c4c51f1232dd5afcdd6d4
This CL builds the .latest.version target which contains the last
finalized version. This version is read and populated inside the
metadata file.
The CL also verifies whether all the java sdk libraries under a module,
which has been finalized atleast once, have the same extension version.
Bug: 242316893
Test: atest --host mainline_modules_sdks_test --no-bazel-mode
Change-Id: I94acf448b25d76c45a4bfd3d54fc276e2f01c7cc
Add additional_transformations for module sdk build. This feature is
only supported by S+ releases.
Bug: 254111089
Test: atest mainline_modules_sdks_test --host
Ignore-AOSP-First: Check into internal branch first to make sure all
testing works.
Merged-In: I94989519011e31c7db33656c6730c4f8fd5e0a4f
Change-Id: Idc8bd682550f59e086c431b0be38bfbdf8a75127
Pass BuildRelease to apply and _apply_transformation methods in class of
transformations, so that transformation could use the build_release
information to decide whether certain steps of the transformation should
be performed.
Test: atest mainline_modules_sdks_test --host
Bug: 254111089
Merged-In: If71a921752f4c481030efb0fd391fb328b5dffb6
Change-Id: I8341541572072ef72ab37dfe816296a1375e4cea
This CL creates a metadata json file for each module that support java
APIs. The metadata file contains the information about -
1. the api-diff file name
2. the size of the api-diff file
3. module extension version
More information will be added to this metadata file, which will be
used by the mainline module api checks gantry gate to improve its
functionality.
Bug: 242316893
Test: Generate metadata file for each module. Passes all tests.
Test: atest --host mainline_modules_sdks_test --no-bazel-mode
Change-Id: I517e59e5a7bb0f6675950d7455a724816d599692
Soong no longer supports the SOONG_SDK_SNAPSHOT_VERSION environment
variable so there is no need to set it. This change removes the setting
of that environment variable and stops passing the sdk_version around.
The few places that need to access the sdk_version in order to
construct file names just use the SDK_VERSION global variable which is
always "current" for legacy purposes.
Bug: 259095197
Test: BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh
# Run the above before and after this change and compare the
# out/dist/*mainline-sdks directory contents to ensure that there are no
# changes.
Change-Id: I9554eb48dfd04877d09d7e8101129d0fe1724101
Bug: 218685706
Test: BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh --build-release=R
# Run before and after making this change and verify that the
# only difference is to remove test_com.android.tethering.
atest --no-bazel-mode --host mainline_modules_sdks_test
Change-Id: Ic24ce4afbe9f84b1afd65dca9cee2154dc436646
Previously, this imported the soong_config_module_type definitions from
a manually curated file. That had a couple of problems:
1. It required that the file be manually updated everytime that a new
module_type was added to an sdk snapshot controlled by a specific
config variable.
2. Adding a new config variable required creating another file to be
manually curated.
3. It made a snapshot that was generated for an earlier release
dependent on an additional file from that release which was in a
separate repository and would require a lot of extra work to update
if necessary.
This change generates the soong_config_module_type inline in the
snapshot Android.bp file and while it makes it slightly bigger it makes
each snapshot much more self contained and makes it easier to add
module specific config variables.
The latter is the main driver for this change as follow up changes will
add module specific variables for the optional modules.
Due to bug 235475711 each soong_config_module_type has to be unique
across all mainline modules so this uses both the module short name
and a prefix associated with the SdkType to generate unique module
names. However, that only affects the S snapshots as from T onwards is
is not necessary to use soong_config_module_type modules at all.
Bug: 233965247
Test: atest --host mainline_modules_sdks_test
packages/modules/common/build/mainline_modules_sdks.sh
# Extract art sdk and module_exports snapshots into
# prebuilts/module_sdk/art and then run "m nothing"
Change-Id: Ib0e2ece1779451dffff6dfaca7ca39b264004b5b
Previously, T+ sdk snapshots used soong_config_module_type boilerplate
like for S. This change switches T+ sdk snapshots to use the
use_source_config_var property that was added in T.
Bug: 235455192
Test: atest --host mainline_modules_sdks_test
packages/modules/common/build/mainline_modules_sdks.sh
# Extract art sdk and module_exports snapshots into
# prebuilts/module_sdk/art and then run "m nothing"
Change-Id: I65cd462358bceede493623e6a51ffdb36bfc1885
In order to detect the addition of any new APIs in any mainline module,
an API diff file (current module sdk vs the last finalized module sdk)
needs to be generated correpsonding to each module sdk.
This CL generates the API diff files corresponding to each module sdk
and copy that to the out/dist directory.
This CL also contains the tests to verify the generation of api diff
files.
Bug: 230609867
Test: builds successfully. Generates API diff files for each module sdk
"atest mainline_modules_sdks_test" passes.
Change-Id: I1d8276a4054b92490ccca3dae36c48c639925f46
Optional modules, i.e. those modules which may be provided by Google or
vendors depending on the vendor, need to have its own Soong config
variable that controls whether prebuilts are used or not. Without that
the build will always attempt to use the Google prebuilt module instead
of the vendor provided module.
This change:
1. Adds support for specifying which modules are optional and will
generate a module specific soong_config_module_type that uses a
module specific Soong config variable.
2. Generates the soong_config_module_type for optional modules inline
in the snapshot Android.bp file (instead of importing from a
manually curated definitions files). That simplifies the cost of
adding optional modules.
3. Adds some extra tests to ensure that S and Tiramisu behave the
same way.
Bug: 233965247
Test: atest mainline_modules_sdks_test
packages/modules/common/build/mainline_modules_sdks.sh
# Check the output to ensure that wifi uses the wifi specific
# Soong config but ipsec (as a non-optional module) does not.
# Unpack the wifi snapshot into prebuilts/module_sdk/Wifi
Change-Id: I6a85b6f9877fc251010ff2bbee75fe8fa99db9b4
This reverts commit d29f8e0597.
Although I tested this well using unit tests they don't test
everything and although I tested the unpacking of some snapshots I did
not test what happens when all the ART sdk/module_exports
snapshots are unpacked. I just did and the build broke.
The build broke because while the soong_config_module_type modules are
marked as being Nameless (and so according to the documentation do not
have to have unique names) it turns out that is not true.
There is at least one place in the build where the names of Nameless
modules are checked and as a result this breaks.
I could fix the build breakage reasonably easily but the problem is
that I would have to apply that fix on all partners otherwise they
wouldn't be able to receive the next set of snapshots which is too
high a cost.
So, I am reverting this and will limit the use of inline
soong_config_module_type modules to optional modules.
Reason for revert: Breaks the build when snapshots are unpacked.
Bug: 233965247
Change-Id: I1faae863c48f1fbfd49d2545f242401bdd4f8f8a
Previously, this imported the soong_config_module_type definitions from
a manually curated file. That had a couple of problems:
1. It required that the file be manually updated everytime that a new
module_type was added to an sdk snapshot controlled by a specific
config variable.
2. Adding a new config variable required creating another file to be
manually curated.
3. It made a snapshot that was generated for an earlier release
dependent on an additional file from that release which was in a
separate repository and would require a lot of extra work to update
if necessary.
This change generates the soong_config_module_type inline in the
snapshot Android.bp file and while it makes it slightly bigger and does
duplicate some of these definitions in snapshots that use the same
config variable it makes each snapshot much more self contained and
makes it easier to add module specific config variables.
The latter is the main driver for this change as follow up changes will
add module specific variables for the optional modules.
Bug: 233965247
Test: atest mainline_modules_sdks_test
packages/modules/common/build/mainline_modules_sdks.sh
# Compare before and after to make sure that they are
# consistent.
Change-Id: Ie6b6d99e1fcb17ebd8ed291a5bc7d4664fceea92
In order to detect the addition of any new APIs in any mainline module,
an API diff file (current module sdk vs the last finalized module sdk)
needs to be generated correpsonding to each module sdk.
To obtain the list of APIs in the last finalized module sdk, the latest
build targets correpsonding to each scope of each module sdk are
obtained and needs to be built.
This CL is fetching all the module sdk scope targets from the .info
file corresponding to each module sdk file and then building all those
targets.
This CL is also adding changes to the test file
mainline_modules_sdks_test.py to mock the generation of module sdk
snapshot info files and the latest api text files for module sdk
scope targets.
The follow up CLs will generate the API diff files corresponding to
each module sdk and copy that to the out/dist directory. And will
also contain the tests to verify the generation of api diff files.
Bug: 230609867
Test: build packages/modules/common/build/mainline_modules_sdks.py
successfully and builds all the scope targets for each module sdk.
"atest mainline_modules_sdks_test" passes
Change-Id: I9e0c08d3a8bbf60e9b5cb61389c155948c4c171e
This change will stop building the following directories:
* mainline-sdks/current
* stubs
Bug: 204763318
Test: packages/modules/common/build/mainline_modules_sdks.sh
Change-Id: I9ed17ff472bfd36d1b5baa365ba33b9e2e283412
This change adds the build_number.txt file into the sdk snapshot for R
to make it easier to debug issues arising in partner builds caused by
the snapshots.
The file is added to the root of the snapshot and is called:
snapshot-creation-build-number.txt
Bug: 201295663
Test: atest --host mainline_modules_sdks_test
packages/modules/common/build/mainline_modules_sdks.sh --build-release=R
# Check the contents of various snapshots.
Change-Id: I6cabf16754c73fc87ee4e9d00bf6ddb8349809f1
Previously, the snapshots would be built in the order in which the
build releases were defined. This change groups the snapshots with the
same build environment and builds them in sequence in order to minimize
the regeneration of the ninja files.
The test revealed that the produce_bundled_dist_for_build_release
method would call build_snapshots even if there were no bundled
snapshots needed which was a waste so that was also optimized as part
of this change.
Test: atest --host mainline_modules_sdks_test
packages/modules/common/build/mainline_modules_sdks.sh
pyformat -s 4 --force_quote_type double -i build/mainline_modules_sdks*.py
/usr/bin/pylint --rcfile $ANDROID_BUILD_TOP/tools/repohooks/tools/pylintrc build/mainline_modules_sdks*.py
Change-Id: I5bb8d864a50318b4b5390a8e5a3601aa33c18fef
This test is added to show the current order so that the impact of a
follow up change that changes the order can be assessed.
Test: atest --host mainline_modules_sdks_test
pyformat -s 4 --force_quote_type double -i build/mainline_modules_sdks*.py
/usr/bin/pylint --rcfile $ANDROID_BUILD_TOP/tools/repohooks/tools/pylintrc build/mainline_modules_sdks*.py
Change-Id: I738cc90024ebc2bec59a7193e1b1f42956763ef8
Moves the generated Android.bp file from the root of the snapshot zip
file into the sdk_library/ sub-directory to allow the zip to be
unpacked in a directory that already contains an Android.bp file that
defines the apex_set for the corresponding APEX.
Also, removes the stubs_src property from the generated Android.bp
file.
Bug: 218685706
Test: packages/modules/common/build/mainline_modules_sdks.sh --build-release=R
atest --host mainline_modules_sdks_test
# deleted vendor/unbundled_google/modules/IpSecGooglePrebuilt from rvc-dev checkout.
# unpacked out/soong/mainline-sdks/for-R-build/ipsec-module-sdk-current.zip
# ran "m nothing"
Change-Id: I03e2060dc06338a8d0102744f79c75d33d567a17
Mainline.
These are used in the ART thin manifest. They are added here to allow
using the common module build scripts for them as well.
The Python script is extended to work for these modules, and their SDKs
end up in DIST_DIR/bundled-mainline-sdks, so DIST_DIR/mainline-sdks
only carries the SDKs for unbundled modules that are to be distributed
to partners etc.
This change has the effect to (re)enable building these SDKs in CI.
Test: env TARGET_BUILD_APPS="com.android.i18n com.android.runtime com.android.tzdata" \
packages/modules/common/build/mainline_modules_sdks.sh \
--build-platform-sdks-for-mainline --build-release=latest
Replace SDK snapshots under prebuilts/runtime/mainline with
those produced in out/dist/bundled-mainline-sdks
in a master-art tree, with https://r.android.com/2046463 applied,
then `banchan com.android.art && m`
Test: env -u TARGET_BUILD_APPS \
packages/modules/common/build/mainline_modules_sdks.sh \
--build-release=latest
Check that all SDKs are built, and that the bundled SDKs get copied
to out/dist/bundled-mainline-sdks so the contents of
out/dist/mainline-sdks and out/dist/stubs remain unchanged.
Test: env -u TARGET_BUILD_APPS \
packages/modules/common/build/mainline_modules_sdks.sh \
--build-release=S
Check that out/dist/bundled-mainline-sdks isn't producted.
Test: env -u TARGET_BUILD_APPS \
vendor/google/build/mainline_modules_sdks.sh
Check that all SDKs are built, and that the bundled SDKs get copied
to out/dist/bundled-mainline-sdks so the contents of
out/dist/mainline-sdks and out/dist/stubs remain unchanged.
Test: atest --host mainline_modules_sdks_test
Bug: 216762037
Change-Id: Ie6182fa42dc9c2c7f35ed95a040c05be0f0e878a
This change replicates the behavior of the generate_ml_bundle.sh that
creates an Android.bp file along with the necessary file structure for
an R build.
It adds the information needed to a ForRBuild object which is set on
those MainlineModules that need to provide SDKs for R. That includes
a list of SdkLibrary objects each of which has a name and a flag that
indicates whether it is a shared library.
The R BuildRelease creator function builds the sdk snapshots as normal
and then extracts information from those snapshot files to construct
snapshots suitable for an R build. It then passes the directory
containing those new snapshot files to populate_dist() which will then
copy them into the correct location in the dist directory.
For each MainlineModule that provides an SDK for R the following is
created in the out/soong/mainline-sdks/for-R-build directory:
* A sub-directory using the apex name containing:
* An Android.bp file with definitions for each SdkLibrary.
* A file structure containing API txt file, removed API txt file,
a srcjar containing the stub sources and a stubs jar.
* A zip file containing the contents of the previous directory whose
name is <sdk-name>-current.zip, to match the files that are
generated by Soong when building sdk snapshots.
As R does not support the Soong config boilerplate code that
transformation is disabled for R.
Test: atest --host mainline_modules_sdks_test
packages/modules/common/build/mainline_modules_sdks.sh
pyformat -s 4 --force_quote_type double -i build/mainline_modules_sdks*.py
/usr/bin/pylint --rcfile $ANDROID_BUILD_TOP/tools/repohooks/tools/pylintrc build/mainline_modules_sdks*.py
Change-Id: I70001782496a3e9805bf56181c0e08419e31e191
Previously, the code for populating the dist directory, e.g.
populate_dist and create_legacy_dist_structures assumed that the sdk
snapshot files they needed could be found in the directory referenced
by SnapshotBuilder.mainline_sdks_dir. This change removes that
assumption and instead passes the snapshot_dirs directory into the
populate_dist method and uses it in the create_legacy_dist_structures
function.
This refactoring is needed to allow a follow up change to construct a
set of sdk snapshot files in a different directory while still using
the populate_dist method to copy them into the correct location in the
dist directory.
Bug: 218685706
Test: atest --host mainline_modules_sdks_test
packages/modules/common/build/mainline_modules_sdks.sh
pyformat -s 4 --force_quote_type double -i build/mainline_modules_sdks*.py
/usr/bin/pylint --rcfile $ANDROID_BUILD_TOP/tools/repohooks/tools/pylintrc build/mainline_modules_sdks*.py
Change-Id: I9fb659df28d7688b3f081bf84f5fae8ee85dc534
Adds a test for a com.google name module and makes the structure of the
test snapshot created by FakeSnapshotBuilder.create_snapshot_file()
match the names used in an actual sdk snapshot.
Bug: 218685706
Test: atest --host mainline_modules_sdks_test
pyformat -s 4 --force_quote_type double -i build/mainline_modules_sdks*.py
/usr/bin/pylint --rcfile $ANDROID_BUILD_TOP/tools/repohooks/tools/pylintrc build/mainline_modules_sdks*.py
Change-Id: Ia11cbb2da717218ba0b2da9a2238856dc8cebf8f
This changes adds support for building target build release specific
snapshots.
It adds a BuildRelease class that defines the characteristics of the
build release, e.g. name, how to create it, etc. It also adds a
first_release field to MainlineModule which specifies the first
BuildRelease for which a snapshot of the module is required and
initializes that field for each module.
After these changes this script will generate:
1. A legacy set of snapshots that match the file structure that was
generated without this change. This is intended to allow existing
consumers of the generated artifacts to continue to work while they
are modified to make use of target build release specific snapshots.
2. The set of snapshots for the latest build release, i.e. the build
release containing the source from which the snapshots are produced.
This includes snapshots for all the modules.
3. For each build release from S onwards a set of snapshots for the
modules it supports.
It does not currently generate snapshots for Q and R releases as Soong
cannot generate a compatible snapshot for them. If it is necessary to
generate snapshots for those target build releases (similar to what the
packages/modules/common/generate_ml_bundle.sh would produce) then a
follow up change will add that capability to this script.
Bug: 204763318
Test: atest mainline_modules_sdks_test
packages/modules/common/build/mainline_modules_sdks.sh
tree out/dist/mainline-sdks out/dist/stubs
- check that it adds the for-latest-build directory but is
otherwise identical to what was produced before this change.
Change-Id: I48eb0b69cbe8664106b826ee0898557c98e039c2
The variable is only used by the tests.
Test: atest mainline_modules_sdks_test
packages/modules/common/build/mainline_modules_sdks.sh
tree out/dist/mainline-sdks out/dist/stubs
- check that it is identical to before this change
Bug: 204763318
Change-Id: I38c4ce218825d79de831868af814c43ce8a6d60f
Previously, the produce_dist() method of SdkDistProducer was not
testable because it attempted to invoke Soong to build the sdk
snapshots. So, the TestPopulateDist tested the populate_dist() method
instead. That was a problem because it meant that future changes to
the produce_dist() method could not be tested, at least not without
duplicating its functionality.
This change extracts the logic for building the snapshots (i.e. the
code that invokes soong to build the snapshot files) from the
SdkDistProducer into a separate SnapshotBuilder class. That allows
the test to substitute a FakeSnapshotBuilder that generates some fake
sdk snapshot zip files and call produce_dist() instead of calling
populate_dist() and populate_stubs().
It also renames the test to TestProduceDist to reflect that change.
This is part of a larger refactoring to improve the testability of the
mainline_modules_sdks.py file in preparation for adding support for
building build release specific snapshots.
Test: atest mainline_modules_sdks_test
packages/modules/common/build/mainline_modules_sdks.sh
tree out/dist/mainline-sdks out/dist/stubs
- check that it is identical to before this change
Bug: 204763318
Change-Id: Icf2e4b0200cc53863e45cf68208fbc8ec13c6f2c
Change the list of modules to match AOSP list.
Bug: 205100626
Test: atest mainline_modules_sdks_test
Change-Id: I250c4aafcda5c3c1d636dac27ff5ffaa78b69b2a
