Commit Graph

3527 Commits

Author SHA1 Message Date
Kelvin Zhang
7c43b85a08 Merge "Support docker for ota generator" am: 091a8fb955 am: d2f6d63acd
Original change: https://android-review.googlesource.com/c/platform/development/+/1780768

Change-Id: Ie625a23636a1c9a58d31ac37bbe2f285d00ef671
2021-07-29 13:22:08 +00:00
Kelvin Zhang
d689350756 Support docker for ota generator
Test: docker run -it -p 8000:8000 -v target:/app/target -v
output:/app/output zhangxp1998/test:latest

Change-Id: Ife050e6a8c85f2d40aff82a752b19f27c2ab23c5
2021-07-28 22:44:34 -04:00
Treehugger Robot
d60a7bf334 Merge "Speed up the payload.bin read in process." am: 1a86d98b6e am: 261de8cf63
Original change: https://android-review.googlesource.com/c/platform/development/+/1777971

Change-Id: Id3330ed4f450df68740ff88feff51c35e37ab1c1
2021-07-29 02:03:36 +00:00
Kelvin Zhang
e5eb5c9d38 Speed up the payload.bin read in process.
A 700MiB OTA package now takes only 100ms to read in, compared with
7000ms previously. That's 70x faster.

When parsing the update_metadata in payload.bin, the entire file has to
be read in previously, because zip.js does not support partial read-in.
In fact, only a small portion of the payload.bin is update_metadata.
Reading the entire payload.bin not only slows down the process but also
occupy excessive memory.

The new class OTAPayloadBlobWriter (inherited from zip.Writer) will read
in the metadata (header, manifest, signature) and throw an StopIteration
exception when finished.

Test: open a large OTA package
Change-Id: Iebf8045325dae9a118d9d8ea5674872aa7c280c4
2021-07-28 23:38:51 +00:00
Treehugger Robot
97c97755ba Merge "Fix a compatibility issue: getBigUint64." am: 74232e7a3f am: 2daf325668
Original change: https://android-review.googlesource.com/c/platform/development/+/1781348

Change-Id: I7ac345518e30dcc8cb7929e89518ab6b160b3442
2021-07-28 22:15:34 +00:00
lishutong
ff2b99cd89 Fix a compatibility issue: getBigUint64.
The method getBigUint64 is not supported by safari until version 15:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView/getBigUint64
This CL mannual reload this method.

Test: Mannual tested.
Change-Id: I1d967eda93f9f628fb7f0f5d9f90efb92429ed5e
2021-07-28 20:31:37 +00:00
Treehugger Robot
e71ef39842 Merge changes I36beff43,Iab7275b9 am: 503e12a866 am: 4d21078ab9
Original change: https://android-review.googlesource.com/c/platform/development/+/1781347

Change-Id: Ibc02cc4e8879b290f5826f691a21f8bf400cb291
2021-07-28 20:10:50 +00:00
Treehugger Robot
49da6642eb Merge "Add prebuild and postbuild info to OTA_analysis." am: afcbcc7149 am: 2b8dd5e2b4
Original change: https://android-review.googlesource.com/c/platform/development/+/1779885

Change-Id: I12447e612c6e8191feadf72482a9edf8aefadd1b
2021-07-28 20:10:43 +00:00
lishutong
30f991bf6e Fix a bug: redirect to homepage for some url.
In a single page application, the url sometimes does not correspond to
the actual resources on the server. Thus, it might cause 404 when users
refresh the page. By redirecting this kind of url back to the homepage,
the proper page can be rendered. If the url does not exist, the front
end will redirect to a Not Found page.

Test: mannual tested.
Change-Id: I36beff436a450ae7fcabe9172df9c7cc217d7305
2021-07-28 18:14:35 +00:00
lishutong
c4b3dd1aef Fix a bug: multiple flags in ota generator.
Previously, when multiple extra flags are set, the subprocess.run
can start generation properly. Now this is fixed.

Test: mannual tested.
Change-Id: Iab7275b9058a088d1fafd8c445c7f0775626c1ba
2021-07-28 17:51:31 +00:00
lishutong
ad6b75df3d Add prebuild and postbuild info to OTA_analysis.
The metadata file in ota package contains prebuild and postbuild info,
now the basic info section will display those.

Test: mannual tested.
Change-Id: I264fe656ff6fab42d5161100c04210ab9a94c7a0
2021-07-28 17:51:29 +00:00
Treehugger Robot
235a534f1b Merge changes I5801bcc3,Ib2290546 am: 1584123dbb am: 9bca36aeee
Original change: https://android-review.googlesource.com/c/platform/development/+/1779346

Change-Id: I158327aac40b17d516c7a721dadd788e85ae63d9
2021-07-27 14:23:45 +00:00
lishutong
2232234d48 Remove the look behind regex for Safari.
The lookbehind/lookahead regular expression is not supported in safari
yet. (https://caniuse.com/js-regexp-lookbehind) Removed it for
compatibility reason.

Test: manual tested.
Change-Id: I5801bcc389b20df31175adbb8a841eb1d31703aa
2021-07-27 03:40:33 +00:00
lishutong
a3c85492cc Change the url to lowercase.
Test: mannual tested.
Change-Id: Ib22905462aa47aeaa700407fe5b90e3261d0b3a3
2021-07-27 03:33:48 +00:00
Treehugger Robot
0a461d9383 Merge "Add a demo page in OTA_analyzer." am: ceadd4cff1 am: 4c11f8c6e0
Original change: https://android-review.googlesource.com/c/platform/development/+/1778765

Change-Id: I258169f4a44674e835ef8b1f38dec5f2b6249dbc
2021-07-26 21:14:57 +00:00
Treehugger Robot
ceadd4cff1 Merge "Add a demo page in OTA_analyzer." 2021-07-26 20:39:20 +00:00
lishutong
cdf499330d Add a demo page in OTA_analyzer.
Please put an ota package and its target build as cf_x86_demo.zip and
cf_x86_target_file_demo into this directory: /public/files/

This OTA package and target build do not have to be complete - only the
manifest part in OTA package and .map files in target build are
necessary. The previous one can be generated using:
https://source.corp.google.com/android/vendor/google_tradefederation/contrib/src/com/google/android/tradefed/ota/util/PayloadUtil.java;l=196;bpv=1;bpt=1?q=PayloadUtil&sq=package:android
The following one can be generated by unzip and keep the .map files
only, zip again.

Add a demo page, now the user can view the complete function of
OTA_analyzer without uploading their own ota packages / android build.

Test: Mannual tested.
Change-Id: I7552e0222fc40e9a4b1aff9750f74cd3ed3f4feb
2021-07-26 17:51:51 +00:00
Treehugger Robot
da5830d990 Merge "Add support for chain OTA generation." am: 4895d0a8b4 am: d5b6e875b0
Original change: https://android-review.googlesource.com/c/platform/development/+/1776605

Change-Id: I629768bbec7e0d9f7ae4ad47c697c70ab3ab89d1
2021-07-26 17:29:50 +00:00
Treehugger Robot
b7b945a034 Merge "Add batch generation to OTAGUI." am: 2e1cfae4fd am: 2f8719ccad
Original change: https://android-review.googlesource.com/c/platform/development/+/1775726

Change-Id: I8a58ce94715ee7034827c6a4bf0b6e0a0f60edd7
2021-07-23 23:07:22 +00:00
lishutong
bb3bc961d4 Add support for chain OTA generation.
To upgrade from the Android version A to Android version D, one could
directly generate an OTA package from A to D. But chances are (a) this
OTA package can be large and unstable (b) there are multiple other
devices are on version B or C. So generation of chain OTA packages
(A-->B-->C-->D) can help life easier.

Users will be able to select and sort the Android build in
`ChainOTAOptions.vue` component, and submit multiple jobs at the same
time using a OTAConfiguration from `JobSubmission.js`.

Test: Mannual tested.
Change-Id: I9f16f981af80900c18a571162146ce218ea96387
2021-07-23 19:52:28 +00:00
lishutong
c4b6c3a7dd Add batch generation to OTAGUI.
Batch generation of OTA packages is a important feature
requested by googler and partners: please refer to go/ota-dashboard-doc.
Given n incremental source builds and m target builds,
batch generation will generate n x m OTA packages in total.
If n=0, full OTA package will be generated.

The front end will be taking in the source/target lists and send the
request to backend one-by-one.

Test: mannual tested.

Change-Id: I769359ee69c7aa8c71704c4e119c374635554dfb
2021-07-23 19:18:52 +00:00
Treehugger Robot
dd360e3272 Merge "Refactor the frontend for batch generation." am: 89e47defef am: 8f08ab906f
Original change: https://android-review.googlesource.com/c/platform/development/+/1775725

Change-Id: I16cc34cf94fc78d7e0e5e741b7295aeb7dfc04a3
2021-07-23 18:17:50 +00:00
lishutong
aaf2c7a933 Refactor the frontend for batch generation.
Following part has been modified:

1. Add tabs for selection between single generation and batch
generation. (src/components/JobConfigure.vue)

2. Change the data structure of OTAConfiguration, now it only records
the flags. The source/target build will be provided when submit jobs.
(src/services/JobSubmission.js)

3. Seperate the OTAOptions as a single component, which only takes in
the flags for backend. The selection of source/target build will be in a
seperate component. (src/components/OTAOptions.vue,
src/components/SingleOTAOptions.vue).

4. Now the partition selection can takes in more than one build, but
only show the partition list of first one. Later on, this will be able
to show the intersection of the partition lists from all given builds.
(src/components/PartialCheckbox.vue)

Point 1 enables the possibility of the dynamical loading of single/batch
ota generation pages. Point 2,3,4 allow the OTAOptions components to
be reused for batch generation.

Test: Mannual tested.
Change-Id: I1a29fa7c605596d717d19da25d31b81ce5b8fcba
2021-07-23 15:09:31 +00:00
Treehugger Robot
5d51e2905e Merge "Fix a bug that job might not be able to be started properly." am: c8c2db4db6 am: d46f1bf46a
Original change: https://android-review.googlesource.com/c/platform/development/+/1773226

Change-Id: Ia97fc623290b246705bafb32e2d1579cd968bde9
2021-07-20 21:37:59 +00:00
Treehugger Robot
c8c2db4db6 Merge "Fix a bug that job might not be able to be started properly." 2021-07-20 21:17:41 +00:00
Treehugger Robot
ea8c89a7f0 Merge "Add all possible flags to the front end." am: 2f5326c0d6 am: 0bfc59bfc4
Original change: https://android-review.googlesource.com/c/platform/development/+/1771648

Change-Id: I563cf8d6fdbb5f19cc46547bb149271fa0bf562a
2021-07-20 20:07:28 +00:00
Treehugger Robot
2f5326c0d6 Merge "Add all possible flags to the front end." 2021-07-20 19:28:33 +00:00
lishutong
f084110dbe Fix a bug that job might not be able to be started properly.
A variable name was mispelled and it could lead to jobs unable to be
started.

Test: Mannual tested.
Change-Id: I4067e2ae243428cb190463f55122b88d471f45f7
2021-07-20 18:27:07 +00:00
Kelvin Zhang
13e7f84a1a Merge changes I1e45d3ef,I4117f5d9 am: c37be89c82 am: 48666960dd
Original change: https://android-review.googlesource.com/c/platform/development/+/1772906

Change-Id: Ib892cf002e5a290f151b481b8f438efa4f8b9052
2021-07-20 18:17:59 +00:00
Logan Chien
8416619ba9 Merge "repo-review: Trim trailing slashes to avoid errors" am: 8ce5136cb5 am: 277c298174
Original change: https://android-review.googlesource.com/c/platform/development/+/1771795

Change-Id: I713ca4a1c8ce80b68a4701def3642afac75c5d50
2021-07-20 18:16:46 +00:00
Kelvin Zhang
c37be89c82 Merge changes I1e45d3ef,I4117f5d9
* changes:
  Show the old and new partition infos in OTA analysis.
  Change the look of cross emoji.
2021-07-20 16:41:04 +00:00
Logan Chien
8ce5136cb5 Merge "repo-review: Trim trailing slashes to avoid errors" 2021-07-20 16:40:48 +00:00
lishutong
c0f6a27222 Show the old and new partition infos in OTA analysis.
Test: Mannual tested.
Change-Id: I1e45d3efaefc7368a72f1ed59d09fd48f01b4132
2021-07-20 16:33:49 +00:00
lishutong
85044ed571 Change the look of cross emoji.
Test: Mannual tested.
Change-Id: I4117f5d9986aece6da8c25757baed8f750cc438e
2021-07-20 15:46:13 +00:00
lishutong
fc0431fda1 Add all possible flags to the front end.
A complete set of flags for the CLI tool can be found in this script:
https://source.corp.google.com/android/build/make/tools/releasetools/ota_from_target_files.py

Now most of the flags are being added to the front end. However, there
are several TODOs:
1. flags that require extra arguements or file operations are not
supported yet.
2. the flags can depend or exclude on other flags. Dependency is
included in file 'JobSubmissions.js'-'OTAExtraFlags'. But the function
is not implemented yet.

Test: Mannual tested.
Change-Id: I3b6b76d48861cff81bfad86b549ff53f6536933b
2021-07-20 15:31:47 +00:00
Treehugger Robot
98a9756c95 Merge "Fix a bug in build library: the uploaded file cannot be shown in table." am: 9b339b3bc1 am: 7eb899863a
Original change: https://android-review.googlesource.com/c/platform/development/+/1771647

Change-Id: Iccedafeeaa3efd0a6166f79919f84282b4420584
2021-07-19 22:43:10 +00:00
Treehugger Robot
9b339b3bc1 Merge "Fix a bug in build library: the uploaded file cannot be shown in table." 2021-07-19 22:15:54 +00:00
Logan Chien
4ee322f25b repo-review: Trim trailing slashes to avoid errors
Bug: b/170858163
Change-Id: I3de16ad77f0436af97eae26816fc99493fba30c3
2021-07-19 14:59:29 -07:00
Treehugger Robot
295fe8afa2 Merge "Move the '--partial' flag and output configuration to the backend." am: 52c76356a5 am: 8b0fdc9535
Original change: https://android-review.googlesource.com/c/platform/development/+/1771645

Change-Id: Ic2250c00e2de9a7fdaaef9c361b81a0495ab91d7
2021-07-19 21:59:23 +00:00
Treehugger Robot
52c76356a5 Merge "Move the '--partial' flag and output configuration to the backend." 2021-07-19 21:27:55 +00:00
lishutong
b9d9f55657 Fix a bug in build library: the uploaded file cannot be shown in table.
Previously, the newly uploaded file cannot be shown immediately in the
datatable, unless refresh the page. Now the bug is fixed.

Similar bug is also fixed in OTAJobTable.vue.

Test: mannual tested.
Change-Id: Ia0531db884a7e1675b2cf51aeb8ca0847694c63c
2021-07-19 20:45:46 +00:00
Treehugger Robot
9dd7474b0f Merge "Add VAB, VABC state checker to OTA_analyzer." am: 8cfacd4509 am: 00a74639a1
Original change: https://android-review.googlesource.com/c/platform/development/+/1771646

Change-Id: Ibc3c22bcd108c3c8e369df2eace8f231c28cc2e1
2021-07-19 19:52:07 +00:00
Treehugger Robot
8cfacd4509 Merge "Add VAB, VABC state checker to OTA_analyzer." 2021-07-19 19:22:31 +00:00
Treehugger Robot
b7d6e8d966 Merge "Refactor the OTA configuration page using several components." am: c532190082 am: 5798a15596
Original change: https://android-review.googlesource.com/c/platform/development/+/1770745

Change-Id: I01d876edbe173d7a83a9af4be0200d5000f5ef45
2021-07-19 19:07:15 +00:00
lishutong
029dbe681e Move the '--partial' flag and output configuration to the backend.
Previously, the front end will generate a string for '--partial' flag.
Now the front end will only record the partial partitions that are going
to be updated as an array, and the backend will convert the array into a
string for CLI tool.

The output now is also calculated in the backend,

Test: mannual tested.
Change-Id: Ib5604e52b0c0d0cdd2bf85c71479b3133479da74
2021-07-19 17:57:07 +00:00
lishutong
f506b57791 Add VAB, VABC state checker to OTA_analyzer.
Add BasicInfo.vue to show some basic facts of the ota package,
including:
- is partial or not
- is incremental or not
- is VAB or not
- is VABC or not

Test: Mannual tested.
Change-Id: Ibc3e0a88ea03fc310b401ffe14c5b63d9e0f9452
2021-07-19 17:32:39 +00:00
lishutong
1ab3abb53a Refactor the OTA configuration page using several components.
The original code for OTA configuration in the frontend is tightly
coupled and hard to maintain. This CL decoupled the job configuration
component (OTAOptions.vue), build library (BuildLibrary.vue) and job
submission (JobSubmission.js).

The BuildLibrary.vue will present the
target build list and take selections. Selections will be sent to
OTAOptions.vue, where flags are set. Once the setting is complete, an
OTAOption object defined by JobSubmission.js will be sent to the
backend. This design can be viewed in go/ota-dashboard-doc : detailed
design - TASK CONFIGURATION.

This has several benefits: (a) easier for
future maintainence. (b) The original code cannot easily be reused for
batch OTA package generation. Now, only the OTAOptions.vue has to be
modified for this purpose.

Test: Mannual tested. Unit tests will be added in following CLs.

Change-Id: I846d0c242cd6ed51478dd4d1a3e4c5fb8878aba1
2021-07-19 16:22:41 +00:00
Treehugger Robot
fe8f41643b Merge "Add responsive design to the UI." am: 1f1f1f7fed am: 67d67e64be
Original change: https://android-review.googlesource.com/c/platform/development/+/1770425

Change-Id: I555e682cc3fbb40c2ea8fdee1d980b8c870e3c05
2021-07-17 02:40:09 +00:00
Treehugger Robot
cf992360dc Merge changes If4e8ff72,I53d861fb am: 91496f424d am: b797b91fca
Original change: https://android-review.googlesource.com/c/platform/development/+/1770026

Change-Id: I19be31d438bfcb203a6369ccac9da7a3e1f97351
2021-07-17 01:23:07 +00:00
lishutong
be609f5b31 Add responsive design to the UI.
Now the columns width can be adjusted according to the window size.

Test: Mannual tested.
Change-Id: I1e2911b757948ee95f487631bcdceb5ede1fa63e
2021-07-16 19:38:16 +00:00