Commit Graph

168 Commits

Author SHA1 Message Date
Michael Bestas
f33c681acf extract_firmware: Unsparse images before copying
Some OEMs like motorola ship sparse images which can't be flashed as-is

Change-Id: I6f5a7e058e02167c242b0a5053fa4f6cc3a351ea
2024-01-02 23:08:02 +02:00
Michael Bestas
696f89ff21 extract-utils: Extract A/B OTA partitions in parallel
Before:
real	1m32.730s
user	1m20.466s
sys	0m11.764s

After:
real	1m10.794s
user	1m21.270s
sys	0m11.592s

Change-Id: Idd4bc94ccb00d4e80ba1aa2efb86f1f29c5667ae
2023-12-13 19:51:16 +02:00
Michael Bestas
644d397e14 extract-utils: Improve KEEP_DUMP logic when extracting from OTA
Change-Id: Ib5ad6e1bc3a7cefd6ffdee3bf1e0ff9840233129
2023-12-13 19:44:44 +02:00
Michael Bestas
222c8a6863 extract-utils: Switch to ota_extractor
The python tools no longer work and google recommends ota_extractor.
This opens up the possibility for incremental extraction as well.

Change-Id: I03a2ce5d75d192f00ebd604d13694f8694965192
2023-12-13 18:01:30 +02:00
LuK1337
f6abdcd93f extract-utils: Use newly introduced add-radio-file-sha1-checked macro
Change-Id: Iccd88254a94971c8c2b402c34ff62320b6ea42c7
2023-12-11 14:03:27 +01:00
LuK1337
700f71489e extract-utils: Add support for extracting from /vendor/odm
Test: ./extract-files.sh --section WiFi with guacamole images
Change-Id: I6d8409f9cefb238c2498e0103f6937bd6ecfd222
2023-12-05 14:04:50 +00:00
LuK1337
9c40eb661a extract-utils: Add support for EROFS extraction
Change-Id: I08a83851bb2f35b68291f1df2fb961c6fb9c82bc
2023-12-05 14:04:19 +00:00
Michael Bestas
e1f42577b2 extract-utils: Add prepare_firmware function
Allows devices to run required actions before attempting
to extract firmware.
Example: motorola star images & pixel fbpk images.

Change-Id: I105c3a17c10f63f0ad72966354aed6baed695d7c
2023-11-30 20:54:00 +02:00
LuK1337
9fa2faa69a extract_firmware: Add support for adb extraction
Change-Id: I2904996e70546ac75b2a8d36e293f7dbb559b2bb
2023-11-30 20:28:22 +02:00
Michael Bestas
f626f9128a extract_firmware: Skip files that can't be found
A/B OTA might be missing some partitions like devinfo and frp
which are required for factory images.

Change-Id: I9d6f91c736476d3540c541e3ffc7a28e72089e4a
2023-11-12 14:13:28 +02:00
Chirayu Desai
88be24b089 extract_firmware: Support A/B OTA
Change-Id: I845c62a63aaab223c0fb5f49a494d4aa1c165c92
2023-11-12 14:13:28 +02:00
Michael Bestas
6af359d867 extract_firmware: Handle renamed firmware files
This allows to extract firmware from OTA images
instead of factory on FP4

Change-Id: Ie52e85f63a3a6e48551a0cd8b6899cc542444587
2023-11-12 14:13:28 +02:00
Michael Bestas
4e4fff479f extract-utils: Simplify firmware extraction support
Merge the two functions into one and explicitly list all partitions
based on proprietary file list instead of scanning the radio folder.

Change-Id: I28a426af04efa5e103dc86bd369f507187e49489
2023-11-12 14:13:28 +02:00
Michael Bestas
5e451aa5f8 extract-utils: Rework firmware extraction support
This allows extracting firmware from factory images.
The following changes have been made:

* Remove filesmap support, this is not used in AOSP releasetools.
* Remove directory check, append_firmware_calls_to_makefiles() is opt-in
  and if some device calls it, firmware will always exist.
* Implement append_firmware_ab_ota_to_makefiles() to automatically
  append firmware images to AB_OTA_PARTITIONS
* Add support for source:target firmware extraction and AB argument

TODO: extract firmware from OTA images

Change-Id: Ib2158925f53657af949a186200c9e98e1a4db810
2023-11-12 14:13:27 +02:00
Woomymy
d3a36605d6 extract_utils: Add patchelf 0.17.2 support
Not enabled by default as >0.9 versions where reported to break some
libraries (see change I6c0cbe76baf241233fdd125f5ed1eccbd5855124), but
patchelf 0.8/9 also seems to break certain libraries while 0.17.2
doesn't. See I079925c0a38a08661accbbf07831e36e6adb4d76 for example.

Signed-off-by: Woomymy <woomy@woomy.be>
Change-Id: Ib68152e7936d8c618d29278a264a141cbab77d05
2023-11-04 11:20:15 +00:00
Sebastiano Barezzi
4b5515a656 extract-utils: Try to use arch specific vdex for JARs
* Also don't print errors when no classes are found

Change-Id: I0c88716f2214f90948378a8773b3d645f6dc3abc
2023-08-10 16:07:13 +02:00
Michael Bestas
561ce192be extract_utils: Remove super.raw after dumping partitions
This saves some space in case we want to keep the extracted content
via the KEEP_DUMP flag.

Change-Id: I0f5812853b5bff8b303a276047bb30d58690920d
2023-06-12 19:58:46 +03:00
Michael Bestas
7e3eb4d7d1 extract_utils: Don't extract multiple times for the same run
Just reuse already extracted content.

Change-Id: I6d41f12258255aedb1db46d619b55abb7527110b
2023-06-12 19:58:46 +03:00
Michael Bestas
c1ea677d24 extract_utils: Add option to save extracted content
Change-Id: Ia38ebcd616d17fc64c59bdb245c4f18e97c322f4
2023-06-12 19:58:43 +03:00
Michael Bestas
40dbd6b9cd extract_utils: Add helper function for carriersettings_extractor.py
Change-Id: Iaa0196054f39fccc86a34dea1243db0963216f31
2023-06-12 14:41:27 +03:00
Michael Bestas
d469bc3a95 extract_utils: Add helper functions to generate RRO package structure
To be used for CarrierSettings extract integration.

Change-Id: I116efbb75aa53d9117076ff0e4e8fa239ab8002f
2023-06-12 13:02:35 +03:00
Michael Bestas
d5ad621ad4 extract_utils: Add helper functions for single items
Allows copying single files or building single packages.
To be used for CarrierSettings extract integration.

Change-Id: I9238355c56673f633e7a0a4afe70e9a9ad62af56
2023-06-12 13:02:35 +03:00
Michael Bestas
0f8eccc3f5 extract_utils: Add support to generate carriersettings file list
Change-Id: Ia92e4f12c22eaf1a25f116f6928540345e3a5e1d
2023-06-12 13:02:35 +03:00
Michael Bestas
09c751c242 Support extracting from sparsechunk super images
Change-Id: If029d717f8001079dab4382b1c91e8b092aae017
2023-03-31 20:47:26 +03:00
pjgowtham
9724a2a410 extract-utils: Allow overriding VENDOR with VENDOR_COMMON
Co-authored-by: LuK1337 <priv.luk@gmail.com>
Change-Id: Ie2f2d686d0803269bb6a7f7ef5796c7cbce75854
2023-03-06 23:51:55 +01:00
Chirayu Desai
cf1fe40c78 Stop using TMPDIR
* This is used by a lot of things, and the script removes
  the entire directory. That can cause issues with other things,
  that don't expect the root TMPDIR to go away, or some contents
  within it.
* Let's use our own variable
* You can still keep setting TMPDIR since mktemp respects that

Change-Id: I6c71ce38bab781cd7a8d285499f127c2bf248eb8
2022-10-26 11:15:00 +02:00
LuK1337
56fbdef3ee extract_utils: Sync adb pull paths with local extract
Fixes extracting the following example line from device:
* system/framework/WfdCommon.jar

Change-Id: I690a871026b89ca591943a3bba3c09310ae821dc
2022-10-15 22:58:51 +02:00
Mohd Faraz
3d1e865abe extract-utils: Fix for sh_binary
* srcs, prefer isn't available in sh_binary

Change-Id: I5f94432748aba4a417520ba850b7a9c4d7dddbe7
Signed-off-by: Mohd Faraz <androiabledroid@gmail.com>
2022-10-13 10:34:28 +02:00
Mohd Faraz
23e4b57ec0 extract_utils: extract recovery and vendor_ramdisk
* This will be beneficial for copying firmware and other stuff to recovery.

Change-Id: Ibc7e887d32f9c00d5914ed4d2c9d79864f1f4745
Signed-off-by: Mohd Faraz <androiabledroid@gmail.com>
Signed-off-by: Mohd Faraz <mohd.faraz.abc@gmail.com>
2022-10-12 23:21:04 +02:00
Chirayu Desai
ebdf73f5b4 extract-utils: egrep: warning: egrep is obsolescent; using grep -E
Change-Id: I99afd62ec559af16ce09eb3d8df8ad29fdb54231
2022-09-28 00:05:07 +02:00
Luca Weiss
04563775b3 extract_utils: fix generating system/bin & system/etc blueprint
Previously the wrong variables were checked to see if the prefix
matches, so e.g. no cc_prebuilt_binary were generated for those
binaries.

Change-Id: I744c6f6a4584921a26547478ac14a160c56e07b7
2022-09-27 14:52:41 +02:00
Chirayu Desai
929c9b99ec Allow re-using extracted content
Usage:
export TMPDIR=/some/dir/$device
export SKIP_CLEANUP=true

Change-Id: I41b992708ca5fc7bc377b22a418f5fd1395cd08a
2022-08-04 20:10:38 +02:00
Chirayu Desai
7ae0b6f6ec Split up setup_vendor
* For usage by regen-vendor.sh

Change-Id: Ideb3d4ea68e7aa6ef4a5a73c15d507d47fda4b56
2022-04-05 03:32:25 +05:30
Michael Bestas
a3457803b7 extract_utils: export SIGSCAN
For use in device scripts

Change-Id: I7a8c9abd0b585a67963e65d5ad042da79d4d3c77
2022-03-23 23:15:35 +02:00
Michael Bestas
ebae91b493 extract_utils: Support extracting from super.img
Change-Id: I2a30da25f2d55df1080b1795e08a14195c1e3ec0
2022-03-23 23:14:41 +02:00
Chirayu Desai
d5dc2b694c Use simg2img from prebuilts/extract-tools
Change-Id: I15772394a24c2ff1fdd6eece86548a72a5d4d748
2022-03-17 06:27:33 +05:30
Chenyang Zhong
2c5c8df873 extract_utils: sort the package list before writing blueprints
"comm" requires both input to be sorted. The T_X_LIB32 and T_X_LIB64
arrays from prefix_match-ing PRODUCT_PACKAGES_LIST may not always
be sorted, which causes certain multilib to be mis-categorized as
32-bit and 64-bit separately. The final blueprint would then have
duplicated targets.

Therefore, sort the package list first so that arrays generated
from the package list are also sorted. Set LC_ALL=C to avoid "sort"
using system locale.

Change-Id: Ia2bc9733d89de66eda52059237a54d624ba4b0ad
Signed-off-by: Chenyang Zhong <zhongcy95@gmail.com>
2022-02-15 23:54:33 +01:00
Michael Bestas
3521e25edc extract_utils: Remove support for /sbin
Android 11 has removed /sbin support completely
Remove rootfs handling as that was only added for /sbin
Remove write_makefile_packages since it's no longer used

Change-Id: Ie44656d5e30da9637a850df53c4bf3fa1ab0fb2b
2022-02-15 15:48:23 +01:00
Tim Zimmermann
64ed76481c extract_utils: do not cut off extension for cc_binary_prebuilt
* Otherwise extracting HAL services breaks

Change-Id: Ic6302df5c9b512adf4ca81c49cfb1d472aa15c84
2022-01-29 05:29:46 +01:00
Tim Zimmermann
75bcf7f337 extract_utils: disable strip for ELF binaries as well
* Just like we do for libraries
* Without this some binaries might break

Change-Id: Ida82f5ee35f02112eb3a35be66c2f7e1642d56b1
2022-01-29 05:29:46 +01:00
Tim Zimmermann
91767c5d9b extract_utils: allow overriding package name
* Useful for BUILD_BROKEN_ELF_PREBUILT_PRODUCT_COPY_FILES
  if package name conflicts with something source-built but the output location differs

Change-Id: I3af45eae6b2d2cfc583128628edfef9e37ea3976
2022-01-29 05:29:46 +01:00
Alexander Koskovich
33012bff42 extract_utils: Support copying out to /vendor_dlkm.
Change-Id: I600c9d9eb8295e92da2016093462c6d658ca4062
2022-01-22 22:35:58 -07:00
Pablo Cholaky
9c6948b058 extract_utils: Ensure comm uses LC_COLLATE=C
comm behavior will fail if does not receive a properly lexicographic
sort.

Not using appropiated LC_COLLATE leads to error. This is matching
current LC_ALL=C already used at sort commands.

Change-Id: I77f8acb419545a74812d3b36ba608719c2aa89e3
2022-01-16 14:50:31 -05:00
Chirayu Desai
47851781e7 extract_utils: Support A/B OTAs
Change-Id: I52e66fa392f7a8fb5401db179b1242d5d216d137
2021-12-09 01:40:35 +02:00
Chirayu Desai
c1566eb2c3 extract_utils: Avoid requiring sudo
* We can use the available debugfs function, which works without root

Change-Id: Ic24f8a7441fe53af3dd18bafd5f6afce7411999c
2021-12-09 01:40:34 +02:00
Chirayu Desai
48f78ad780 regen-vendor: Support directly extracting from sparse images
Change-Id: I1707b013fe661c45ede57960fa448eccfc4350d5
2021-12-09 01:40:34 +02:00
Michael Bestas
3e6a2fe5eb regen-vendor: Allow proprietary list generation for all partitions
Change-Id: I7fa77cc0406e47b2553ea20f8eb527fd3f7a0296
2021-12-09 01:40:34 +02:00
Chirayu Desai
4af145c186 extract_utils: Support images as a source
* Handles both sparse images as well as filesystem images
* Uses debugfs to extract, no root needed!
* Usecase: Factory images

Change-Id: I3e1bc3834dea108e63ab707a1a89dc2d0743913e
2021-12-09 01:24:49 +02:00
Michael Bestas
376460eac2 extract_utils: Update debugfs error message
Change-Id: I3fb0a3a0c4f52de03009b68036a16cef01ead9c6
2021-12-09 01:24:48 +02:00
Chirayu Desai
a12add2279 extract_utils: Handle prebuilt APEX
Change-Id: I38314fc54d67c3f44374fa945d82954d4c1bd2bc
2021-12-09 01:24:48 +02:00