Commit Graph

120 Commits

Author SHA1 Message Date
Michael Bestas
376460eac2 extract_utils: Update debugfs error message
Change-Id: I3fb0a3a0c4f52de03009b68036a16cef01ead9c6
2021-12-09 01:24:48 +02:00
Chirayu Desai
a12add2279 extract_utils: Handle prebuilt APEX
Change-Id: I38314fc54d67c3f44374fa945d82954d4c1bd2bc
2021-12-09 01:24:48 +02:00
Michael Bestas
d4c47bf0b8 extract_utils: Automatically add apk/jar/vintf fragments to PRODUCT_PACKAGES
Change-Id: I9d12e00c294d02b40fde2b66d7797f69f6504c35
2021-12-09 01:22:10 +02:00
Michael Bestas
acaaa7b60e Remove default skipped files from generate_prop_list_from_image()
* This list hasn't been updated in a long time and doesn't contain
  all the default vendor modules. Simply drop it and regenerate the
  full list device side automatically with a script.

Change-Id: I8cf99d1b726f390f9a3bc2903e69eb26da7ebea9
2021-11-15 20:13:55 +02:00
Michael Bestas
484e2db8c1 Use LC_ALL=C sorting in generate_prop_list_from_image()
* Generate consistent output

Change-Id: Ibab238d15a64fbad83fcfad19cb7c6f266d6ecb1
2021-11-15 19:28:56 +02:00
Arian
d3e0c026bc extract_utils: Seperate multiple override targets with comma
This follows the style google uses, for example in [1].

With this change it will generate 'overrides: ["TargetOne", "TargetTwo"],'
instead of 'overrides: ["TargetOne TargetTwo"],' for this line:
-system/app/TargetZero/TargetZero.apk;OVERRIDES=TargetOne,TargetTwo

[1]: 2daf3ecf89/Android.bp (L53)

Change-Id: I154ee3a4cc51e0938c860dcfc125eacf279d102e
2021-09-27 22:05:37 +02:00
Arian
8b335032f4 extract_utils: Don't fixup pinned files when they are supposed to be kept
Change-Id: I6f38bf5928061b0f09eda04f85e00b11912d0227
2021-09-09 15:40:02 +02:00
Arian
302226bb77 extract_utils: Display the fixup hash when keeping pinned files
Change-Id: I32e288b1b06f103179d894d0a68a2ce34df31913
2021-09-09 15:18:35 +02:00
LuK1337
72d5bdfb73 fixup! extract_utils: Add support for overriding packages
This unifies arguments parsing + fixes case where OVERRIDES=pkg would
generate .bp without `certificate: "platform"`.

Change-Id: I0036e0e942ff9272da10f14aeb7235f0b6b92b00
2021-08-23 18:21:58 +02:00
TheStrix
3749ce5b29 extract_utils: Add support for overriding packages
* An example of how this can be used is say a device
   wants to use the original OEM camera, e.g. Essential PH-1.
   Klik is required in order to use the external camera
   module that you could buy for the device so it's preferred
   to use it over Camera2 or Snap.

  * An example of how to use this argument:
    -system/priv-app/Klik/Klik.apk;OVERRIDES=Camera2

Change-Id: I2504cbb760b20ed86c2dc088cc4bd74b78ba64d9
2021-08-23 17:58:01 +02:00
Bruno Martins
eed0c8ca3b extract_utils: Skip colored echo when piped
Change-Id: I8a03a36f3fa022e6858cc5c1dbca8b8037efed03
2021-08-23 11:20:39 +02:00
Bruno Martins
f12ce8b81d extract_utils: Add a bit of color to the output
Change-Id: I2a779362121645d6d1cee7858a305425190eab56
2021-08-23 11:20:33 +02:00
Bruno Martins
ac8ec32080 extract_utils: Introduce colored echo helper function
Change-Id: I25db42087232c646525f281cc0a1404fe60d1c8c
2021-08-23 11:20:25 +02:00
Sebastiano Barezzi
b49ac3f9a9 extract_utils: Add "check_elf_files: false" also for binaries
Change-Id: I93c1524fac13d8bcb0558277b7f0bc76706457c2
2021-07-14 21:43:15 +02:00
Sebastiano Barezzi
7f0c788e9c extract_utils: Implement versioned patchelf and patchelf 0.8
* Result of patchelf v0.9:
E dlopen failed: "/vendor/lib64/libvendor.goodix.hardware.fingerprint@1.0-service.so" loaded phdr 0x70d4cd6000 not in loadable segment
* While searching up this issue, I found a LineageOS Gerrit change (Ibeb09b6ec37ef62b6a5abe40bae1e8bd8d92e7af) with the same error
* Hamsi2k mentioned in the change that patchelf v0.8 is needed to fix the issue
* Building patchelf v0.8 and trying to patch the blobs works
* Implement versioned patchelf logic, allowing the script to both select the default patchelf and a specific patchelf version only for some files
* Leave 0.9 as the default version to not change the default behaviour

Change-Id: I6c0cbe76baf241233fdd125f5ed1eccbd5855124
2021-02-23 18:12:01 +01:00
Michael Bestas
41cd5a0798 Replace Apache 2.0 boilerplate/reference with SPDX
Change-Id: I691c4893e500aa791868adb82573ef2357805faf
2020-12-29 03:47:01 +02:00
Michael Bestas
8e4a039f11 Remove copyright headers from automatically generated files
It doesn't make sense to place a copyright header in
automatically generated makefiles, plus this is a grey
area since we are extracting proprietary OEM files.

Let's be safe and replace the Apache 2.0 header with
a warning regarding manual modification of the makefiles.

Change-Id: I7cafd3f0def33efff96af60280646c44c1ba9a53
2020-12-28 10:14:46 +00:00
Michael Bestas
ab77884b8e Remove branding from script variables
No need to force branding down everyone's throat,
most of the other android forks using these scripts are
rebranding the variables anyway.
Let's reduce fragmentation.

Change-Id: Ie9c223d1a42728076f45b40a11cd0820010db8b1
2020-12-28 10:14:46 +00:00
Michael Bestas
5a82907bb9 Adapt for new tools location
Change-Id: I2eea1b75e5cc1c5afc0311a8818c54317f750d3f
2020-12-28 10:14:46 +00:00
Michael Bestas
8f82a58987 Adapt for new script location
Change-Id: I8758d7b167b96bd04308bf3d3fcb059feb7fdcc2
2020-12-28 10:14:46 +00:00
Sebastiano Barezzi
490e38efd7 fixup! extract_utils: implement patchelf
* Move $PATCHELF exporting from oat2dex to setup_vendor
* Since it requires $HOST to be set, let's make it global, so oat2dex can also use it

Change-Id: I4556a3c19cd01c9b3a68d358d19a361217d9c3c1
2020-12-27 22:02:20 +02:00
Alessandro Astone
e14de31ff5 extract_utils: generate_prop_list: Ignore vdex/odex files too
Change-Id: Ibe77e8b93466a71d4d0c48ca8f4ff2e00c6e40af
(cherry picked from commit ee669788bc86330215f2444b3beb1f3870329c9e)
2020-12-15 19:22:12 +01:00
Volodymyr Zhdanov
c2a40faabf extract_utils: implement patchelf
Change-Id: I406f70ef1ce9ec7a0998f77439b7d3fa3ec4e62a
2020-11-26 20:40:02 +01:00
LuK1337
3f39ee5eed extract_utils: Set filename_from_src: true for CLASS=ETC targets
Change-Id: I881f6a67d84d8de0a4bef17bf3c9eac049679c38
2020-10-06 19:29:53 +02:00
dianlujitao
27346e2a4a extract_utils: Disable ELF check for shared library
* This is indeed a nice feature, but it's absolutely a bad idea to
   hardcode dependencies of prebuilt modules in proprietary-files.txt.

Change-Id: I8c2d75ff62c0c7862f40e777bcbad4d9cebc074c
2020-09-20 14:56:47 +01:00
Alexander Koskovich
abe640021e extract_utils: Fix system_ext framework blueprint generation.
Change-Id: I20e44be6ea48b9bd5b05055d3cf465f63c908463
2020-09-17 10:00:17 +02:00
Luca Stefani
6c1eb09602 extract_utils: Add support for system_ext
Change-Id: I0760cdd9bec1a0a0551aa68a60c34f9c00932668
2020-09-09 20:17:37 +02:00
Andreas Schneider
bfd61d3c96 extract-utils: Fix usage for ETC files which need sub_dir
Thanks to Tad.

Change-Id: I6a0e7978afce6444d0827f97d7686f23aa7500c9
2020-05-25 23:52:52 +02:00
Andreas Schneider
4ae6048e3f extract_utils: Fix vendor Android.bp creation with subdirectory
error: vendor/samsung/beyond0lte/Android.bp:37:9: unrecognized property "sub_dir"

Change-Id: I23becda0345f41d2cb1462e74b66a787e9ad4c96
2020-05-23 16:00:38 +02:00
dianlujitao
8426f8236d extract_utils: Handle Brotli compressed block image
Change-Id: I1b3ef34ebba9b43fd684c6abaa1dbc77c8786da7
2020-04-25 16:21:32 +02:00
dianlujitao
1dc0058553 extract_utils: Support odm/product/vendor partition for zip extract
Change-Id: Iec06835f1703e276f8882c5ee327ef241a2f8f23
2020-04-25 16:21:32 +02:00
dianlujitao
c6992a51da extract_utils: Generate deodexed apk/jar with consistent checksum
* zip stores timestamp for all included files. The timestamps of dex
   files are different across different runs, result in inconsistent
   checksum of output apk/jar.
 * Workaround the issue by using fixed timestamp for dex files.

Change-Id: I21f3a7e32cdfdb07c5f5c140df2e797efd4a8005
2020-04-06 13:10:36 +08:00
dianlujitao
129e14bc53 extract_utils: Fix doubled path separator
* $DST_FILE is already prefixed with "/", don't double it.

Change-Id: I2655313abd7720a54b20f541a95960ae9ec373ad
2020-04-06 12:43:17 +08:00
Han Wang
05468e5c36 extract_utils: Fix up b64d23778f
Change-Id: I9e5ea799812ca932de95ec6cbdedb604cbde5736
2020-03-10 09:41:34 +02:00
XiNGRZ
b64d23778f extract_utils: Fix bad substitution for zsh
Change-Id: I5f4fabc62b1ac3da159b36c3ae6080e145948530
2020-03-10 07:49:55 +01:00
dianlujitao
dd0cb42de7 extract_utils: Drop string after semicolon when parsing destination
This fixes parsing when arguments contain colons, a typical usecase
would be:
-vendor/app/TimeService/TimeService.apk;:timeservice_app_cert

Change-Id: I7500ae09632632ddc10734d9b1df267e28286b67
2020-01-10 15:19:47 +01:00
Mohd Faraz
39021282ba extract_utils: Added Suffix uses COMMON_SUFFIX
* COMMON_JAVA_PACKAGE_SUFFIX for jar

 * COMMON_ANDROID_PACKAGE_SUFFIX for apk

Change-Id: I812405dac12ef7183985c66a6e43b0ea5f85989c
Signed-off-by: Mohd Faraz <mohd.faraz.abc@gmail.com>
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
8c5d006a68 extract_files: Implement blueprint file generation
Switch to blueprint on:
 - shared objects
 - $partiton/etc/ files
 - JARs
 - executable binaries and scripts
 - APKs

Only /sbin binaries are still in Android.mk because blueprint
doesn't handle sbin installation yet

Change-Id: I1dfd7e8bb575367b2a7fa9e333c4c6fa3aa68180
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
b9b42e18a4 extract_files: Add initial blueprint creation
Start the copyrights for these at 2019 because the blueprints won't
have existed before then.

Change-Id: I5300109949785dd19555e35d17a1e80daef498b0
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
31ba2a3dbb extract_files: Match non-treble path prefixes
Some devices put stuff on /system, /system/vendor or even
/system/vendor/odm. Search for these paths too when generating
TARGET_COPY_OUT_$partition variables.

Change-Id: Ie2c087e57aaca02d5ea93f290d5fc50d1315a600
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
9ff5c50021 extract_files: Support system/ prefixes
With support for 4 independent partitions now, we seriously
need to start putting /system blobs in their own directory.
Add support for file lists with system/ prefixes while
maintaining support for old file lists without it.

Also, TARGET_COPY_OUT_SYSTEM is a thing now, and all devices,
regardless of treble or not, set TARGET_COPY_OUT_$partition
so let's get rid of the treble compat option and default it
to true.

Change-Id: I5b798d293768d7c1e16db3ba01e2de3e083088d7
2019-10-27 11:24:06 +00:00
PIPIPIG233666
0f41bf4e83 extract_utils: Fix typo
* ./../../xiaomi/sm6150-common/../../../vendor/lineage/build/tools/extract_utils.sh: line 1: /#!/bin/bash: No such file or directory.

Signed-off-by: PIPIPIG233666 <2212848813@qq.com>
Change-Id: I178f745d4ecb818c38706ff100611df19221065d
2019-09-29 12:49:44 -04:00
Bruno Martins
bb05ee5be5 extract_utils: Source the prebuilts from the new repo
The prebuilt tools now live in prebuilts/tools-lineage.

Change-Id: I4022210c52e1002aae0c6e622a480e5c7e10c366
2019-09-29 03:06:49 +02:00
Rashed Abdel-Tawab
f085bac858 extract_files: Add ODM partition blob support
Change-Id: Id94bbef879b6f4bcc20645c259981f9d163649c9
2019-09-27 06:46:44 -07:00
razorloves
c4be3d8541 extract_utils: Support extracting from product partition
Change-Id: I2d0d01253fdf93d63a26556208d45cd69e76329f
(cherry picked from commit ea15be6e92e3be0c103948d220de64fd2e1d781f)
2019-09-10 11:28:05 +02:00
Rashed Abdel-Tawab
3bab983abe extract_utils: Support multidex
Co-authored-by: Erfan Abdi <erfangplus@gmail.com>
Change-Id: I4881658d303b6c5c7f0b141beb50c1e9100af611
2019-07-18 07:31:36 +02:00
Rashed Abdel-Tawab
df5b192087 extract_utils: Add functions to extract vendor blobs from vendor.img
Change-Id: I6f37adc955856fa78e719eb30249800abe80020c
2019-06-26 04:37:44 +02:00
Vladimir Oltean
16138be007 extract_utils: introduce support for executing blob fixups
* Traditionally, the task of hex-editing blobs has been approached in 2 ways:
  (1) Do it out-of-band, commit the modified blob, and record its edited
      sha1sum in proprietary-files.txt (aka pin it).
  (2) Do it in-band, by adding code to the device-level extract-files.sh
      (usually this performs patchelf or sed). This code runs after the
      extract_utils functions were invoked.
* Problems of approach (1):
    - It relies on verbal (basically commit message) documentation of
      the hex-editing that was done. Makes it more difficult to reproduce.
    - Each time blobs are updated, pinning needs to be temporarily removed,
      hex-editing done again manually and new hash put back.
* Problems of approach (2):
    - It is incompatible with the concept of pinning, which is useful
      for kanging blobs from another device. A pinned blob would either:
        - Match the hash, get hex-edited, then it won't match the hash
          next time around.
        - Not match the hash (because of, say, hex-editing), then the
          extraction script would use an unwanted blob version instead of the
          pinned one (either that, or say "!! file not found in source").

* In summary, this patch adds system-wide support for approach (2) in order
  to address the aforementioned shortcomings.

* At device level, users of extract_utils who wish to perform blob
  fixups can override a blob_fixup() Bash function in their
  extract-files.sh immediately after running "source ${HELPER}". The
  blob_fixup() function will be called by the common extract() function
  after extracting every individual blob, giving the user the
  opportunity to hook custom code after this operation takes place.

* In proprietary-files.txt, the line corresponding to this blob which
  needs fixups can look in one of 2 ways:

(a) vendor/lib64/vendor.qti.gnss@1.0_vendor.so

    Do this if you are taking the blob from the stock ROM. The fixup
    script will always run after the blob is extracted.

(b) vendor/lib64/vendor.qti.gnss@1.0_vendor.so|249c76153f8de014bf2dd2ab623ee3d87741fbc8|f7e9ee8e3804887a2f3939128e860767e6f27258

    Do this if you are kanging the blob from somebody else. The pinning
    logic now applies for both the pre- and the post-fixup hashes. The
    fixup script will only run if the blob doesn't match the hex-edited blob,
    although the fixup script should really be idempotent.

Change-Id: Ifdd73c885d995c645f6210597537693d1a2f903f
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:44:53 +02:00
Vladimir Oltean
c463ac8e00 extract_utils: make --section match words anywhere in the line
Change-Id: Ia3fe2160f0f4bc7da1b5bd31a21080af51b41fab
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
f88e6d3034 extract_utils: introduce -k (kang mode) option
* The use case is easier updating of pinned blobs. When --kang is set,
  pinning is automatically ignored, and the script will print lines at
  its output that can be directly copied back into the
  proprietary-files.txt.
* Best served together with --section ${SECTION}, and proper grouping
  of the proprietary-files.txt.

Change-Id: I648fbcbd4580a4a002b00828bcfee18d1e265d7b
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00