Commit Graph

109 Commits

Author SHA1 Message Date
Bruno Martins
f12ce8b81d extract_utils: Add a bit of color to the output
Change-Id: I2a779362121645d6d1cee7858a305425190eab56
2021-08-23 11:20:33 +02:00
Bruno Martins
ac8ec32080 extract_utils: Introduce colored echo helper function
Change-Id: I25db42087232c646525f281cc0a1404fe60d1c8c
2021-08-23 11:20:25 +02:00
Sebastiano Barezzi
b49ac3f9a9 extract_utils: Add "check_elf_files: false" also for binaries
Change-Id: I93c1524fac13d8bcb0558277b7f0bc76706457c2
2021-07-14 21:43:15 +02:00
Sebastiano Barezzi
7f0c788e9c extract_utils: Implement versioned patchelf and patchelf 0.8
* Result of patchelf v0.9:
E dlopen failed: "/vendor/lib64/libvendor.goodix.hardware.fingerprint@1.0-service.so" loaded phdr 0x70d4cd6000 not in loadable segment
* While searching up this issue, I found a LineageOS Gerrit change (Ibeb09b6ec37ef62b6a5abe40bae1e8bd8d92e7af) with the same error
* Hamsi2k mentioned in the change that patchelf v0.8 is needed to fix the issue
* Building patchelf v0.8 and trying to patch the blobs works
* Implement versioned patchelf logic, allowing the script to both select the default patchelf and a specific patchelf version only for some files
* Leave 0.9 as the default version to not change the default behaviour

Change-Id: I6c0cbe76baf241233fdd125f5ed1eccbd5855124
2021-02-23 18:12:01 +01:00
Michael Bestas
41cd5a0798 Replace Apache 2.0 boilerplate/reference with SPDX
Change-Id: I691c4893e500aa791868adb82573ef2357805faf
2020-12-29 03:47:01 +02:00
Michael Bestas
8e4a039f11 Remove copyright headers from automatically generated files
It doesn't make sense to place a copyright header in
automatically generated makefiles, plus this is a grey
area since we are extracting proprietary OEM files.

Let's be safe and replace the Apache 2.0 header with
a warning regarding manual modification of the makefiles.

Change-Id: I7cafd3f0def33efff96af60280646c44c1ba9a53
2020-12-28 10:14:46 +00:00
Michael Bestas
ab77884b8e Remove branding from script variables
No need to force branding down everyone's throat,
most of the other android forks using these scripts are
rebranding the variables anyway.
Let's reduce fragmentation.

Change-Id: Ie9c223d1a42728076f45b40a11cd0820010db8b1
2020-12-28 10:14:46 +00:00
Michael Bestas
5a82907bb9 Adapt for new tools location
Change-Id: I2eea1b75e5cc1c5afc0311a8818c54317f750d3f
2020-12-28 10:14:46 +00:00
Michael Bestas
8f82a58987 Adapt for new script location
Change-Id: I8758d7b167b96bd04308bf3d3fcb059feb7fdcc2
2020-12-28 10:14:46 +00:00
Sebastiano Barezzi
490e38efd7 fixup! extract_utils: implement patchelf
* Move $PATCHELF exporting from oat2dex to setup_vendor
* Since it requires $HOST to be set, let's make it global, so oat2dex can also use it

Change-Id: I4556a3c19cd01c9b3a68d358d19a361217d9c3c1
2020-12-27 22:02:20 +02:00
Alessandro Astone
e14de31ff5 extract_utils: generate_prop_list: Ignore vdex/odex files too
Change-Id: Ibe77e8b93466a71d4d0c48ca8f4ff2e00c6e40af
(cherry picked from commit ee669788bc86330215f2444b3beb1f3870329c9e)
2020-12-15 19:22:12 +01:00
Volodymyr Zhdanov
c2a40faabf extract_utils: implement patchelf
Change-Id: I406f70ef1ce9ec7a0998f77439b7d3fa3ec4e62a
2020-11-26 20:40:02 +01:00
LuK1337
3f39ee5eed extract_utils: Set filename_from_src: true for CLASS=ETC targets
Change-Id: I881f6a67d84d8de0a4bef17bf3c9eac049679c38
2020-10-06 19:29:53 +02:00
dianlujitao
27346e2a4a extract_utils: Disable ELF check for shared library
* This is indeed a nice feature, but it's absolutely a bad idea to
   hardcode dependencies of prebuilt modules in proprietary-files.txt.

Change-Id: I8c2d75ff62c0c7862f40e777bcbad4d9cebc074c
2020-09-20 14:56:47 +01:00
Alexander Koskovich
abe640021e extract_utils: Fix system_ext framework blueprint generation.
Change-Id: I20e44be6ea48b9bd5b05055d3cf465f63c908463
2020-09-17 10:00:17 +02:00
Luca Stefani
6c1eb09602 extract_utils: Add support for system_ext
Change-Id: I0760cdd9bec1a0a0551aa68a60c34f9c00932668
2020-09-09 20:17:37 +02:00
Andreas Schneider
bfd61d3c96 extract-utils: Fix usage for ETC files which need sub_dir
Thanks to Tad.

Change-Id: I6a0e7978afce6444d0827f97d7686f23aa7500c9
2020-05-25 23:52:52 +02:00
Andreas Schneider
4ae6048e3f extract_utils: Fix vendor Android.bp creation with subdirectory
error: vendor/samsung/beyond0lte/Android.bp:37:9: unrecognized property "sub_dir"

Change-Id: I23becda0345f41d2cb1462e74b66a787e9ad4c96
2020-05-23 16:00:38 +02:00
dianlujitao
8426f8236d extract_utils: Handle Brotli compressed block image
Change-Id: I1b3ef34ebba9b43fd684c6abaa1dbc77c8786da7
2020-04-25 16:21:32 +02:00
dianlujitao
1dc0058553 extract_utils: Support odm/product/vendor partition for zip extract
Change-Id: Iec06835f1703e276f8882c5ee327ef241a2f8f23
2020-04-25 16:21:32 +02:00
dianlujitao
c6992a51da extract_utils: Generate deodexed apk/jar with consistent checksum
* zip stores timestamp for all included files. The timestamps of dex
   files are different across different runs, result in inconsistent
   checksum of output apk/jar.
 * Workaround the issue by using fixed timestamp for dex files.

Change-Id: I21f3a7e32cdfdb07c5f5c140df2e797efd4a8005
2020-04-06 13:10:36 +08:00
dianlujitao
129e14bc53 extract_utils: Fix doubled path separator
* $DST_FILE is already prefixed with "/", don't double it.

Change-Id: I2655313abd7720a54b20f541a95960ae9ec373ad
2020-04-06 12:43:17 +08:00
Han Wang
05468e5c36 extract_utils: Fix up b64d23778f
Change-Id: I9e5ea799812ca932de95ec6cbdedb604cbde5736
2020-03-10 09:41:34 +02:00
XiNGRZ
b64d23778f extract_utils: Fix bad substitution for zsh
Change-Id: I5f4fabc62b1ac3da159b36c3ae6080e145948530
2020-03-10 07:49:55 +01:00
dianlujitao
dd0cb42de7 extract_utils: Drop string after semicolon when parsing destination
This fixes parsing when arguments contain colons, a typical usecase
would be:
-vendor/app/TimeService/TimeService.apk;:timeservice_app_cert

Change-Id: I7500ae09632632ddc10734d9b1df267e28286b67
2020-01-10 15:19:47 +01:00
Mohd Faraz
39021282ba extract_utils: Added Suffix uses COMMON_SUFFIX
* COMMON_JAVA_PACKAGE_SUFFIX for jar

 * COMMON_ANDROID_PACKAGE_SUFFIX for apk

Change-Id: I812405dac12ef7183985c66a6e43b0ea5f85989c
Signed-off-by: Mohd Faraz <mohd.faraz.abc@gmail.com>
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
8c5d006a68 extract_files: Implement blueprint file generation
Switch to blueprint on:
 - shared objects
 - $partiton/etc/ files
 - JARs
 - executable binaries and scripts
 - APKs

Only /sbin binaries are still in Android.mk because blueprint
doesn't handle sbin installation yet

Change-Id: I1dfd7e8bb575367b2a7fa9e333c4c6fa3aa68180
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
b9b42e18a4 extract_files: Add initial blueprint creation
Start the copyrights for these at 2019 because the blueprints won't
have existed before then.

Change-Id: I5300109949785dd19555e35d17a1e80daef498b0
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
31ba2a3dbb extract_files: Match non-treble path prefixes
Some devices put stuff on /system, /system/vendor or even
/system/vendor/odm. Search for these paths too when generating
TARGET_COPY_OUT_$partition variables.

Change-Id: Ie2c087e57aaca02d5ea93f290d5fc50d1315a600
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
9ff5c50021 extract_files: Support system/ prefixes
With support for 4 independent partitions now, we seriously
need to start putting /system blobs in their own directory.
Add support for file lists with system/ prefixes while
maintaining support for old file lists without it.

Also, TARGET_COPY_OUT_SYSTEM is a thing now, and all devices,
regardless of treble or not, set TARGET_COPY_OUT_$partition
so let's get rid of the treble compat option and default it
to true.

Change-Id: I5b798d293768d7c1e16db3ba01e2de3e083088d7
2019-10-27 11:24:06 +00:00
PIPIPIG233666
0f41bf4e83 extract_utils: Fix typo
* ./../../xiaomi/sm6150-common/../../../vendor/lineage/build/tools/extract_utils.sh: line 1: /#!/bin/bash: No such file or directory.

Signed-off-by: PIPIPIG233666 <2212848813@qq.com>
Change-Id: I178f745d4ecb818c38706ff100611df19221065d
2019-09-29 12:49:44 -04:00
Bruno Martins
bb05ee5be5 extract_utils: Source the prebuilts from the new repo
The prebuilt tools now live in prebuilts/tools-lineage.

Change-Id: I4022210c52e1002aae0c6e622a480e5c7e10c366
2019-09-29 03:06:49 +02:00
Rashed Abdel-Tawab
f085bac858 extract_files: Add ODM partition blob support
Change-Id: Id94bbef879b6f4bcc20645c259981f9d163649c9
2019-09-27 06:46:44 -07:00
razorloves
c4be3d8541 extract_utils: Support extracting from product partition
Change-Id: I2d0d01253fdf93d63a26556208d45cd69e76329f
(cherry picked from commit ea15be6e92e3be0c103948d220de64fd2e1d781f)
2019-09-10 11:28:05 +02:00
Rashed Abdel-Tawab
3bab983abe extract_utils: Support multidex
Co-authored-by: Erfan Abdi <erfangplus@gmail.com>
Change-Id: I4881658d303b6c5c7f0b141beb50c1e9100af611
2019-07-18 07:31:36 +02:00
Rashed Abdel-Tawab
df5b192087 extract_utils: Add functions to extract vendor blobs from vendor.img
Change-Id: I6f37adc955856fa78e719eb30249800abe80020c
2019-06-26 04:37:44 +02:00
Vladimir Oltean
16138be007 extract_utils: introduce support for executing blob fixups
* Traditionally, the task of hex-editing blobs has been approached in 2 ways:
  (1) Do it out-of-band, commit the modified blob, and record its edited
      sha1sum in proprietary-files.txt (aka pin it).
  (2) Do it in-band, by adding code to the device-level extract-files.sh
      (usually this performs patchelf or sed). This code runs after the
      extract_utils functions were invoked.
* Problems of approach (1):
    - It relies on verbal (basically commit message) documentation of
      the hex-editing that was done. Makes it more difficult to reproduce.
    - Each time blobs are updated, pinning needs to be temporarily removed,
      hex-editing done again manually and new hash put back.
* Problems of approach (2):
    - It is incompatible with the concept of pinning, which is useful
      for kanging blobs from another device. A pinned blob would either:
        - Match the hash, get hex-edited, then it won't match the hash
          next time around.
        - Not match the hash (because of, say, hex-editing), then the
          extraction script would use an unwanted blob version instead of the
          pinned one (either that, or say "!! file not found in source").

* In summary, this patch adds system-wide support for approach (2) in order
  to address the aforementioned shortcomings.

* At device level, users of extract_utils who wish to perform blob
  fixups can override a blob_fixup() Bash function in their
  extract-files.sh immediately after running "source ${HELPER}". The
  blob_fixup() function will be called by the common extract() function
  after extracting every individual blob, giving the user the
  opportunity to hook custom code after this operation takes place.

* In proprietary-files.txt, the line corresponding to this blob which
  needs fixups can look in one of 2 ways:

(a) vendor/lib64/vendor.qti.gnss@1.0_vendor.so

    Do this if you are taking the blob from the stock ROM. The fixup
    script will always run after the blob is extracted.

(b) vendor/lib64/vendor.qti.gnss@1.0_vendor.so|249c76153f8de014bf2dd2ab623ee3d87741fbc8|f7e9ee8e3804887a2f3939128e860767e6f27258

    Do this if you are kanging the blob from somebody else. The pinning
    logic now applies for both the pre- and the post-fixup hashes. The
    fixup script will only run if the blob doesn't match the hex-edited blob,
    although the fixup script should really be idempotent.

Change-Id: Ifdd73c885d995c645f6210597537693d1a2f903f
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:44:53 +02:00
Vladimir Oltean
c463ac8e00 extract_utils: make --section match words anywhere in the line
Change-Id: Ia3fe2160f0f4bc7da1b5bd31a21080af51b41fab
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
f88e6d3034 extract_utils: introduce -k (kang mode) option
* The use case is easier updating of pinned blobs. When --kang is set,
  pinning is automatically ignored, and the script will print lines at
  its output that can be directly copied back into the
  proprietary-files.txt.
* Best served together with --section ${SECTION}, and proper grouping
  of the proprietary-files.txt.

Change-Id: I648fbcbd4580a4a002b00828bcfee18d1e265d7b
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
e3f910b114 extract_utils: cleanup variable names and arguments passed to extract() function
* This also makes the --section argument non-positional, since otherwise
  it is not possible to easily support more than one optional positional
  argument. This is in preparation of one more optional argument to come
  in a follow-up patch: --kang.

Change-Id: Ieb142e0854319defb9a278ab68cd4aeefd0fbdd5
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
ee437e3264 extract_utils: be compatible with system-as-root layouts
* The use case is that if you have the following layout:
  $TOP --- system.img
       |
       +-- vendor.img
  you should be able (from $TOP) to:
    mkdir system; mount -o ro,loop system.img system
    mkdir vendor; mount -o ro,loop vendor.img vendor
  and then (from device tree)
    ./extract-files.sh $TOP

  But this doesn't work if system.img is SAR and contains another
  "system" dir inside. This patch makes sure it searches for a "system"
  dir in the provided path as well, if it couldn't find the blob
  anywhere else.

Change-Id: Ib49cd5b587b3a57478a66ff69cf840270c2b1403
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
eac2f563fd extract_utils: drop the "/system/" prefix from prints of blob names
* This makes the printed output closer to the proprietary-files.txt syntax

Change-Id: I81b844bb6bb1d1a2f91a39151a892fbfc0bed20b
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
3a02fc8144 extract_utils: fix mismatch with non-lowercase pinned sha1sum
Change-Id: I86421ca69007ad1992aa67deedf7d2e32bbe4dba
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-01-17 11:26:34 +01:00
Vladimir Oltean
864160111b extract_utils: avoid doing stuff on files that don't exist
* If an apk/jar doesn't exist, the script would still try to deodex it.
* If an xml doesn't exist, the script would still try to "fix" it.
* Take it easier, man, it's not your fault.

Change-Id: I3061fb48b403da5121e3c17dd9ecdb6cd148bf97
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-12-24 01:32:53 +02:00
Luca Stefani
d70ffb6ffc extract_utils: Redirect vdexextractor log to /dev/null
Change-Id: I55028997c0396d57c024e6c42a53907f6d1510c8
2018-11-13 22:06:17 +01:00
Luca Stefani
4682cc364a Add CDEX support for boot JAR files
Change-Id: I0039256c883a58c5a5125fd9bff6e6ec3b98688d
2018-10-21 10:28:37 +02:00
TheStrix
7a64931732 extract_utils: Fix output dex file name generated by vdexExtractor
- Reflect the changes required by:
   ff58255706
 - Also fix JARVDEX file path for deodexing jars

Change-Id: Ief355b0bd9e69a67b552d0bd2060f43adb39fb5f
2018-10-07 12:17:35 +02:00
codeworkx
02233784a8 extract_utils: add CDEX support via compact_dex_converter
Source: https://github.com/anestisb/vdexExtractor#compact-dex-converter

Change-Id: I63664cae02e51a05e5a62c0c5c0595eead6c6513
2018-10-05 18:25:46 +02:00
Luca Stefani
8e388bcddc extract_utils: Newline love for adb extraction
Change-Id: Ibbf7a03ed6920d556ed7658c277f37b02d8876b8
2018-07-12 21:28:21 +02:00
Vladimir Oltean
91110b6fbb extract_utils: make get_file() able to search paths with and w/o /system prefix
Change-Id: I75eba627646caa8ceb8a30575124c57e5ee9445c
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00