Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tools: xxd: replace with xxdi.pl script #10566

Closed
wants to merge 8 commits into from

Conversation

ynezz
Copy link
Member

@ynezz ynezz commented Aug 31, 2022

Several users (#10555 for example) are currently reporting build issues which seems to be related to xxd package download issues. This utility seems to be mostly needed by uboot-mediatek (default environment) and uboot-bcm4908 (HTML artifacts) packages, where it's being used in xxd -i mode which allows outputing of such C include file style artifacts.

Dependency on xxd was added in commit c4dd244 ("tools: add xxd (from vim)") as U-Boot requires xxd to create the default environment from an external file.

Later in commit 2b94aac ("tools: xxd: use more convenient source tarball"), xxd from another source was used instead, but that source is currently not working properly either.

So this PR aims to replace dependency on xxd package with self-contained xxdi.pl Perl script, drop in replacement for xxd -i mode only, mode which outputs C include file style, with aim for byte to byte xxd identical output, so the eventual difference in the generated output by this xxdi.pl is easily spottable.

FYI @dangowrt @rmilecki

ynezz and others added 7 commits August 31, 2022 07:57
xxdi.pl is a Perl script that implements vim's 'xxd -i' mode so that
packages do not have to use all of vim just to get this functionality.

References: openwrt#10555
Signed-off-by: Petr Štetiar <ynezz@true.cz>
In order to make it more portable.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
So it can serve as a standalone drop in replacement for xxd utility used
currently mostly in U-Boot packages with `xxd -i` mode which outputs C
include file style, with aim for byte to byte identical output, so the
eventual difference in the generated output is easily spottable.

References: openwrt#10555
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Dependency on xxd was added in commit c4dd244 ("tools: add xxd
(from vim)") as U-Boot requires xxd to create the default environment
from an external file.

Later in commit 2b94aac ("tools: xxd: use more convenient source
tarball"), xxd from another source was used instead, but that source is
currently unavailable, so let's fix it by using simple xxdi.pl Perl
script instead.

Fixes: openwrt#10555
Signed-off-by: Petr Štetiar <ynezz@true.cz>
It shouldn't be needed anymore as we've now `scripts/xxdi.pl`, which
should be self contained and fully compatible `xxd -i` replacement.

Fixes: openwrt#10555
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Instead of waiting for complete workflow finish, then downloading the
artifacts, unpack them and inspect them, lets try to make the build
failure immediately visible in the log output:

  ====== Showing Make errors found in the log files ======
  ====== Make errors from logs/target/linux/compile.txt ======
  * Legacy (non-UHI/non-FIT) Boards
  *
  Support MIPS SEAD-3 boards (LEGACY_BOARD_SEAD3) [N/y/?] (NEW)
  Error in reading or end of file.

  make[6]: *** [scripts/kconfig/Makefile:77: syncconfig] Error 1
  make[5]: *** [Makefile:616: syncconfig] Error 2
  make[4]: *** [Makefile:736: include/config/auto.conf.cmd] Error 2
  make[3]: *** [Makefile:24: /__w/openwrt/openwrt/openwrt/build_dir/target-mipsel-openwrt-linux-musl_musl/linux-ramips_mt7620/linux-5.15.62/.modules] Error 2
  make[2]: *** [Makefile:11: compile] Error 2
  time: target/linux/compile#30.09#11.30#37.92

Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit 622d342f1abd7987081fb4bb429f32a61cc48377)
Signed-off-by: Petr Štetiar <ynezz@true.cz>
@ynezz ynezz added build/scripts/tools pull request/issues for build, scripts and tools related changes core packages pull request/issue for core (in-tree) packages target/mediatek pull request/issue for mediatek target target/bcm4908 pull request/issue for bcm4908 target labels Aug 31, 2022
@ynezz ynezz self-assigned this Aug 31, 2022
Applied jo's perl-fu.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
@graysky2
Copy link
Contributor

graysky2 commented Aug 31, 2022

@ynezz - as we wait for the PR to be evaluated do you know of a source mirror for the existing source tarball? I am coming up empty googling and recently cleaned out my toolchain including ./dl/ so I cannot build without it.

EDIT: http://openwrt.missorforget.pub/lean/lede/dl/xxd-1.10.tar.gz

@mpratt14
Copy link
Contributor

@graysky2 also

https://sources.cdn.openwrt.org/xxd-1.10.tar.gz

@dangowrt
Copy link
Member

dangowrt commented Sep 6, 2022

I've tested the resulting U-Boot image on Linksys E8450 (mediatek/mt7622), it works fine and the embedded environment still looks like it should. I conclude this works fine and can be merged (obviously after folding up the patches)

@ynezz
Copy link
Member Author

ynezz commented Sep 6, 2022

Merged in 88c9056.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build/scripts/tools pull request/issues for build, scripts and tools related changes core packages pull request/issue for core (in-tree) packages target/bcm4908 pull request/issue for bcm4908 target target/mediatek pull request/issue for mediatek target
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants