-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
expat-2.2.9.tar.gz Not Found #13
Comments
I had the same issue trying to build the ESP Toolchain from scratch on my RPi 4B+ Ubuntu 20.10 aarch64 dev box... On initial attempt to configure and build with: ./ct-ng xtensa-esp32-elf #// Configures AOK
./ct-ng build #// Build Fails I received the following error: cbishop@cbishop-ubuntu:~/esp/crosstool-NG$ ./ct-ng build
[INFO ] Performing some trivial sanity checks
[INFO ] Build started 20210702.233915
[INFO ] Building environment variables
[INFO ] =================================================================
[INFO ] Retrieving needed toolchain components' tarballs
[ERROR]
[ERROR] >>
[ERROR] >> Build failed in step 'Retrieving needed toolchain components' tarballs'
[ERROR] >> called in step '(top-level)'
[ERROR] >>
[ERROR] >> Error happened in: do_expat_get[scripts/build/companion_libs/210-expat.sh@12]
[ERROR] >> called from: do_companion_libs_get[scripts/build/companion_libs.sh@15]
[ERROR] >> called from: main[scripts/crosstool-NG.sh@591]
[ERROR] >>
[ERROR] >> For more info on this error, look at the file: 'build.log'
[ERROR] >> There is a list of known issues, some with workarounds, in:
[ERROR] >> 'docs/B - Known issues.txt'
[ERROR]
[ERROR] (elapsed: 0:21.56)
[00:22] / gmake: *** [ct-ng:152: build] Error 1
cbishop@cbishop-ubuntu:~/esp/crosstool-NG$ After reviewing my resulting build.log file - I determined that the issue seems to root from a hard coded URL in the following build script's do_expat_get() method: Note the hard coded sourceforge URL which when I paste in to my browser, results in a URL rewrite to new sourceforge URL structure: do_expat_get() {
CT_GetFile "expat-${CT_EXPAT_VERSION}" .tar.gz \
http://downloads.sourceforge.net/project/expat/expat/${CT_EXPAT_VERSION}
} Note that there are two issues here - first, when attempt to nav to what would be the resulting URL as-is in browser: http://downloads.sourceforge.net/project/expat/expat/2.1.0
https://sourceforge.net/projects/expat/files/expat/2.1.0/ second issue, note that version 2.1.0 has been deprecated due to vulnerabilities, and 2.3.0 is the minimum recommended version now... For giggles and grins, I decided to manually update the offending build script hard coded URL to the current URL structure, and latest expat version (2.4.1 at time of comment) - so my resulting: now is: do_expat_get() {
CT_GetFile "expat-${CT_EXPAT_VERSION}" .tar.gz \
https://sourceforge.net/projects/expat/files/expat/${CT_EXPAT_VERSION}
} my next attempt at building does NOT immediately fail like it did the first time... I am actually trying this as I am commenting, so clearly my build will take some time to complete (if successful this time). I will report back later with final results, and submit a pull request with my suggested fix, should it work. For reference, here are the particulars regarding my local development environment: cbishop@cbishop-ubuntu:~$ uname -a
Linux cbishop-ubuntu 5.8.0-1029-raspi #32-Ubuntu SMP PREEMPT Thu Jun 17 09:56:18 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
cbishop@cbishop-ubuntu:~$
cbishop@cbishop-ubuntu:~$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.10 (Groovy Gorilla)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.10"
VERSION_ID="20.10"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=groovy
UBUNTU_CODENAME=groovy
cbishop@cbishop-ubuntu:~$
cbishop@cbishop-ubuntu:~$ cat /proc/cpuinfo
processor : 0
BogoMIPS : 108.00
Features : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x0
CPU part : 0xd08
CPU revision : 3
processor : 1
BogoMIPS : 108.00
Features : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x0
CPU part : 0xd08
CPU revision : 3
processor : 2
BogoMIPS : 108.00
Features : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x0
CPU part : 0xd08
CPU revision : 3
processor : 3
BogoMIPS : 108.00
Features : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x0
CPU part : 0xd08
CPU revision : 3
Hardware : BCM2835
Revision : d03114
Serial : 1000000019dc0f0c
Model : Raspberry Pi 4 Model B Rev 1.4
cbishop@cbishop-ubuntu:~$ |
Further, strangely it looks like the very issue I note above was already noted and fix suggested (and slightly better suggestion to use the expat official Git Repo vs fixing sourceforge URLs I might add) some time ago here: However despite comments to the contrary - seems that fix must not have been merged into the xtensa-1.2.2.x branch... That change IS however merged in to later release branches - seems the REAL root of my issue was that I was working with an old version of the Espressif esp32 docs... https://docs.espressif.com/projects/esp-idf/en/release-v3.0/get-started/linux-setup-scratch.html when I should have been working with this one: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/linux-setup-scratch.html So - for anyone else who bangs their heads against the wall with any kind of similar toolchain source download failures during build in your local environment - make sure you are using steps to build from the latest version of the docs before diving in to the code head first like is my default setting :) Cheers! |
@chris-bishop Thanks for these suggestions! Unfortunately I keep getting the same error:
The full log is below and has some additional information, not sure if due to a difference between setups or if your report omits a few lines. I've also noticed that the espressif version of crosstool-NG for 210-expat.sh (https://github.com/espressif/crosstool-NG/blob/esp-1.23.x/scripts/build/companion_libs/210-expat.sh) is just
Is Many thanks for any further pointers you may be able to provide!
|
I think retrieving expat-2.2.9 fails primarily because sourceforge has renamed the tarball due to vulnerabilities and recommends using 2.4.1 instead. |
I managed to build the tools by first downloading expat-2.2.9 tar to ~/src
(don't ask me where this path comes from), giving it the correct name, and
then run the install. Later ran into other issues and ultimately just
decided to wait for the official espressif build. Hopefully it's coming
soon (they promised spring, and for osx/aarch64 a version is already
available it seems).
…On Tue, Jul 13, 2021 at 2:16 PM Jan Wieck ***@***.***> wrote:
I think retrieving expat-2.2.9 fails primarily because sourceforge has
renamed the tarbal due to vulnerabilities and recommends using 2.4.1
instead.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#13 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ATHYSZIPHLGJET4IF7CJZLTTXSUKXANCNFSM47LOXLPQ>
.
|
@iot49 my build using expat-2.4.1 is still under way. I'm building this on CentOS 7.9.2009 (AltArch). Let's see how far I can get. There were 4 packages that the process is pulling out of git (binutils, gcc, newlib and gdb). All of them report in the log that they switched to a different commit hash. This also caused problems when I had to restart the build. It resulted in a symlink in .build/tarballs pointing to itself, then later on unpacking it fails (of course). |
@iot49 update: the build with expat-2.4.1 succeeded. However, this is only one step further. Later on installing the esp-idf fails because there are no versions for linux-arm64 of the ulp and openocd components available either and linux-armel ones don't work because they are dynamically linked (statically linked armel 32-bit code seems to run on a PI4). |
There were some changes between build versions to the do_expat_get() {
CT_Fetch EXPAT
} Functions and others like it, where in the older version branch I initially cloned locally (blindly following along with Espressif Docs...) the dependency mirror URLs were all hardcoded, vs the current implementation which does expansion and interpolation as you note. If it rains again all weekend and I am stuck in my bat cave I plan to take another crack at building a proper cross compiler for the esp toolchain on my x86-x64 laptop targeting my aarch64 RPi4B+ Ubuntu 21.04 box, because as you have also noted unfortunately some of the dependency in toolchain still will not properly install on aarch64. I'll gladly share my experience here, as well as working toolchain cross compiler code and config should I be successful. Cheers! |
I also forgot to mention these lines in the stdout log snippet you shared: ...
[ERROR] >> For more info on this error, look at the file: 'build.log'
[ERROR] >> There is a list of known issues, some with workarounds, in:
[ERROR] >> https://crosstool-ng.github.io/docs/known-issues/
[ERROR] >>
... Please forgive me if this is a "blinding flash of the obvious", but whenever you run in to issues I suggest taking a look in the ./build.log file as it generally contains more verbosity and deeper stack traces. |
@chris-bishop yes, that build.log is helpful to get to the actual build problems. As said the build for the xtensa-esp32-elf cross compiler succeeded with using expat 2.4.1. But the other components (like the ultra low power compiler) are also needed for a complete esp-idf, and I am stuck on those now. Checked out https://github.com/espressif/binutils-esp32ulp.git tag v2.28.51-esp-20191205. But that fails with some really strange errors like If I find the time over the weekend I might try to create a different build environment for it. Like Raspbian. |
Ugh, yea those look very similar to issues I kept running in to when I last tried cross compiling for my RPi. Given all the "math" going on there, most likely more of the same juxtaposition of some toolchain submodules compiling without any issue and others which speak a completely different language than our ARM SoCs 😝 I'm hoping to get some time to try again soon. Maybe between the two of us we can submit some Pull Requests that may help others down the line. Cheers! |
Servus, I have found a "dirty" workaround for this problem: With this solution, I have successfully compiled the tool. Saludos. PS. If you compile tool on Raspberry Pi 3+/4 you should also change the value of CT_EXTRA_CFLAGS_FOR_HOST flag to: |
this should be the proper way to fix it: |
That depends on the distro you are using on the PI. Some (like Raspbian) aim for maximum compatibility with whatever and allow to run a lot, with sometimes interesting results. Others (like CentOS) aim for maximum correctness and almost nothing will work, unless it was explicitly compiled for it. |
FYI, I performed a canadian build of this and it worked. But I had to throw 32 cores at it to get the build time down from 2.5 hours on a PI4 to 1 hour on a dual-Xeon-Silver. Very few people have this sort of resources, so I'm not going to continue with that. What I am aiming for now is to put the build of each IDF component into a docker/podman container, then extract the resulting binaries. I often build X86-64 RPMs that way, so nothing new there. If they are linked static they should work on any PI3/4. At least that is my current theory. |
Updated GDB to 9.2 version for xtensa chips Fixed coredump work for xtensa chips Fixed backtrace for xtensa chips Fixed multilib for riscv32 chips Fixed running GDB on some RaspberryPi configuration for riscv32 chip Fixed support of fnmatch(), iconv() and some other posix functions in stdlib Closes #6124 Closes #2484 Closes #3264 Closes espressif/crosstool-NG#13 Closes espressif/crosstool-NG#16
Updated GDB to 9.2 version for xtensa chips Fixed coredump work for xtensa chips Fixed backtrace for xtensa chips Fixed multilib for riscv32 chips Fixed running GDB on some RaspberryPi configuration for riscv32 chip Fixed support of fnmatch(), iconv() and some other posix functions in stdlib Closes #6124 Closes #2484 Closes #3264 Closes espressif/crosstool-NG#13 Closes espressif/crosstool-NG#16
Issue submission guidelines
See detailed guidelines. In short:
build.log
(unless crosstool-NG fails before the build starts). Excerpt:I'm trying this since the official esp-idf does not (yet?) support aarch64.
The text was updated successfully, but these errors were encountered: