How to build Cargo from GIT on FreeBSD? #429

ghost · 2014-08-23T18:24:04Z

/usr/local/bin/python src/etc/dl-snapshot.py x86_64-unknown-freebsd
Traceback (most recent call last):
  File "src/etc/dl-snapshot.py", line 31, in <module>
    raise Exception("no snapshot for the triple: " + triple)
Exception: no snapshot for the triple: x86_64-unknown-freebsd
gmake: *** [target/snapshot/bin/cargo] Error 1

The text was updated successfully, but these errors were encountered:

steveklabnik · 2014-08-23T19:06:24Z

This means there's no Rust snapshot for FreeBSD, which is true.

Since Cargo builds with its own snapshot, I'm not sure.

ghost · 2014-08-23T19:57:27Z

I've already built rust from source on FreeBSD, is there a way to tell cargo's configure to use it instead of pulling a snapshot?

FreeFull · 2014-08-23T20:33:35Z

Cargo also requires a snapshot of itself to be built. I'm not sure if that exists for FreeBSD either.

mahkoh · 2014-08-23T21:51:04Z

https://gist.github.com/anonymous/f676d09b0bcaab0825d6

This builds cargo without cargo.

ghost · 2014-08-24T04:01:39Z

Refreshed to latest nightly and this script ends with:

Cloning into 'cargo'...
remote: Counting objects: 211, done.
remote: Compressing objects: 100% (195/195), done.
remote: Total 211 (delta 17), reused 124 (delta 10)
Receiving objects: 100% (211/211), 8.99 MiB | 2.63 MiB/s, done.
Resolving deltas: 100% (17/17), done.
Checking connectivity... done.
error: Argument to option 'extern' missing.
cargo-bs.sh: --extern: not found
cargo-bs.sh: cargo/src/cargo/lib.rs: Permission denied

But that's closer than where I was earlier.

ghost · 2014-08-24T04:15:07Z

Ok, that's a copy and paste problem. Checking my copy of the script.

ghost · 2014-08-24T04:26:28Z

Ok, @mahkoh 's gist gives me a functional cargo suite on FreeBSD. Thank you! You rock!

ebfe · 2014-10-03T15:50:42Z

Heres an updated version that works with the current master branch: https://gist.github.com/ebfe/dcb914d907c4a54a7b8d

MattWindsor91 · 2014-12-10T01:21:42Z

@ebfe's gist no longer works. I've tried to hack on it to get it working, but I've hit ICEs and given up. Anyone have a working way of getting a cargo snapshot?

csperkins · 2015-01-02T14:27:21Z

I hacked that Gist to get it working with a more recent snapshot, and added the ability to bootstrap up to the latest version. See https://csperkins.org/research/misc/2015-01-02-cargo-freebsd.html for details.

FrankReh · 2015-01-10T19:30:38Z

When might we be able to build cargo on FreeBSD 10.1 more directly? I'm new to rust and just want to play with it awhile. I'm inclined to try reading through the book without a working cargo. I understand it is at 1.0 alpha so an update and rebuild every couple of days or so is okay but I probably don't have the time for a long rebuild of cargo each time I want to update rust.

csperkins · 2015-01-10T22:13:51Z

Once it's bootstrapped, updates are pretty quick.

FrankReh · 2015-01-11T04:54:16Z

Thanks. So decided to give it a try but v5 and v4 both stop when trying to
build the first stage with gmake. I don't see where this error message is
even coming from. Earlier, I was able to get rustc built from git
directly, so I think I have the correct tools in place for the
environment. Maybe tomorrow this error message will make more sense to
me...

Specify at least 1 interface

/home/frank/src/rust-nightly-2014-12-12/mk/stage0.mk:17: recipe for target
'x86_64-unknown-freebsd/stage0/bin/rustc' failed

On Sat, Jan 10, 2015 at 5:13 PM, Colin Perkins notifications@github.com
wrote:

Once it's bootstrapped, updates are pretty quick.

—
Reply to this email directly or view it on GitHub
#429 (comment).

yarwelp · 2015-01-29T20:31:44Z

@csperkins I downloaded your build-cargo-freebsd-v7.sh you linked from your blog post (the one you linked four comments before my reply) on my FreeBSD/amd64 10.1-RELEASE-p5 VPS and added a new user which I ran the script as in a screen (to have a clean homedir for it so I can easily make a tarball of everything should I want to). As expected, it is going slow and it's not yet finished running.

The VPS does not have a lot of RAM, only 512MB. I have 1024MB swap. (I'm still not sure whether there is any point to having more swap than RAM. I also don't know if swap helps at all when there is pretty much just one process eating all of the RAM.) It might happen that it will be killed before finishing (as has happened to me with other RAM hungry processes previously), we'll see.

Anyway, I was wanting to let you know that there is a little thing I changed in the script. Here is the diff:

--- build-cargo-freebsd-v7.sh.orig  2015-01-29 19:52:54.000000000 +0000
+++ build-cargo-freebsd-v7.sh   2015-01-29 20:19:54.000000000 +0000
@@ -69,7 +69,7 @@

 if [ ! -d $PREFIX/target ]; then
   echo "*** mkdir $PREFIX/target"
-  mkdir $PREFIX/target
+  mkdir -p $PREFIX/target
 fi

 # =================================================================================================

edit: I was watching top and the amount of memory consumed by rustc scared me a bit so I added another 4G of swap in the hopes that doing so would keep it from getting killed. I am watching it still (though I don't intend to be watching the whole time of course) and its state is occasionally changing between RUN, pfault and swread so that seems promising.

csperkins · 2015-01-29T21:21:34Z

Thanks for the fix.

With only 512MB RAM the initial build is going to be alarmingly slow – last time I tried to bootstrap from scratch it ran overnight on a server with 64 cores and 512GB RAM...

yarwelp · 2015-01-29T23:12:31Z

On first attempt, it stopped though I didn't see any message about a process getting killed. I somewhat suspect that the old version of rust (0.12.0_1) I installed using pkg install rust earlier may have caused trouble. I removed the old version and deleted everything in the homedir of the user I created and am now rerunning the script, this time doing ./build-cargo-freebsd-v7.sh 2>&1 | tee -a build-cargo-freebsd-v7.log so I'll have a complete log which is easier to read than using page up and page down inside the screen. Still running that in screen, though, so I don't have to remain connected to the server. (I have also reduced the amount of swap from 4G extra to 2G extra for a total of 3G instead of 5G because I saw in dmesg that it warned about total configured swap exceeding maximum recommended amount, which I found strange since it's amd64 and not i386.)

yarwelp · 2015-01-30T14:15:09Z

The second attempt ended with an error, unfortunately.

It said cc: error: linker command failed with exit code 1 (use -v to see invocation) and on the line before that, note: /usr/bin/ld: cannot find -lflate. I'm not sure whether this is referring to a rust library or an external library.

Full log in case anybody could take a look: https://gist.githubusercontent.com/erikano/1ccb7ad4eb37e23a7d2c/raw/build-cargo-freebsd-v7.log

edit: I made a post about it to /r/rust on reddit. https://www.reddit.com/r/rust/comments/2u76ox/building_cargo_from_git_on_freebsd_fails_could/

yarwelp · 2015-01-30T15:47:20Z

I have gotten it to go further. After where it failed, I first did pkg install cmake since that was missing and I hadn't noticed. Then, manually as the user and without removing the failed stuff, I did:

export TARGET=/home/rust/src/cargo-freebsd/target
export LD_LIBRARY_PATH=$TARGET/rust-nightly-2014-12-12/lib
export CARGOLIB=$TARGET/cargo-bootstrap/lib
export RUSTC=$TARGET/rust-nightly-2014-12-12/bin/rustc
cd cargo/bootstrap/libgit2/build/
cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/apps/cargo -DBUILD_SHARED_LIBS=ON
cmake --build . --target install
cd

Then attempted to run $RUSTC cargo/bootstrap/cargo/src/bin/cargo.rs -L $HOME/apps/cargo/lib -L $CARGOLIB --out-dir $TARGET/cargo-bootstrap/bin -lflate-4e7c5e5c -lcurl -lssl -lcrypto -lz -lssh2 -lgit2 (notice I've replaced -lflate with -lflate-4e7c5e5c) and found out that libssh2 was missing so I did pkg install libssh2 and ran the rustc command for cargo.rs again.

Then checked to see that file $TARGET/cargo-bootstrap/bin/cargo existed, which it now did so proceeded copying from the script:

mkdir -p $HOME/apps/cargo/bin
install -m 755 $TARGET/cargo-bootstrap/bin/cargo  $HOME/apps/cargo/bin/
echo "0caa5b58fd4aac95d3388bed878797becf544215" > $HOME/apps/cargo/cargo-version.txt

Attempted to run ./apps/cargo/bin/cargo --version but got Shared object "libgit2.so.21" not found, required by "cargo". Ok, export LD_LIBRARY_PATH=$HOME/apps/cargo/lib:$LD_LIBRARY_PATH. Try again ./apps/cargo/bin/cargo --version.

cargo 0.0.1

So far, so good. So now I think the script by @csperkins should be able to continue. I'm going to try. First, I log out and back in just to be sure that the environment variables I had set don't affect the script. I then start a screen and do ./build-cargo-freebsd-v7.sh 2>&1 | tee -a build-cargo-freebsd-v7.cont.log. We'll see how it goes.

edit: The first thing that happened next was it couldn't find python even though I have python2.7 so I aborted and did pkg install python rather than create a symbolic link myself so now I have a python which is a symbolic link to python2 which is a symbolic link to python2.7.

Retrying then, I got:

*** Clone Cargo repository
Cloning into 'cargo/head'...
*** Installed Cargo is       72 merges behind HEAD
*** Processing cargo/a0e7e7a88b23a0d372655f52769ef0d36dc34d49
*** git checkout cargo/a0e7e7a88b23a0d372655f52769ef0d36dc34d49 a0e7e7a88b23a0d372655f52769ef0d36dc34d49
*** Run git submodule update --init in cargo/a0e7e7a88b23a0d372655f52769ef0d36dc34d49
*** rust-nightly-2014-12-12 already exists
*** Configure cargo/a0e7e7a88b23a0d372655f52769ef0d36dc34d49 - already configured
*** Build cargo/a0e7e7a88b23a0d372655f52769ef0d36dc34d49
rustc 0.13.0-dev
<jemalloc>: Error in dlsym(RTLD_NEXT, "pthread_create")
Makefile:78: recipe for target 'cargo-x86_64-unknown-freebsd' failed
gmake: *** [cargo-x86_64-unknown-freebsd] Abort trap (core dumped)
*** Build failed - skip? (y/n)
n
*** Abort

I deleted the directory cargo/a0e7e7a88b23a0d372655f52769ef0d36dc34d49 thinking it might be because the configure had been incomplete and attempted again. It configures but gets the same error. Don't know what to do. Log: https://gist.githubusercontent.com/erikano/e4adc8f66a198c6c57b0/raw/d26a8b6eb1455113687f3df0c6d6da18f51d611b/build-cargo-freebsd-v7.cont.log.

hnakamur · 2015-05-26T21:59:28Z

I updated the script referred at #429 (comment)
https://gist.github.com/hnakamur/4b85c051bfbc518c64df

However rustc panicked while building rust-encoding.
https://gist.github.com/hnakamur/4b85c051bfbc518c64df#comment-1461070

I want to fix this but don't know how. Could someone help me, please?

PS: Now I am trying CaptainHayashi's script at #429 (comment)
I will write the results when it finishes.

hnakamur · 2015-05-26T23:56:39Z

I got the following errors while running CaptainHayashi's script at #429 (comment)

-- Could NOT find HTTP_Parser (missing:  HTTP_PARSER_INCLUDE_DIR HTTP_PARSER_LIBRARY)
-- http-parser was not found or is too old; using bundled 3rd-party sources.
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/home/vagrant/build-cargo/cargo/9d550782bd9aa36341e4a3efcef3dd59507f3c0a/target/x86_64-unknown-freebsd/build/libgit2-sys-97337dcdf429fa77/out/build
running: cmake '--build' '.' '--target' 'install'
[  0%] Building C object CMakeFiles/git2.dir/src/openssl_stream.c.o
CMakeFiles/git2.dir/build.make:1710: recipe for target 'CMakeFiles/git2.dir/src/openssl_stream.c.o' failed
CMakeFiles/Makefile2:60: recipe for target 'CMakeFiles/git2.dir/all' failed
Makefile:117: recipe for target 'all' failed

--- stderr
/home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/openssl_stream.c:100:18: error: variable has incomplete type 'struct in6_addr'
        struct in6_addr addr6;
                        ^
/home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/openssl_stream.c:100:9: note: forward declaration of 'struct in6_addr'
        struct in6_addr addr6;
               ^
/home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/openssl_stream.c:111:18: error: use of undeclared identifier 'AF_INET'
        if (p_inet_pton(AF_INET, host, &addr4)) {
                        ^
/home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/unix/posix.h:31:40: note: expanded from macro 'p_inet_pton'
#define p_inet_pton(a, b, c) inet_pton(a, b, c)
                                       ^
/home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/openssl_stream.c:115:18: error: use of undeclared identifier 'AF_INET6'
                if(p_inet_pton(AF_INET6, host, &addr6)) {
                               ^
/home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/unix/posix.h:31:40: note: expanded from macro 'p_inet_pton'
#define p_inet_pton(a, b, c) inet_pton(a, b, c)
                                       ^
3 errors generated.
gmake[3]: *** [CMakeFiles/git2.dir/src/openssl_stream.c.o] Error 1
gmake[2]: *** [CMakeFiles/git2.dir/all] Error 2
gmake[1]: *** [all] Error 2
task '<main>' panicked at 'assertion failed: cmd.stdout(InheritFd(1)).stderr(InheritFd(2)).status().unwrap().success()', build.rs:80

Makefile:78: recipe for target 'cargo-x86_64-unknown-freebsd' failed
gmake: *** [cargo-x86_64-unknown-freebsd] Error 101
*** Build failed - skip? (y/n)

alexcrichton · 2015-05-27T00:00:46Z

@hnakamur there's a new script written by @dhuseby which was used to bootstrap Cargo on bitrig which may help on FreeBSD as well!

hnakamur · 2015-05-27T00:01:43Z

@alexcrichton Thanks! I will try that.

csperkins · 2015-05-27T07:13:34Z

This is a known bug in libgit2-sys, which is missing some includes for FreeBSD. The simplest fix is to edit the downloaded source, adding:

 #include <sys/socket.h>
 #include <netinet/in.h>

just before the #include <arpa/inet.h> around line 56 of common.h in the libgit2-sys download (looking at the errors, this will be /home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/common.h on your machine).

Colin

On 27 May 2015, at 00:56, Hiroaki Nakamura notifications@github.com wrote:

I got the following errors while running CaptainHayashi's script at #429 (comment)

[ 57%] Building C object CMakeFiles/git2.dir/src/openssl_stream.c.o
CMakeFiles/git2.dir/build.make:1710: recipe for target 'CMakeFiles/git2.dir/src/openssl_stream.c.o' failed
CMakeFiles/Makefile2:60: recipe for target 'CMakeFiles/git2.dir/all' failed
Makefile:117: recipe for target 'all' failed

--- stderr
/home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/openssl_stream.c:100:18: error: variable has incomplete type 'struct in6_addr'
struct in6_addr addr6;
^
/home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/openssl_stream.c:100:9: note: forward declaration of 'struct in6_addr'
struct in6_addr addr6;
^
/home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/openssl_stream.c:111:18: error: use of undeclared identifier 'AF_INET'
if (p_inet_pton(AF_INET, host, &addr4)) {
^
/home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/unix/posix.h:31:40: note: expanded from macro 'p_inet_pton'
#define p_inet_pton(a, b, c) inet_pton(a, b, c)
^
/home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/openssl_stream.c:115:18: error: use of undeclared identifier 'AF_INET6'
if(p_inet_pton(AF_INET6, host, &addr6)) {
^
/home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.1.3/libgit2/src/unix/posix.h:31:40: note: expanded from macro 'p_inet_pton'
#define p_inet_pton(a, b, c) inet_pton(a, b, c)

3 errors generated.
gmake[3]: *** [CMakeFiles/git2.dir/src/openssl_stream.c.o] Error 1
gmake[2]: *** [CMakeFiles/git2.dir/all] Error 2
gmake[1]: *** [all] Error 2
task '
' panicked at 'assertion failed: cmd.stdout(InheritFd(1)).stderr(InheritFd(2)).status().unwrap().success()', build.rs:80

Makefile:78: recipe for target 'cargo-x86_64-unknown-freebsd' failed
gmake: *** [cargo-x86_64-unknown-freebsd] Error 101
*** Build failed - skip? (y/n)
—
Reply to this email directly or view it on GitHub.

Colin Perkins
https://csperkins.org/

hnakamur · 2015-05-27T13:30:31Z

@csperkins Thanks for the info.
I found the it is already fixed in the latest libgit2: Merge pull request #2844 from rakuco/netinet-include-fixes · libgit2/libgit2@65bc22f
So I forked git2-rs and update Cargo.toml to use the latest version of libgit2-sys.
alexcrichton/git2-rs@master...hnakamur:update_libgit2_sys
Also I updated my build script.
Revisions · build-cargo-freebsd.sh

Now I got the following error:

+ rustc deps/openssl-sys/src/lib.rs --crate-type lib --crate-name openssl_sys --out-dir target -L target/
deps/openssl-sys/src/probe.rs:2:5: 2:32 error: unresolved import `std::io::fs::PathExtensions`. Could not find `fs` in `std::io`
deps/openssl-sys/src/probe.rs:2 use std::io::fs::PathExtensions;
                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~
error: aborting due to previous error

I googled about this, but I couldn't find a workaround.

hnakamur · 2015-05-27T13:31:36Z

I tried https://github.com/dhuseby/cargo-bootstrap and got "can't find crate for openssl_sys" error.
I made an issue about that: can't find crate for openssl_sys on FreeBSD10.1 · Issue #3 · dhuseby/cargo-bootstrap

dhuseby · 2015-05-27T17:06:01Z

@hnakamur which toolchain are you using on FreeBSD 10.1? Is it clang or gcc? IIRC you can choose at install, right?

hnakamur · 2015-05-27T20:51:15Z

@dhuseby I use clang 3.4.1.

$ cc --version
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
Target: x86_64-unknown-freebsd10.1
Thread model: posix

I built rustc with the following steps.

sudo pkg install -y python gmake git
git clone https://github.com/rust-lang/rust.git
cd rust
./configure
gmake
sudo gmake install

dhuseby · 2015-05-27T23:44:48Z

I just made some fixes to my bootstrap.py script that makes it succesfully bootstrap cargo on FreeBSD. I was able to generate a cargo executable with the script and then build cargo from source using the exe as --local-cargo. Cargo works great on FreeBSD now. 👍

hnakamur · 2015-05-27T23:46:09Z

@dhuseby Thanks! I will try the latest version of dhuseby/cargo-bootstrap.

ghost closed this as completed Aug 24, 2014

hnakamur mentioned this issue May 27, 2015

can't find crate for openssl_sys on FreeBSD10.1 dhuseby/cargo-bootstrap#3

Closed

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to build Cargo from GIT on FreeBSD? #429

How to build Cargo from GIT on FreeBSD? #429

ghost commented Aug 23, 2014

steveklabnik commented Aug 23, 2014

ghost commented Aug 23, 2014

FreeFull commented Aug 23, 2014

mahkoh commented Aug 23, 2014

ghost commented Aug 24, 2014

ghost commented Aug 24, 2014

ghost commented Aug 24, 2014

ebfe commented Oct 3, 2014

MattWindsor91 commented Dec 10, 2014

csperkins commented Jan 2, 2015

FrankReh commented Jan 10, 2015

csperkins commented Jan 10, 2015

FrankReh commented Jan 11, 2015

yarwelp commented Jan 29, 2015

csperkins commented Jan 29, 2015

yarwelp commented Jan 29, 2015

yarwelp commented Jan 30, 2015

yarwelp commented Jan 30, 2015

hnakamur commented May 26, 2015

hnakamur commented May 26, 2015

alexcrichton commented May 27, 2015

hnakamur commented May 27, 2015

csperkins commented May 27, 2015

hnakamur commented May 27, 2015

hnakamur commented May 27, 2015

dhuseby commented May 27, 2015

hnakamur commented May 27, 2015

dhuseby commented May 27, 2015

hnakamur commented May 27, 2015

How to build Cargo from GIT on FreeBSD? #429

How to build Cargo from GIT on FreeBSD? #429

Comments

ghost commented Aug 23, 2014

steveklabnik commented Aug 23, 2014

ghost commented Aug 23, 2014

FreeFull commented Aug 23, 2014

mahkoh commented Aug 23, 2014

ghost commented Aug 24, 2014

ghost commented Aug 24, 2014

ghost commented Aug 24, 2014

ebfe commented Oct 3, 2014

MattWindsor91 commented Dec 10, 2014

csperkins commented Jan 2, 2015

FrankReh commented Jan 10, 2015

csperkins commented Jan 10, 2015

FrankReh commented Jan 11, 2015

yarwelp commented Jan 29, 2015

csperkins commented Jan 29, 2015

yarwelp commented Jan 29, 2015

yarwelp commented Jan 30, 2015

yarwelp commented Jan 30, 2015

hnakamur commented May 26, 2015

hnakamur commented May 26, 2015

alexcrichton commented May 27, 2015

hnakamur commented May 27, 2015

csperkins commented May 27, 2015

hnakamur commented May 27, 2015

hnakamur commented May 27, 2015

dhuseby commented May 27, 2015

hnakamur commented May 27, 2015

dhuseby commented May 27, 2015

hnakamur commented May 27, 2015