Unable to build eden #86
Comments
|
Thank you for the detailed write-up! Our open source story is somewhere nonexistent and lacking. Almost all of the pieces should be there, but we have not yet written up "here's how you run a Mononoke instance", "here's how you configure EdenFS on your local machine", "here's how you package and deploy EdenSCM", etc. The hostcaps issue is a recent regression. It sounds like rust-shed needs open source build support including C FFI. I'm not sure about that Thrift error. |
|
Right. It's possible I'd run into a wall pretty quickly even the build were to succeed, but I assume there are still useful things to learn even if starting a VFS or a server is a challenge. Say, what the CLI even looks like. Is there still a notion of draft and obsolescence markers. What's the speed of operation, say commit/status/update, if I import a working copy from some other repo. |
|
A few things in summary:
|
|
I did some digging around these build failures and I believe one problem is the
The generated files are not good for the OSS build since they are pointing to non-existent files. Namely Not really sure what's the right fix it, but regenerating the files pointing to github might do the trick: @chadaustin Could you fix the build script and regenerate the files whenever you find time? I'd do it if I had access to fbcode :) |
|
I also hit the hostcaps error and I think the OSS build can simply remove it since it's FB only. In fact, should this code even be in github @chadaustin ? I deleted this code and the corresponding include to get past the error, without copying rust-shed: |
I got rid of those by removing the fb feature from here: |
|
woohoo! |
|
@gabrielrussoc thanks for this, I see the problem with fbthrift and tokio_shim. We've hardcoded the internal fbcode path rather than letting it be generated differently internally vs externally. I'll take a look, I've already landed some build fixes for internal eden_scm OSS builds so for that at least we're getting close to it working |
|
Thanks! I managed to build edenfs with #88. It's a bit hard to tell whether the resulting exe work normally though, because I haven't built the other parts of eden. I think there may be something weird with the edenfs executable, because its dependency on libgit2 is dynamic: $ ../eden-install/eden/bin/edenfs
../eden-install/eden/bin/edenfs: error while loading shared libraries: libgit2.so.28: cannot open shared object file: No such file or directory
$ ldd ../eden-install/eden/bin/edenfs | grep 'not found'
libgit2.so.28 => not found
LD_PRELOAD=$PWD/../eden-install/libgit2-kH6rIzSaNvZTBjTar1kL8hrJu4z_QA5f3nbqrz7216g/lib/libgit2.so.28 ../eden-install/eden/bin/edenfs --help
... seems to work ok ...So anyway I looked for a way to build the client. I think $ build/fbcode_builder/getdeps.py build eden_scm
Building on {distro=ubuntu, distro_vers=21.04, fb=off, os=linux, test=on}
Assessing fb303-source...
Using pinned rev 56ad59fa12e136cae61478c13ddb7651c4a31add for https://github.com/facebook/fbthrift.git
Assessing fbthrift-source...
Using pinned rev 56ad59fa12e136cae61478c13ddb7651c4a31add for https://github.com/facebook/fbthrift.git
Assessing openssl...
...
Assessing wangle...
Using pinned rev 56ad59fa12e136cae61478c13ddb7651c4a31add for https://github.com/facebook/fbthrift.git
Assessing fbthrift...
Using pinned rev 56ad59fa12e136cae61478c13ddb7651c4a31add for https://github.com/facebook/fbthrift.git
Assessing rust-shed...
Assessing eden_scm...
Building eden_scm...
---
+ GETDEPS_BUILD_DIR=/tmp/fbcode_builder_getdeps-fbcode_builder/build \
+ GETDEPS_INSTALL_DIR=/tmp/fbcode_builder_getdeps-fbcode_builder/installed \
+ OPENSSL_DIR=/tmp/fbcode_builder_getdeps-fbcode_builder/installed/openssl-OE70IQim5vtlKk6ZHvvzHHZikHn-NXo3s7YCd-58ws8 \
+ cd /tmp/fbcode_builder_getdeps-fbcode_builder/repos/github.com-facebookexperimental-eden.git/eden/scm && \
+ make \
+ -j6 \
+ getdepsbuild \
+ PREFIX=/tmp/fbcode_builder_getdeps-fbcode_builder/installed/eden_scm \
+ prefix=/tmp/fbcode_builder_getdeps-fbcode_builder/installed/eden_scm
mkdir -p /tmp/fbcode_builder_getdeps-fbcode_builder/build/eden_scm
ln -sfn /tmp/fbcode_builder_getdeps-fbcode_builder/build/eden_scm build
ln -sfn /tmp/fbcode_builder_getdeps-fbcode_builder/installed/fbthrift-source/thrift ../../thrift
ln -sfn /tmp/fbcode_builder_getdeps-fbcode_builder/installed/fb303-source/fb303 ../../fb303
mkdir -p ../../common/rust
ln -sfn /tmp/fbcode_builder_getdeps-fbcode_builder/installed/rust-shed/source/shed ../../common/rust/shed
GETDEPS_BUILD=1 \
PYTHON_SYS_EXECUTABLE=/usr/bin/python3 \
THRIFT="/tmp/fbcode_builder_getdeps-fbcode_builder/installed/fbthrift/bin/thrift1" \
/usr/bin/python3 setup.py \
build
/tmp/fbcode_builder_getdeps-fbcode_builder/repos/github.com-facebookexperimental-eden.git/eden/scm/setup.py:31: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
stderr from '/usr/bin/python3 hg log -r. -T {sub("([:+-]|\d\d\d\d$)", "",date|isodatesec)} {node|short}':
b" /usr/bin/python3: can't open file '/tmp/fbcode_builder_getdeps-fbcode_builder/repos/github.com-facebookexperimental-eden.git/eden/scm/hg': [Errno 2] No such file or directory"
running build
running build_mo
warning: hgbuildmo: could not find msgfmt executable, no translations will be built
running build_scripts
cc -g -c contrib/whochanges/whochanges.c -o contrib/whochanges/whochanges.o
cc contrib/whochanges/whochanges.o -o build/scripts-3.9/whochanges
running build_py
error: [Errno 2] No such file or directory: '../../thrift/lib/py'
make: *** [Makefile:97 : getdepsbuild] Erreur 1
Command '['make', '-j6', 'getdepsbuild', 'PREFIX=/tmp/fbcode_builder_getdeps-fbcode_builder/installed/eden_scm', 'prefix=/tmp/fbcode_builder_getdeps-fbcode_builder/installed/eden_scm']' returned non-zero exit status 2.
!! Failedwhich I suspect comes from https://github.com/facebookexperimental/eden/blob/c7762ae593a9d15ea0b134fc57882849e2f32909/eden/scm/setup.py#L755-L769 There's perhaps some kind of local hack I can use to make progress, but I haven't figured it out yet. [1] diff --git a/build/fbcode_builder/manifests/eden_scm b/build/fbcode_builder/manifests/eden_scm
index 52cde91bc2..698507089d 100644
--- a/build/fbcode_builder/manifests/eden_scm
+++ b/build/fbcode_builder/manifests/eden_scm
@@ -6,6 +6,8 @@ shipit_fbcode_builder = true
[git]
repo_url = https://github.com/facebookexperimental/eden.git
+rev = c7762ae593a9d15ea0b134fc57882849e2f32909
+# rev = main
[build.not(os=windows)]
builder = make
diff --git a/build/fbcode_builder/manifests/rust-shed b/build/fbcode_builder/manifests/rust-shed
index c94b3fdd60..afe67c9e8b 100644
--- a/build/fbcode_builder/manifests/rust-shed
+++ b/build/fbcode_builder/manifests/rust-shed
@@ -6,6 +6,7 @@ shipit_fbcode_builder = true
[git]
repo_url = https://github.com/facebookexperimental/rust-shed.git
+rev = main
[build]
builder = cargo |
|
my edenfs binary has a bunch of dynamic dependencies: not sure how to build a static binary, maybe some flag to the build script? For libgit in particular, it is on the binary's rpath which in my case seems to have: I'm also trying this on osx, which probably makes things slightly different Anyway, these binaries are definitely hard to move around. Using |
|
Current state: The OSS build completes now for Eden CLI build and test, Mononoke build and test and Eden FS build. README.md is updated with build steps using getdeps.py to coordinate. EdenFS tests hang about half way through when I run them on github actions, probably due to config missing that is present on our internal hosts. |
I've spent maybe an hour or two trying to build eden, with no success.
The first issue was that it was not entirely clear how to build eden. The root README.md says to use build.sh to build EdenFS, but doesn't mention anything about building the rest of eden, which are the bits I'm interested in. So I looked in eden/scm, but the README there seems to be leftover from hg, and trying to build something there gave me an error saying something like "this needs to be built using getdeps.py", so I assumed maybe the root
build.shbuilds everything and the README is stale.So running build.sh, I got a compilation error in eden/fs/config/EdenConfig.h, due to common/rust/shed/hostcaps/hostcaps.h not existing. So I cloned the rust-shed repo and copied it under common/rust.
Then I got a bunch more compilation errors because the Cargo.toml files request the master branch of the rust-shed repo, which doesn't exist. I rewrote all these from "master" to "main".
Then I got multiple rust build failures on
mod fb;lines, presumably because--features fbwas being passed to cargo/rustc (which I think should be only be used for internal builds?) first in the rust-shed code I copied into the tree, then elsewhere (don't remember where). I couldn't quickly find why the feature was set, and./build.sh --no-facebook-internaldidn't help, so I replaced all"feature = "fb"by"feature = "zz"to deactivate all this and proceed.Then I got a build error because eden/scm/lib/third-party/rust/fbthrift/Cargo.toml mentions ../../../../../../thrift/lib/rust, which does not exist. I replaced it with ../../../../../.././external/fbthrift/thrift/lib/rust, which stopped this error but resulting in multiple new ones, so I don't know if that was the right fix:
external/fbthrift/thrift/lib/rust/Cargo.toml mentions the support_old_nightly feature of async-trait, which apparently doesn't exist. I removed that.
then I got rust errors in external/fbthrift/thrift/lib/rust/src/processor.rs in the NullServiceProcessor implemenation of ServiceProcessor, because the unused parameters of handle_methods are not Send/Sync (??). My rust compiler is already as up to date as it can. So I made everything relevant Send/Sync.
finally, I ran into screenfuls of errors like this:
That's where I'm stuck currently. Any idea how to make progress here?
The text was updated successfully, but these errors were encountered: