Skip to content

rocksdb: fix cross-compiling on darwin#216450

Open
yihuang wants to merge 2 commits intoNixOS:masterfrom
yihuang:master
Open

rocksdb: fix cross-compiling on darwin#216450
yihuang wants to merge 2 commits intoNixOS:masterfrom
yihuang:master

Conversation

@yihuang
Copy link
Copy Markdown
Contributor

@yihuang yihuang commented Feb 15, 2023

install_name_tool not found when crossing compiling on darwin

Closes: #216431

Description of changes
Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot added the 6.topic: cross-compilation Building packages on a different platform than they will be used on label Feb 15, 2023
@ofborg ofborg bot requested a review from adevress February 15, 2023 07:43
@ofborg ofborg bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Feb 15, 2023
@yihuang yihuang marked this pull request as draft February 15, 2023 08:02
@yihuang yihuang force-pushed the master branch 2 times, most recently from ef62570 to 91d07e7 Compare February 15, 2023 09:41
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. and removed 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. labels Feb 15, 2023
@yihuang yihuang marked this pull request as ready for review February 15, 2023 10:09
Copy link
Copy Markdown
Member

@winterqt winterqt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit with regards to the commit message (if you think it's fine, feel free to ignore this): maybe mention "fix cross-compilation on darwin" in the first line? It doesn't really matter since you mention it in the description though :)

@yihuang
Copy link
Copy Markdown
Contributor Author

yihuang commented Feb 22, 2023

Nit with regards to the commit message (if you think it's fine, feel free to ignore this): maybe mention "fix cross-compilation on darwin" in the first line? It doesn't really matter since you mention it in the description though :)

done, fixed.

@yihuang yihuang changed the title rocksdb: fix cross-compiling rocksdb: fix cross-compiling on darwin Feb 22, 2023
@ofborg ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Feb 22, 2023
@yihuang yihuang requested review from winterqt and removed request for adevress February 22, 2023 05:51
@yihuang
Copy link
Copy Markdown
Contributor Author

yihuang commented Feb 22, 2023

@winterqt may I ask why some derivations don't need to include this prefix but still build succesfully?

@winterqt
Copy link
Copy Markdown
Member

winterqt commented Feb 22, 2023

@yihuang Can you remove the Update ... from the commit message (and optionally the Co-authored-by trailer, I don't really think I deserve it for just fixing a small thing)?

may I ask why some derivations don't need to include this prefix but still build succesfully?

Do you have an example in mind? I assume you mean when cross-compiling -- that does sound weird.

@yihuang
Copy link
Copy Markdown
Contributor Author

yihuang commented Feb 22, 2023

Do you have an example in mind? I assume you mean when cross-compiling -- that does sound weird.

zstd cross-build fine without this prefix.

install_name_tool not found when crossing compiling on darwin
@yihuang
Copy link
Copy Markdown
Contributor Author

yihuang commented Feb 22, 2023

@yihuang Can you remove the Update ... from the commit message (and optionally the Co-authored-by trailer, I don't really think I deserve it for just fixing a small thing)?

done

@winterqt
Copy link
Copy Markdown
Member

zstd cross-build fine without this prefix.

Huh, that is weird, as even the hook it uses specifies the prefix.

I assume I can reproduce this by just trying to cross build from x86_64-darwin to aarch64-darwin?

@yihuang
Copy link
Copy Markdown
Contributor Author

yihuang commented Feb 22, 2023

zstd cross-build fine without this prefix.

Huh, that is weird, as even the hook it uses specifies the prefix.

I assume I can reproduce this by just trying to cross build from x86_64-darwin to aarch64-darwin?

yes, that's what I'm doing.

@stephank
Copy link
Copy Markdown
Contributor

Do you have an example in mind? I assume you mean when cross-compiling -- that does sound weird.

zstd cross-build fine without this prefix.

Are you referring to this install_name_tool invocation?

install_name_tool -change @rpath/libzstd.1.dylib $out/lib/libzstd.1.dylib $bin/bin/pzstd

That is dependant on the buildContrib flag, which is disabled when cross-compiling:

, buildContrib ? stdenv.hostPlatform == stdenv.buildPlatform

@yihuang
Copy link
Copy Markdown
Contributor Author

yihuang commented Feb 22, 2023

Do you have an example in mind? I assume you mean when cross-compiling -- that does sound weird.

zstd cross-build fine without this prefix.

Are you referring to this install_name_tool invocation?

install_name_tool -change @rpath/libzstd.1.dylib $out/lib/libzstd.1.dylib $bin/bin/pzstd

That is dependant on the buildContrib flag, which is disabled when cross-compiling:

, buildContrib ? stdenv.hostPlatform == stdenv.buildPlatform

I see, that makes sense, thanks

@ofborg ofborg bot requested a review from adevress February 23, 2023 02:12
@ofborg ofborg bot added 10.rebuild-darwin: 101-500 This PR causes between 101 and 500 packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 501-1000 This PR causes many rebuilds on Linux and should normally target the staging branches. and removed 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Feb 23, 2023
@winterqt
Copy link
Copy Markdown
Member

Was the second commit meant to go in a different PR? I thought rocksdb worked with just the install_name_tool reference fix?

@yihuang
Copy link
Copy Markdown
Contributor Author

yihuang commented Feb 23, 2023

Was the second commit meant to go in a different PR? I thought rocksdb worked with just the install_name_tool reference fix?

I just found out, the cross-compiled library is like this:

$ otool -L ./result/lib/librocksdb.7.dylib
./result/lib/librocksdb.7.dylib:
	@rpath/librocksdb.7.dylib (compatibility version 7.0.0, current version 7.9.2)
	/nix/store/13aj4w3gzw1xf0clx81nkd5jvl63fmnq-jemalloc-aarch64-apple-darwin-5.3.0/lib/libjemalloc.2.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libsnappy.1.dylib (compatibility version 1.0.0, current version 1.1.9)
...

there are two link path not replaced with absolute path, it doesn't happen when doing native build. I don't know where is the best place to fix this.

And the @rpath/librocksdb.7.dylib I can't replace it even with the install_name_tool command.

@wegank wegank added 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 2.status: merge conflict This PR has merge conflicts with the target branch labels Mar 19, 2024
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 20, 2024
@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2.status: merge conflict This PR has merge conflicts with the target branch 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: cross-compilation Building packages on a different platform than they will be used on 6.topic: darwin Running or building packages on Darwin 10.rebuild-darwin: 101-500 This PR causes between 101 and 500 packages to rebuild on Darwin. 10.rebuild-linux: 501-1000 This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rocksdb: install_name_tool not found when cross compiling on darwin

5 participants