Skip to content

Misc cargo builder improvements#5459

Merged
simi merged 23 commits intoruby:masterfrom
ianks:cargo-builder-target
Jun 6, 2022
Merged

Misc cargo builder improvements#5459
simi merged 23 commits intoruby:masterfrom
ianks:cargo-builder-target

Conversation

@ianks
Copy link
Copy Markdown
Contributor

@ianks ianks commented Apr 7, 2022

Hey there!

After battle testing the cargo code which making Rust play nice with rake-compiler-dock, I've come up with some improvements that will fix some bugs, and make the development experience a bit better all around. Here's some of the things I adjusted

  1. Make sure we can find binaries when CARGO_BUILD_TARGET is detected
  2. Iron out some kinks when parsing link flags, namely the -l:namespec form
  3. Add a configurable runner for the extension builder, so I can print out logs more easily when debugging / developing
  4. Set all of the RbConfig::CONFIG values and environment variables when invoking cargo rustc, so gem extensions do not have to figure out have to invoke the correct ruby binary to print out the correct config

With all of these changes, I am able to cross compile Rust extensions on the following platforms (ruby 2.4, 2.5, 2.6, 2.7, 3.0, 3.1):

  • x86_64-linux
  • aarch64-linux
  • arm-linux
  • x86_64-darwin
  • arm64-darwin
  • x64-mingw32
  • x64-mingw-ucrt

Comment thread lib/rubygems/ext/cargo_builder.rb Outdated
Comment thread lib/rubygems/ext/cargo_builder.rb Outdated
Comment thread lib/rubygems/ext/cargo_builder.rb Outdated
Comment thread lib/rubygems/ext/cargo_builder.rb Outdated
Comment thread test/rubygems/test_gem_ext_cargo_builder.rb Outdated
end

def self.run(command, results, command_name = nil, dir = Dir.pwd)
def self.run(command, results, command_name = nil, dir = Dir.pwd, env = {})
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When cross-compiling with multiple ruby versions present, it's necessary to pass information about RbConfig to cargo so we can ensure proper linking. I found using environment variables is a simple and effective way to do this.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand it well, this argument is not used directly by RubyGems now. Can you confirm?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I understand, all RubyGems does is pass on the given environment to cargo. This feels nit to me, because it's more explicit than setting up the environment globally just so it is inherited by the cargo subprocess.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RubyGems does not use this, other than Cargo builder. It allow us to pass env variables without setting them globally for the entire process.

build_env
end

def cargo_command(cargo_dir, dest_path, args = [])
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to expose the actual cargo command to run, so it can be used in a Makefile later.

@ianks
Copy link
Copy Markdown
Contributor Author

ianks commented Apr 11, 2022

Not sure why, but it looks like the Windows actions were interrupted causing CI to ❌. @simi could you trigger a re-run?

@ianks ianks requested review from deivid-rodriguez and simi April 13, 2022 13:31
Comment thread test/rubygems/test_gem_ext_cargo_builder/rust_ruby_example/Cargo.toml Outdated
Comment thread test/rubygems/test_gem_ext_cargo_builder_unit.rb
@ianks ianks requested a review from simi April 13, 2022 16:55
@deivid-rodriguez
Copy link
Copy Markdown
Contributor

@ianks Sorry for dropping the ball here, I will do my best to get this reviewed and shipped with the next release!

@ianks
Copy link
Copy Markdown
Contributor Author

ianks commented May 9, 2022

All good @deivid-rodriguez! I need to make a couple of adjustments on my end as well to get CI passing

@ianks
Copy link
Copy Markdown
Contributor Author

ianks commented Jun 3, 2022

@deivid-rodriguez Should be good to go.

@simi simi self-assigned this Jun 3, 2022
assert_match "Finished release [optimized] target(s)", output
assert_ffi_handle bundle, 'Init_rust_ruby_example'
rescue Exception => e
pp output if output
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intended to be left in here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it really does make all of the difference when debugging. I can remove it if you prefer though!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks strange to me too, but I can see how it's useful, and the code was already there, so ok!

[dependencies]
# Needed until bindgen has the `allowlist_file` feature
rb-sys = { git = "https://github.com/ianks/rb-sys", tag = "v0.9.0" }
rb-sys = "0.9.3"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💪 finally :)

@simi
Copy link
Copy Markdown

simi commented Jun 3, 2022

I have left minor comments. Otherwise looks good to me.

Copy link
Copy Markdown
Contributor

@deivid-rodriguez deivid-rodriguez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If @simi is fine with this, I'm fine too :)

@simi simi merged commit 632a8bc into ruby:master Jun 6, 2022
@simi
Copy link
Copy Markdown

simi commented Jun 6, 2022

Now I need to finish support for bundle gem and we're feature complete here. 🙏

deivid-rodriguez pushed a commit that referenced this pull request Jun 15, 2022
Misc cargo builder improvements

(cherry picked from commit 632a8bc)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants