Provide an option in ExtensionTask to "only cross compile"

[apolcyn]

Background

When rake cross native gem RUBY_CC_VERSION=.... is used to create gems with pre-compiled binaries for different platforms and ruby versions, all of the binaries are placed under a <gem lib directory>/<ruby version> directory. However, a "native" binary for the current ruby platform and version is also created, and it is placed in <gem lib directory>.

E.g., in the grpc-1.8.7-x86_64 gem, precompiled binaries are found in these locations:

./grpc-1.8.7-x86_64-linux/src/ruby/lib/grpc/2.2/grpc_c.so
./grpc-1.8.7-x86_64-linux/src/ruby/lib/grpc/2.4/grpc_c.so
./grpc-1.8.7-x86_64-linux/src/ruby/lib/grpc/2.0/grpc_c.so
./grpc-1.8.7-x86_64-linux/src/ruby/lib/grpc/grpc_c.so`
./grpc-1.8.7-x86_64-linux/src/ruby/lib/grpc/2.3/grpc_c.so
./grpc-1.8.7-x86_64-linux/src/ruby/lib/grpc/2.1/grpc_c.so

and the /grpc-1.8.7-x86_64-linux/src/ruby/lib/grpc/grpc_c.so binary is the "extra" "native" binary that I'm talking about here. The grpc binary loading process loads the binary under the directory of the current ruby version if it exists, and falls back to lib/grpc/grpc_c.so if it doesn't (the only time this is the case is for packages that were fetched and built from source).

Normally, having that extra lib/grpc/grpc_c.so in the package would not be a problem, since it's not even used anyways. But it turns out to interfere with certain software that doesn't know that it isn't used, because it's dynamically linked to the libruby.so.2.4 in the ruby 2.4.3 installation of rake-compiler-dock's Dockerfile and thus not portable (ldd thinks it's broken).

Suggestion

rake-compiler could provide an option on the ExtensionTask called, e.g., only_cross_compile (which defaults to false). If only_cross_compile was set, then rake-compiler would not generate the "extra" "native" binary, and it would only generate cross-compiled binaries under ruby version-specific directories.

E.g., binaries in the grpc-1.8.7-x86_64 gem would then only be found in these locations:

./grpc-1.8.7-x86_64-linux/src/ruby/lib/grpc/2.2/grpc_c.so
./grpc-1.8.7-x86_64-linux/src/ruby/lib/grpc/2.4/grpc_c.so
./grpc-1.8.7-x86_64-linux/src/ruby/lib/grpc/2.0/grpc_c.so
./grpc-1.8.7-x86_64-linux/src/ruby/lib/grpc/2.3/grpc_c.so
./grpc-1.8.7-x86_64-linux/src/ruby/lib/grpc/2.1/grpc_c.so

This might also be at least somewhat helpful to reduce package size as well.

I have an example patch in this commit.
I'd be happy to create a PR for this too.

[kou]

Umm. It isn't occurred in Rronga. You can confirm by gem unpack https://rubygems.org/downloads/rroonga-7.1.1-x64-mingw32.gem .

Can you show full command lines to reproduce this?

[apolcyn]

It isn't occurred in Rronga

I see... this is interesting but I'm not sure what the cause is yet.

Can you show full command lines to reproduce this?

This is far from a minimal repro but the steps to reproduce using the grpc package build is:

git clone https://github.com/grpc/grpc.git --recursive
cd grpc
bundle install

# This next command will take a while, removing ruby versions and platforms
# from the cross-compilation matrix can speed it up. Note that this rake task
# internally invokes RakeCompilerDock.sh with the "rake cross native gem ... " command
bundle exec rake gem:native
cd pkg
gem unpack grpc-*-x86_64-linux.gem
# when inspecting the unpacked gem, grpc_c.so files can be found under version-specific directories like src/ruby/lib/grpc/2.5/grpc_c.so, but there will also be the undesired
src/ruby/lib/grpc/grpc_c.so file.

please let me know if there are any problems with this repro.

[kou]

Thanks. I can reproduce this case.

How about this patch?

diff --git a/Rakefile b/Rakefile
index 1eac37dc55..dfeb063b0e 100755
--- a/Rakefile
+++ b/Rakefile
@@ -23,6 +23,7 @@ end
 
 # Add the extension compiler task
 Rake::ExtensionTask.new('grpc_c', spec) do |ext|
+  ext.no_native = true
   ext.source_pattern = '**/*.{c,h}'
   ext.ext_dir = File.join('src', 'ruby', 'ext', 'grpc')
   ext.lib_dir = File.join('src', 'ruby', 'lib', 'grpc')

[kou]

You may like this patch:

diff --git a/Rakefile b/Rakefile
index 1eac37dc55..c79db2c61d 100755
--- a/Rakefile
+++ b/Rakefile
@@ -23,6 +23,7 @@ end
 
 # Add the extension compiler task
 Rake::ExtensionTask.new('grpc_c', spec) do |ext|
+  ext.no_native = true
   ext.source_pattern = '**/*.{c,h}'
   ext.ext_dir = File.join('src', 'ruby', 'ext', 'grpc')
   ext.lib_dir = File.join('src', 'ruby', 'lib', 'grpc')
@@ -32,12 +33,15 @@ Rake::ExtensionTask.new('grpc_c', spec) do |ext|
     'x86_64-linux', 'x86-linux',
     'universal-darwin'
   ]
-  ext.cross_compiling do |spec|
-    spec.files = %w( etc/roots.pem grpc_c.32.ruby grpc_c.64.ruby )
-    spec.files += Dir.glob('src/ruby/bin/**/*')
-    spec.files += Dir.glob('src/ruby/ext/**/*')
-    spec.files += Dir.glob('src/ruby/lib/**/*')
-    spec.files += Dir.glob('src/ruby/pb/**/*')
+  ext.cross_compiling do |cross_spec|
+    cross_spec.files -= %w( Makefile .yardopts )
+    cross_spec.files += %w( grpc_c.32.ruby grpc_c.64.ruby )
+    cross_spec.files += Dir.glob('src/ruby/bin/**/*')
+    cross_spec.files += Dir.glob('src/ruby/ext/**/*')
+    cross_spec.files -= Dir.glob('src/boringssl/**/*')
+    cross_spec.files -= Dir.glob('src/core/**/*')
+    cross_spec.files -= Dir.glob('include/**/*')
+    cross_spec.files -= Dir.glob('third_party/**/*')
   end
 end

[apolcyn]

@kou thanks a lot for looking into this. Indeed the first patch seems to work for me, and it also seems to reduce package size significantly by leaving out all of the source files.

BTW, the second patch seems to also work. But if I'm correct, that will effectively cause the same files to be copied into the cross-compiled gem package as are copied in currently (do I have a misunderstanding here?).

[kou]

BTW, the second patch seems to also work. But if I'm correct, that will effectively cause the same files to be copied into the cross-compiled gem package as are copied in currently (do I have a misunderstanding here?).

I'm sorry but I can't understand what you say. (I'm not a native English speaker...)

The second patch just reduces the same logic in Rakefile and grpc.gemspec. The behavior isn't changed.

Can we close this issue?

[apolcyn]

The second patch just reduces the same logic in Rakefile and grpc.gemspec. The behavior isn't changed.

thanks, that answers my question :)

I think this can be closed now