Support for alternatives implementations: TruffleRuby, JRuby, etc

[eregon]

There are multiple implementations of Ruby (see for example https://www.ruby-lang.org/en/about/).
Ruby library (gem) maintainers like to test against these other Ruby implementations in CI, to make sure their library works on that implementation too.

Should actions/setup-ruby be able to setup those Ruby implementations?
Or should it be a separate repository to setup those Ruby implementations?

I think having it in actions/setup-ruby would be more convenient, as one would be able to reuse the same workflow file for testing both on MRI and e.g. on JRuby and on TruffleRuby:

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        ruby: [ '2.6.x', 'jruby-9.2.8.0', 'truffleruby-19.2' ]
    name: Ruby ${{ matrix.ruby }} sample
    steps:
      - uses: actions/checkout@master
      - name: Setup ruby
        uses: actions/setup-ruby@v1
        with:
          ruby-version: ${{ matrix.ruby }}
      - run: ruby hello.rb

I think most alternative Ruby implementations provide their own binaries (while MRI does not), so the setup is usually fairly simple, and usually consists of just downloading + adding to PATH, sometimes with an extra command to run.
For instance for TruffleRuby it's just:

curl -L https://github.com/oracle/truffleruby/releases/download/vm-$VERSION/truffleruby-$VERSION-linux-amd64.tar.gz | tar xz
export PATH="$PWD/truffleruby-$VERSION-linux-amd64/bin:$PATH"
$PWD/truffleruby-$VERSION-linux-amd64/lib/truffle/post_install_hook.sh

[damccorm]

This is an interesting question. I don't see us doing this immediately since there are some other, probably more pressing needs even in this repo, but I'm not quite sure what makes sense here.

Questions that immediately pop up if we decide to do this:

• How do we pick which implementations we support? Even on the site you linked to, some versions like IronRuby are listed that are discontinued.
• Should this be in this repo or another one (you brought up this question)? I think I tend to agree that having it all in this repo makes workflows cleaner, but there's also an argument to be had that these are different tools and should be downloaded seperately. By adding this, we probably add a lot of ugly if branching logic and a non-trivial support burden.
• How much is this needed? That feeds in to the previous question - if this is going to serve a lot of people then adding a support burden is a no brainer. If its not a common scenario then we might want to stay away, especially since its relatively easy to grab these with your own script.

I'm not really convinced on what we should do either way - @chrispat any thoughts here?

[eregon]

How do we pick which implementations we support?

Currently, the major Ruby implementations targeting full compatibility with MRI are JRuby, TruffleRuby and Rubinius.

Maybe a good way to know is simply letting people contribute the setup for new implementations to this repository. I'd be happy to add support for TruffleRuby for instance.

By adding this, we probably add a lot of ugly if branching logic and a non-trivial support burden.

A few if for sure, but I think it would still be only 1-2 if per implementation, and there is opportunity to share logic for e.g., downloading the tarball. It can also be done in a more object-oriented manner using polymorphism of course.
Maybe the best way to evaluate this would be to discuss on a PR for one of these implementations.

Re support, I would expect little maintenance is needed, the process to install an alternate Ruby should not change over time. If it changes, it seems fair to ask the community to update the logic.

How much is this needed?

I would say it's very frequent for Ruby libraries. I think many Ruby library developers expect to be able to test against multiple Ruby implementations.
For instance the most downloaded gems seem to almost all test on JRuby: rspec, bundler, rubygems, rake, json, ...

[damccorm]

Thanks for the thoughtful issue/responses. As I think more about it, I think I'm on board with this - lets do it!

I don't have the bandwidth to get to this immediately, but am more than happy to review a PR and/or leave this issue open and do this in the future.

If you do end up doing a PR for TruffleRuby (or something else), I'd recommend taking a look at the pattern we follow in other actions like setup-node. The implementation doesn't have to look the same, but would love if we can get benefits of caching the downloaded tool (which will matter a lot more when we introduce private runners)

[MSP-Greg]

This may require some changes to the Actions job matrix. From what I can tell, the 'include:' section only adds 'data/inputs/env' to jobs in the existing matrix.

Conversely, in Travis, the 'include:' section adds additional jobs to the matrix, and this is typically where JRuby jobs are added. Maybe another section could be added like 'additional-jobs:' or something...

[eregon]

@damccorm Great, I'll try to find some time to make a PR, thank you for the link.

@MSP-Greg I think that's mostly orthogonal to/independent of this issue, could you file it as a general feature request for GitHub Actions? I've seen both JRuby as part of the matrix (I think that's more frequent, and it ensures everything done on MRI is also tested on JRuby) or separated and then it can be a separate workflow.

[MSP-Greg]

@eregon

There's thread somewhere at https://github.community/t5/GitHub-Actions/bd-p/actions where I added a post about the quirks of the current Actions matrix system. Regardless of how it's done, adding Ruby builds other than MRI to Actions would be helpful...

[ioquatix]

I'm pretty keen to try this out. However, I do need the ability to specify custom environment variables for different versions of Ruby. Not sure what is the best way to do that.

Here is my current workflow: https://github.com/socketry/async/blob/master/.github/workflows/tests.yml

Here is my travis configuration: https://github.com/socketry/async/blob/master/.travis.yml

Also, do you think it would make sense for setup-ruby to also include bundler by default?

[eregon]

@ioquatix You can already try #28
And here is an example how to use it:
https://github.com/eregon/setup-ruby-test/blob/master/.github/workflows/ci.yml
Although the next TruffleRuby release is going to make the setup much nicer.

The docs for matrix are at https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix
Maybe this is possible with exclude:?

[eregon]

https://github.community/t5/GitHub-Actions/GitHub-Actions-Matrix-options-dont-work-as-documented/m-p/29558 is the discussion @MSP-Greg was mentioning above.

[bryanmacfarlane]

covered in this dupe: #44 (comment)