Cross-version compatibility and support

[dgollahon]

How would yarp handle the various versions of ruby? So, for instance, in rubocop you can set your target ruby version and it will only apply rules that matter to that ruby version. For example, targetting 2.7 and running rubocop (even though my local ruby version is 3.2) will give you:

tmp.rb:1:15: E: Lint/Syntax: unexpected token tEQL
(Using Ruby 2.7 parser; configure using TargetRubyVersion parameter, under AllCops)
def square(x) = x**2

This is often different from the actual version of ruby that you are running. I assume other tools might have similar/related needs and also likely need easy ways of recognizing and tolerating different versions of the ASTs if they want to support multiple versions. I know that there will be versioned metadata based on the design document, but was curious about how this will be managed. See logic from rubocop-ast for an example of how parser works.

[mbj]

I'd like to also add tools like mutant need to support a wider range of ruby runtime versions. I'd not need 2.7 support as its as of this post EOL, but ideally YARP would support all non EOL versions from the same release.

[enebo]

I had opened an issue earlier about packaging gem by version (e.g. 3.2 support could be 32.0 -- just a proposal) and I think the result is we would address it later on when we got closer to being usable. For JRuby (and CRuby), we just need one version per release line but will have to support multiple language levels in parallel (3.1, 3.2, ...). This is tangential but I am just mentioning it because it is another use case of compatibilty and support (albeit a pretty simple one).

My understanding is that YARP is going to start at 3.2 support. Obviously, to support your use cases there still needs to be a strategy but I think the oldest YARP version will be 3.2.

[mbj]

Yeah, so as soon 3.3 changes (usually additions) are around the corner: I'd love if YARP could keep 3.2 support.

For mutant, I'll likely be able to use YARP For the version we have support, and keep using the parser gem for older.

[eregon]

I think part of the initial thoughts from @kddnewton about this is newer Rubies simply accept more syntax than older versions, so to some extent it might be enough (?) to just use the latest version of the parser.
For RuboCop for instance, what would be the harm in doing so? Why is a SyntaxError useful there? Such error in syntax would be found when running tests anyway, it's not really RuboCop's job to validate the files are compatible with Ruby x.y (nor can it do so fully, that needs running the tests not just checking syntax).
But there are probably some edge cases which might not make this practical.
I think this still needs to be figured out, as @enebo said.

[kddnewton]

I'm imagining this will be solved with branches going forward. We're going to release the initial version with Ruby 3.3. At that point we'll create a tag (just like CRuby will) and backport issues as necessary. If you want to only parse Ruby 3.3, then you'll be able to link back to the shared library for version 3.3.

[enebo]

Can a gem (e.g. rubocop-ast) rely on different versions of the same gem? An idea could be yarp as a gem which is always latest official C Ruby version but also release yarp-3.2 gem for specific 3.2 support (yarp-3.4, ... yarp-n.m). This would involve always releasing twice but it would allow a pretty simple path for supporting multiple version.

[kddnewton]

We could do that. My thinking was more that you would never want to parse two versions of Ruby at the same time. So you would just have the correct version of YARP in your bundle, and then rubocop would use that.

[enebo]

Someone who uses rubocop should weigh in on this. I see you can specify which C Ruby version of cops to run in the rubocop .yml file. If that also means adding an explicit line to Gemfile to match that then perhaps that is acceptable? It sort of feels like an extra step but with that said having rubocop install n versions of YARP also feels strange.

[dgollahon]

I think part of the initial thoughts from @kddnewton about this is newer Rubies simply accept more syntax than older versions, so to some extent it might be enough (?) to just use the latest version of the parser.

Coming back to this now that ruby 3.3 is out: 3.3 restricted multiple syntax cases, see this example. That means that ruby code running on < 3.3 which is perfectly valid couldn't be parsed using the 3.3.x parser. I think there will probably need to be some kind of versioning mechanism for rubocop, mutant, and other similar tools that want to be able to run on multiple versions of ruby.

[kddnewton]

Yeah we haven't solved this yet, but there is a new version flag that you can pass in that will restrict the syntax that it parses to a particular version of Ruby. Right now the only valid values are 3.3.0 and latest. This change will currently only impact whether or not it is allowed.