Mirror ruby/yarp into ruby/ruby

[jemmaissroff]

This PR accomplishes the following:

• Add yarp to the tool/sync_default_gems.rb sync script
• Adds yarp files to common.mk
• Does an initial sync of all files in YARP

[k0kubun]

The changes look good to me.

Could you include [Feature #19741] in the message of one (or all) of the commits? We could squash them and add it there, but I assume you prefer keeping the commit history.

This PR accomplishes the following

In terms of features added to CRuby, are the following features what this PR is adding?

• yarp standard library (not gemified)
  • yarp.rb: for require "yarp", YARP's Ruby API
  • yarp.so: for require "yarp.so", the actual parser implementation used by yarp.rb internally

Are the both files upgradable by installing yarp.gem? My general understanding is that it would require the standard library to be gemified, but I wonder there's any trick that you used inside ruby_init_ext or somewhere.

---

Also, since it's not clarified in https://bugs.ruby-lang.org/issues/19741, could we clarify the next steps that you're going to work on before Ruby 3.3 release? In particular, I'm interested in whether you plan to do the following stuff or not:

• Auto-generate files in the Ruby build process instead of on sync_default_gems
• Add a way to optionally switch the parser of the interpreter from parse.y to YARP
• Provide C API in the interpreter that can be used by other C extension gems
  • Or is YARP supposed to be used as a C code in third-party extensions?

(I'm not requesting them for 3.3 but just curious about the timeline)

[jemmaissroff]

In terms of features added to CRuby, are the following features what this PR is adding?

• yarp standard library (not gemified)

  • yarp.rb: for require "yarp", YARP's Ruby API
  • yarp.so: for require "yarp.so", the actual parser implementation used by yarp.rb internally

Yes!

Are the both files upgradable by installing yarp.gem? My general understanding is that it would require the standard library to be gemified, but I wonder there's any trick that you used inside ruby_init_ext or somewhere.

We haven't looked into this in detail yet, but we'd like to enable upgrading of yarp.gem independent of the Ruby release schedule. We will figure this out before the Ruby 3.3 release.

Also, since it's not clarified in https://bugs.ruby-lang.org/issues/19741, could we clarify the next steps that you're going to work on before Ruby 3.3 release? In particular, I'm interested in whether you plan to do the following stuff or not:

• Auto-generate files in the Ruby build process instead of on sync_default_gems

Yes, we will do this after merging this PR!

• Add a way to optionally switch the parser of the interpreter from parse.y to YARP

Yes, this will require more work to integrate YARP's AST with the compiler. I plan to work on this after merging this PR.

• Provide C API in the interpreter that can be used by other C extension gems
  • Or is YARP supposed to be used as a C code in third-party extensions?

We haven't made a decision about this yet.