v2.0.0

Highlights

• Significant performance and memory usage improvements.
• Precompiled platform-specific variants on rubygems.org.
• Many documentation improvements, including a new playground.
• Prism now ships accurate and complete RBS and RBI definitions.
• The license is now owned by the Ruby developers.
• Prism now ships an official C++ interface.

Added

• Support for parsing Ruby 4.1 syntax:
  • Add support for &nil to declare that a method accepts no block.
  • Thoroughly implement void value expression check.
  • Allow trailing commas in method signatures like def foo(a,); end.
  • Reject END { break } and END { next }.
  • a /b/ no longer warns about ambiguous /.
• version: "nearest" is now supported. It behaves similarly to version: "current" but will not raise when the version is unsupported, instead falling back to the nearest syntax version prism currently supports.
• raise_error was added as a valid option. Prism will raise a SyntaxError when given syntax-invalid code. When true is given, prism decides the formatting (tty?, NO_COLOR, etc.). If you need a specific style, :plain, :style, and :color are also supported values.
• Added Prism::ParseResult#continuable to check if the parsed source can become valid by appending more code.
• Prism.find takes a Method, UnboundMethod, Proc, or Thread::Backtrace::Location and returns the associated prism AST node.
• ForwardingSuperNode now has keyword_loc.
• Allow pattern matching with prism nodes to match on location slices, like node in Prism::StringNode[content: "foo"].
• Prism.parse_file and similar now accept anything responding to to_path (like Pathname) directly.
• attach_comments was added as a valid option. Each comment is attached to the node that ends latest on the comment's line before it, otherwise to the innermost node that contains it (or to that node's statements, if only whitespace and comments come between the comment and the statements), otherwise to the top-level statements. The attached comments are available through Node#comments and through pm_node_comments_size and pm_node_comment in the C API, as well as pm_parser_comment_attachment, pm_parser_comment_ranges_size, and pm_parser_comment_range, which expose the attachments grouped by node. When the option is set, the comments on the parse result are sorted by start offset.
• StringQuery#keyword? was added to check if the string query matches a keyword.
• Source#checksum was added to retrieve the checksum of the source content.

Changed

Note that for this major version, almost every change listed below is breaking.

• Prism now defaults to version: "nearest" instead of version: "latest", so code is parsed with the syntax of the running Ruby (or the closest version prism supports). Pass version: "latest" to keep the previous behavior.
• MissingNode has been replaced by ErrorRecoveryNode. ErrorRecoveryNode wraps unexpected nodes, so when walking syntax-invalid code you will now only need to handle this one specific node type.
• Some fields were renamed for consistency. Previous names are aliased and will continue to work in the Ruby layer.
  • InNode#{in,then}_loc -> InNode#{in,then}_keyword_loc
  • MatchPredicateNode#operator_loc -> MatchPredicateNode#keyword_loc
  • UnlessNode#keyword_loc -> UnlessNode#unless_keyword_loc
  • UntilNode#{keyword,closing}_loc -> UntilNode#{until,end}_keyword_loc
  • WhenNode#keyword_loc -> WhenNode#when_keyword_loc
  • WhileNode#{keyword,closing}_loc -> WhileNode#{while,end}_keyword_loc
• Optional locations like StringNode#closing_loc in 'foo that were empty for syntax-invalid code will now be nil instead.
• Prism.lex now returns an array of tokens instead of an array of [token, internal_parse_state]. Previous access like token[0] or token.first to extract the actual token will continue to work. Additionally the following tokens were changed:
  • PARENTHESIS_LEFT was split up into PARENTHESIS_LEFT and PARENTHESIS_LEFT_GROUPING.
  • BRACE_LEFT was split up into BRACE_LEFT, BRACE_LEFT_ARGUMENT, and BRACE_LEFT_HASH.
  • BACKTICK was split up into BACKTICK and XSTRING_BACKTICK.
  • NEWLINE was split up into NEWLINE and NEWLINE_TERMINATOR.
  • WORDS_SEP was split up into WORDS_SEP and WORDS_SEP_IMPLICIT.

Ruby

• Require arguments for Prism::Source.for. The second and third positional parameters are no longer optional.
• Comments now live on the source. Prism::Source#comments returns them and Result#comments delegates to it, so ParseResult.new, LexResult.new, and ParseLexResult.new no longer take a comments argument.
• Comment attachment now happens during parsing with the attach_comments option instead of afterward with ParseResult#attach_comments!, which was removed. Node#comments now returns the comments attached to the node by that option. Location#comments, Location#leading_comments, Location#trailing_comments, Node#leading_comments, Node#trailing_comments, and the comment fields of Prism::Relocation were removed.

C

• Most structs are now opaque (except for nodes), and many internal functions have been hidden.
• Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
• Most allocations are now performed through an arena.
• You may now build side tables of comment attachments through the attach_comments option.
• newline_list was renamed to list_offset_list.
• Visitor callbacks passed to pm_visit_node and pm_visit_child_nodes now return a pm_visit_result_t instead of a bool. Returning PM_VISIT_DESCEND or PM_VISIT_SKIP replaces returning true or false, and returning PM_VISIT_STOP ends the walk. Both functions now return false if the walk was stopped.
• pm_string_t is now a view of bytes owned by someone else. It no longer has a type field, pm_string_owned_init was removed, and pm_string_constant_init was renamed to pm_string_init, which takes the bytes as const uint8_t *.
• pm_options_free no longer frees the names of scope locals. The parser copies them when it is created, so they only need to remain valid until then.

Rust

• Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
• Most allocations are now performed through an arena.
• newline_list was renamed to list_offset_list.
• Diagnostic::message now returns &[u8] instead of &str.
• StringQuery is now exposed.

JavaScript

• The parse method now only accepts Uint8Array, instead of forcing a transcoding to UTF-16.

Removed

• Removed Prism::Pack.