Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: sorbet/sorbet
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: Shopify/sorbet
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: prism-squashed
Choose a head ref
Can’t automatically merge. Don’t worry, you can still create the pull request.

Commits on Feb 18, 2025

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    14cd050 View commit details
  2. Implement Prism-based parser

    Initial attempt at Prism to Sorbet translation layer
    
    Create and run Prism benchmarking script
    
    Translate more node types in `pipeline.cc`
    
    Wrap Prism C API into new `Prism::Parser` class
    
    And move all other Prism related code to the new parser/prism component
    
    Translate more node types
    
    Introduce memory comparison script for Prism and Sorbet parsers
    
    Translate more node types
    
    Write memory comparison script for Prism and Sorbet parser
    
    Translate more node types
    
    Add VS Code tasks for common operations
    
    Use `setup-bazel` to cache Bazel in CI
    
    Compare location data between Prism and Sorbet parsers
    
    To make sure our translation layer correctly translates node locations, we create a task that, for every Prism regression test, runs the test with location comparison on.
    
    Merge location and regression testing tasks
    
    Translate more node types
    
    CI improvements
    
    Translate more node types
    
    Use a `repository_rule` to get the RUBY_ROOT env var
    
    Up to this point, in order to build Prism, we've needed to manually pass the path to the correct RUBY version to the build process because Bazel does not take the user's environment into account and will just use the system's default Ruby version (which is often very old).
    
    This is a workaround for our workaround, where we use a Bazel `repository_rule` to get the `RUBY_ROOT` environment variable, write it to a file within the Bazel file system, and then pull the value from that file while building Prism.
    
    Fix output of Prism location test errors
    
    Co-Authored-By: Vinicius Stock <vinicius.stock@shopify.com>
    Co-Authored-By: Alex Momchilov <alexander.momchilov@shopify.com>
    Co-Authored-By: Emily Samp <emily.samp@shopify.com>
    Co-Authored-By: Alexander Momchilov <amomchilov@users.noreply.github.com>
    4 people authored and st0012 committed Feb 18, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    b9cf0f2 View commit details
  3. Extract up_cast helper

    amomchilov authored and st0012 committed Feb 18, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    6991f89 View commit details
  4. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    679d5a1 View commit details
  5. Add launch.json config to debug Sorbet with Prism

    This debug config launches the main Sorbet executable to type check
    the provided file path (with Prism).
    
    Add vscode-lldb to recommended extensions list
    
    Add support for debugging Prism tests in VSCode
    
    Remove unnecessary task to set BAZEL_EXEC_ROOT
    
    It turns out that we only need `sourceMap`'s value to be set to the
    workspace folder with any arbitrary key. So not setting `BAZEL_EXEC_ROOT`
    is fine.
    st0012 committed Feb 18, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    82f5288 View commit details
  6. Fix edge case for lambda with no params

    egiurleo authored and st0012 committed Feb 18, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    4b19f96 View commit details
  7. nullptr check during ENFORCE in downcast

    In our `downcast` helper, we check that the node argument is actually
    a Prism node. However, doing this requires us to check the `type` field,
    which will fail if the node is a `nullptr`. By doing a `nullptr`
    check in the `ENFORCE`, we guarantee we won't crash by checking for
    a field on a `nullptr`.
    
    Co-authored-by: Alex Momchilov <alexander.momchilov@shopify.com>
    2 people authored and st0012 committed Feb 18, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    8006ffb View commit details
  8. Fix bugs in translation of PM_RESCUE_NODE

    Only do dynamic constant assignment workaround in specific cases
    
    Only do the dynamic constant assignment workaround when translating
    a constant from a `PM_CONSTANT_WRITE_NODE` and
    `PM_CONSTANT_PATH_NODE`.
    
    Test dynamic constant workaround for all constant assignment types
    
    Add more test cases to rescue prism regression tests
    
    Handle all constant path assignment node types
    
    Shopify#317
    egiurleo authored and st0012 committed Feb 18, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    0f67c16 View commit details
  9. Report Prism parse errors

    Stop surfacing Prism's EOF error
    
    It usually appears when there's an unclosed statements/assignments, where
    the root error will also be surfaced. So it's not exactly important to
    surface this error.
    
    But more importantly, because Prism puts this error at the very last line
    of the file when there's a missing `end`, it can't be represented in
    Sorbet's error comments.
    
    Keep Prism error id's type for easy comparison
    
    We currently don't use the converted value and keeping the `pm_diagnostic_id_t`
    type makes it easy to compare the value with `PM_ERR_*` constants directly.
    
    Add a task to display Prism's parse errors
    
    Address feedback
    
    Change location test's naming convention
    
    By not replacing `/` with `_`, the test naming structure is more consistent
    with normal regression tests. But more importantly, it allows us to run
    location tests that are under a folder like `error_recovery/assign`, more
    easily.
    
    Generate error for Sorbet's `assign.rb` error recovery fixture
    
    The error location doesn't match what Sorbet currently generates. But the
    idea is to first make parse trees match, accumulate more recovery tests,
    and then come up with a strategy to generate errors that match locations.
    
    Shopify#318
    Shopify#319
    st0012 committed Feb 18, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    4806905 View commit details
  10. Run Corpus tests with Prism

    To make it clearer that these are the regression tests we've built,
    as opposed to pre-existing Sorbet tests
    
    Add test corpus with Prism
    
    The purpose of these tests is to run the Sorbet tests that already exist,
    but to use the Prism parser instead of the Sorbet parser. As we fix
    more edge cases, we will include more of the test suite and run them
    in CI.
    
    Add tasks to run corpus tests with Prism
    
    Update the test target on CI
    
    Add Prism handling for packager tests too
    
    Because pipeline_tests handles packager tests differently, we need to add
    Prism handling for them too. Otherwise, we'll get test conflict errors
    when `test_corpus_prism`'s pattern covers packager tests.
    
    Follow up on #323
    
    - Fix the command of single Prism regression test task
    - Update launch.json with the new task name
    
    Fix `launch.json` for prism regression tests
    
    Make test Pipeline parse with Prism in the case of `indexOne`
    
    Seperate test input for regression and corpus tests
    
    Remove error recovery tests from prism test corpus
    
    And start running corpus tests in CI
    
    Co-Authored-By: Emily Samp <emily.samp@shopify.com>
    egiurleo authored and st0012 committed Feb 18, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    a267bfa View commit details
  11. Fix float translation

    st0012 committed Feb 18, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    91f54de View commit details
  12. Fix integer translation

    st0012 committed Feb 18, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    5aaf5c9 View commit details
  13. Translator should not treat ~[Integer] as a method call

    Sorbet's legacy parser treats `~[Integer]` and `-[Integer]` as integer literals,
    but Prism treats `~[Integer]` as a method call.
    
    (Technically, both of them are method calls.)
    
    So in the translator, we need to give `~` special handling when its receiver is an integer.
    We don't need the same for Float or Numeric because only Integer has the `~` method.
    st0012 committed Feb 18, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    c6bb135 View commit details
  14. Properly translate complex literals with prefix - or +

    Unlike Integer or Float, Complex literals with prefix `-` or `+` should
    be translated to method calls on the Complex literal.
    st0012 committed Feb 18, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    24d5c1d View commit details
  15. Copy the full SHA
    413da3c View commit details
  16. Copy the full SHA
    a9dff1d View commit details
  17. Copy the full SHA
    3459680 View commit details
  18. Persist unique counter value between translator instances

    When we create a new translator instance, we haven't been maintaining
    the value of the unique counter, which keeps track of anonymous arguments
    in order to give them unique names. This was resulting in different parse
    trees for Sorbet and Prism.
    egiurleo authored and st0012 committed Feb 18, 2025
    Copy the full SHA
    7395cc3 View commit details
  19. Add desugar tests to Prism corpus

    Exclude any that are failing because of error recovery or
    legitimate errors.
    egiurleo authored and st0012 committed Feb 18, 2025
    Copy the full SHA
    511a563 View commit details
  20. Make LHS of conditional call operator assignment CSend

    Remove desugar/csend test from exclusions
    egiurleo authored and st0012 committed Feb 18, 2025
    Copy the full SHA
    548f088 View commit details
  21. Improve handling of uniqueCounter in Prism translator

    `uniqueCounter` is a counter that is used to give unique ids to anonymous arguments
    while parsing them in Sorbet. Thus far, we have coordinated the `uniqueCounter` between
    multiple Translators by passing the value back and forth, but this approach is brittle.
    
    With this new approach, we can maintain the same `uniqueCounter` between multiple
    translators:
    
    1. The parent translator maintains the source-of-truth counter in a variable called
      `uniqueCounterStorage`
    2. It also maintains a pointer to that value called `uniqueCounter`
    3. Child translators are created with a dummy value in `uniqueCounterStorage` that
      will never be used, as well as the `uniqueCounter` pointer that points to their
      parent translator's storage
    4. Incrementing the `uniqueCounter` happens via the pointer, which will always
      point to the parent's `uniqueCounter` value
    
    Co-authored-by: Alexander Momchilov <alexander.momchilov@shopify.com>
    2 people authored and st0012 committed Feb 18, 2025
    Copy the full SHA
    8d5c96e View commit details
  22. Translate def nodes with receivers as DefS

    Add prism regression test for singleton method defs
    
    Add more newly-passing tests to Prism corpus
    
    Add `desugar/defs_not_self` to Prism corpus tests
    egiurleo authored and st0012 committed Feb 18, 2025
    Copy the full SHA
    ddb7168 View commit details
  23. Standardize handling of block arguments in Prism -> Sorbet translation

    Handle forwarding super node with block argument
    
    Correctly handle calls to `super` with block
    
    Change argument name to `blockArgumentNode`
    
    Refactor `translateArguments` to remove unused parameter
    
    And make block parameter optional
    
    Refactor index operator writes to use refactored arguments logic
    
    Refactor `PM_CALL_NODE` case to use refactored arguments logic
    
    Refactor `PM_INDEX_TARGET_NODE` case to use refactored arguments logic
    
    Refactor `PM_SUPER_NODE` case to use refactored arguments logic
    
    Refactor `translateArguments` to handle block argument as well
    
    Add a `super` test case to `def_block_param` regression test
    
    Pass block arguments to super calls
    
    Add case to `def_singleton` Prism regression test
    
    Testing the translation of a call to super that passes a block argument.
    egiurleo authored and st0012 committed Feb 18, 2025
    Copy the full SHA
    058e653 View commit details
  24. Fix splat arg translation in Prism

    When the splat arg's expression is an `Arg`, we need to translate it to a `Restarg`
    instead of a `SplatLhs`.
    st0012 committed Feb 18, 2025
    Copy the full SHA
    bd62043 View commit details
  25. Handle implicit rest nodes in array patterns

    1. We don't need to translate implicit rest nodes in array patterns
    2. We need to return an `ArrayPatternWithTail` instead of an `ArrayPattern`
       when the pattern ends with an implicit rest node
    st0012 committed Feb 18, 2025
    Copy the full SHA
    039adb0 View commit details
  26. Copy the full SHA
    4b230df View commit details
  27. Skip error message checking when running Corpus tests with Prism

    Currently, corpus tests may fail with Prism for a few reasons:
    
    1. The error checking fails with Prism. This is expected.
    2. There are tree mismatches.
    
    And sometimes, a test fail because of both 1 and 2.
    
    This means that by skipping the test altogether, we are potentially skipping
    legitimately failing tests.
    
    This commit updates the test runner to skip the error message checking when
    running corpus tests with Prism and updates the skipped tests in the BUILD file.
    
    So now if a test is skipped, it's only because of the second reason. And
    we should minimize the number of tests that are skipped in that case.
    
    Co-authored-by: Emily Samp <emily.samp@shopify.com>
    st0012 and egiurleo committed Feb 18, 2025
    Copy the full SHA
    9feb9fb View commit details
  28. Fix dynamic constant handling

    Rather than specifying whether to *skip* the logic, we should
    say whether to replace the constant node with a dynamic const assignment.
    
    Then, remove unnecessary type check and move this logic to
    the top of the method so we short-circuit any unnecessary
    code in that case.
    
    Run two more tests that are only failing because of error differences
    
    Remove `desugar/regex` from failing tests
    
    Remove `desugar/range` from excluded tests
    
    Co-Authored-By: Stan Lo <stan.lo@shopify.com>
    egiurleo and st0012 committed Feb 18, 2025
    Copy the full SHA
    de154bc View commit details
  29. Improve pattern matching support with hash patterns

    1. It fixes the case where a hash pattern doesn't have a value, which will
       generate a `PM_IMPLICIT_NODE` that we currently don't handle.
    2. It fixes the translation of hash patterns with a named splat node, like
       `in **o`.
    st0012 committed Feb 18, 2025
    Copy the full SHA
    07e70be View commit details
  30. Copy the full SHA
    7cb802e View commit details
  31. Copy the full SHA
    c51836c View commit details
  32. Introduce isKeywordHashElement method to fix def_delegator test

    Run `rewriter/def_delegator` test as part of prism corpus
    
    Remove `rewriter/rails/delegate` test from skipped rewriter tests
    
    Refactor `translateHash` into `translateKeyValuePairs`
    
    The logic of `translateHash` was kind of confusing because it was handling
    both vanilla hashes and also assocs/assoc splat nodes. By isolating one
    part of that method, which generates key/value pairs, and then calling
    that method in both cases, the two code paths become cleaner and more
    obvious.
    egiurleo authored and st0012 committed Feb 18, 2025
    Copy the full SHA
    5fd9a35 View commit details
  33. Copy the full SHA
    69f69c2 View commit details
  34. Fix bracket method call's location translation

    When the method call is `[]`, the message location should be just the begin
    location of `[]`.
    st0012 committed Feb 18, 2025
    Copy the full SHA
    efa115a View commit details
  35. Copy the full SHA
    26ce359 View commit details
  36. Handle translation of post params

    Add `parser/kwnil_errors` to skipped tests with a note
    
    Run `local_vars/z_super_args_splats_blocks` test in Prism corpus
    egiurleo authored and st0012 committed Feb 18, 2025
    Copy the full SHA
    379e603 View commit details
  37. Copy the full SHA
    b31dd3d View commit details
  38. Copy the full SHA
    3cc25ae View commit details
  39. Add namer tests to prism corpus

    And exclude failing tests
    egiurleo authored and st0012 committed Feb 18, 2025
    Copy the full SHA
    220ec4f View commit details
  40. Modify locations of kw arguments and params to match legacy parser

    Add `namer/arguments` test to Prism corpus
    
    Run two more namer tests that now pass
    egiurleo authored and st0012 committed Feb 18, 2025
    Copy the full SHA
    96babd5 View commit details
  41. Fix call with empty message's location translation

    In cases like `foo.()`, the message would be `:call`, but the message location
    would have null start and end. In this case, we need to fall back to use
    the call operator location.
    
    Include LSP tests in Prism Corpus tests
    st0012 committed Feb 18, 2025
    Copy the full SHA
    b026c49 View commit details
  42. Correctly translate constant write nodes' location

    This is not covered in parse tree location tests, but reflected in symbol
    table tests: when we write `FOO = 1` (or other constant write nodes), the
    location of `FOO` should just be the name, not the entire assignment.
    
    In those cases, we should use `name_loc` instead of `base.location`.
    st0012 committed Feb 18, 2025
    Copy the full SHA
    1b2de19 View commit details
  43. Fix symbol location for hash, keyword argument keys

    When the symbol is used as `a: 1`, the location should not include the colon.
    
    Run Prism Corpus against all Sorbet component tests
    st0012 committed Feb 18, 2025
    Copy the full SHA
    c5d2dc7 View commit details
  44. Use libprism to build Prism

    Use improved libprism source to build Prism without dependency on Ruby
    
    The main blocker for upstreaming Prism parser to Sorbet is that we currently
    rely on Ruby to build Prism, as its required by Prism's `rake template` task.
    
    But by package the files generated by `rake template` in Prim's release,
    we will be able to remove the dependency on Ruby while significantly simplify
    the build configurations for Prism.
    
    Use another mock source
    st0012 committed Feb 18, 2025
    Copy the full SHA
    cb10db7 View commit details
  45. Add formatter for Prism node types

    When ENFORCE macros fail, they use fmt (via spdlog) to format error messages.
    Add a formatter specialization for pm_node_type to properly display Prism node
    types in these error messages. Instead of showing numeric values, the formatter
    uses pm_node_type_to_str to display human-readable node type names.
    egiurleo authored and st0012 committed Feb 18, 2025
    Copy the full SHA
    fdbf9b6 View commit details
  46. Copy the full SHA
    c1f8f1b View commit details
  47. Copy the full SHA
    8266cdb View commit details
  48. Add sliceLocation helper to simplify location operations (#406)

    * Add sliceLocation helper to simplify location operations
    
    * Run CI on prism-squahed too
    st0012 committed Feb 18, 2025
    Copy the full SHA
    55a42d4 View commit details
  49. Always raise in unreachable code (#407)

    Given that some problem may only be found when running a release build
    against a large codebase, raising in unreachable code makes sure we
    spot the issues at its source instead of crashing later downstream.
    
    Example: https://github.com/sorbet/sorbet/pull/8485/files#r1943210573
    st0012 committed Feb 18, 2025
    Copy the full SHA
    9c27038 View commit details
  50. Match Sorbet's latest support for implicit rest nodes

    Reference: #8549
    st0012 committed Feb 18, 2025
    Copy the full SHA
    fc831c6 View commit details
Showing with 15,027 additions and 22 deletions.
  1. +34 −0 .github/workflows/ci.yml
  2. +3 −1 .vscode/extensions.json
  3. +38 −1 .vscode/launch.json
  4. +136 −0 .vscode/tasks.json
  5. +1 −0 Gemfile
  6. +12 −0 Gemfile.lock
  7. 0 break.rb
  8. +21 −0 main/options/options.cc
  9. +16 −0 main/options/options.h
  10. +2 −0 main/pipeline/BUILD
  11. +52 −2 main/pipeline/pipeline.cc
  12. +3 −0 main/pipeline/pipeline.h
  13. +22 −0 parser/prism/BUILD
  14. +202 −0 parser/prism/Helpers.h
  15. +49 −0 parser/prism/Parser.cc
  16. +103 −0 parser/prism/Parser.h
  17. +1,982 −0 parser/prism/Translator.cc
  18. +110 −0 parser/prism/Translator.h
  19. +49 −0 prism_benchmarks/memory/data/parser/shopify/2024-08-22-0739d1462.txt
  20. +49 −0 prism_benchmarks/memory/data/parser/shopify/2024-09-18-5df44846d.txt
  21. +48 −0 prism_benchmarks/memory/data/parser/yjit-bench/2024-08-22-0739d1462.txt
  22. +48 −0 prism_benchmarks/memory/data/parser/yjit-bench/2024-09-18-5df44846d.txt
  23. +50 −0 prism_benchmarks/memory/data/pipeline/prism_regression/2024-08-22-0739d1462.txt
  24. +50 −0 prism_benchmarks/memory/data/pipeline/prism_regression/2024-09-18-5df44846d.txt
  25. +49 −0 prism_benchmarks/memory/data/pipeline/rbi/2024-09-18-5df44846d.txt
  26. +67 −0 prism_benchmarks/memory/run_benchmarks.sh
  27. +78 −0 prism_benchmarks/time/data/parser/sorbet-tests/2024-07-18-f75db5824.json
  28. +78 −0 prism_benchmarks/time/data/parser/sorbet-tests/2024-09-18-fe802b67a.json
  29. +78 −0 prism_benchmarks/time/data/parser/yjit-bench/2024-07-18-f75db5824.json
  30. +78 −0 prism_benchmarks/time/data/parser/yjit-bench/2024-09-18-fe802b67a.json
  31. +78 −0 prism_benchmarks/time/data/pipeline/prism_regression/2024-07-18-f75db5824.json
  32. +78 −0 prism_benchmarks/time/data/pipeline/prism_regression/2024-09-18-fe802b67a.json
  33. +78 −0 prism_benchmarks/time/data/pipeline/rbi/2024-09-18-5df44846d.json
  34. +60 −0 prism_benchmarks/time/run_benchmarks.sh
  35. +78 −0 test/BUILD
  36. +15 −0 test/helpers/position_assertions.cc
  37. +4 −0 test/helpers/position_assertions.h
  38. +18 −2 test/pipeline_test.bzl
  39. +62 −15 test/pipeline_test_runner.cc
  40. +14 −0 test/prism_location_test.bzl
  41. +42 −0 test/prism_location_test.sh
  42. +28 −0 test/prism_regression/alias.parse-tree.exp
  43. +10 −0 test/prism_regression/alias.rb
  44. +140 −0 test/prism_regression/assign_to_class_variable.parse-tree.exp
  45. +23 −0 test/prism_regression/assign_to_class_variable.rb
  46. +691 −0 test/prism_regression/assign_to_constant.parse-tree.exp
  47. +96 −0 test/prism_regression/assign_to_constant.rb
  48. +140 −0 test/prism_regression/assign_to_global_variable.parse-tree.exp
  49. +23 −0 test/prism_regression/assign_to_global_variable.rb
  50. +713 −0 test/prism_regression/assign_to_index.parse-tree.exp
  51. +45 −0 test/prism_regression/assign_to_index.rb
  52. +140 −0 test/prism_regression/assign_to_instance_variable.parse-tree.exp
  53. +23 −0 test/prism_regression/assign_to_instance_variable.rb
  54. +140 −0 test/prism_regression/assign_to_local_variable.parse-tree.exp
  55. +23 −0 test/prism_regression/assign_to_local_variable.rb
  56. +228 −0 test/prism_regression/assign_to_method_result.parse-tree.exp
  57. +24 −0 test/prism_regression/assign_to_method_result.rb
  58. +59 −0 test/prism_regression/assoc_splat.parse-tree.exp
  59. +7 −0 test/prism_regression/assoc_splat.rb
  60. +57 −0 test/prism_regression/back_reference_read.parse-tree.exp
  61. +18 −0 test/prism_regression/back_reference_read.rb
  62. +21 −0 test/prism_regression/begin.parse-tree.exp
  63. +10 −0 test/prism_regression/begin.rb
  64. +26 −0 test/prism_regression/break.parse-tree.exp
  65. +11 −0 test/prism_regression/break.rb
  66. +58 −0 test/prism_regression/call.parse-tree.exp
  67. +7 −0 test/prism_regression/call.rb
  68. +237 −0 test/prism_regression/call_all_params.parse-tree.exp
  69. +9 −0 test/prism_regression/call_all_params.rb
  70. +296 −0 test/prism_regression/call_block_param.parse-tree.exp
  71. +45 −0 test/prism_regression/call_block_param.rb
  72. +17 −0 test/prism_regression/call_forwarding_param.parse-tree.exp
  73. +6 −0 test/prism_regression/call_forwarding_param.rb
  74. +53 −0 test/prism_regression/call_kw_rest_params.parse-tree.exp
  75. +9 −0 test/prism_regression/call_kw_rest_params.rb
  76. +96 −0 test/prism_regression/call_required_kw_params.parse-tree.exp
  77. +9 −0 test/prism_regression/call_required_kw_params.rb
  78. +46 −0 test/prism_regression/call_required_params.parse-tree.exp
  79. +9 −0 test/prism_regression/call_required_params.rb
  80. +43 −0 test/prism_regression/call_rest_params.parse-tree.exp
  81. +10 −0 test/prism_regression/call_rest_params.rb
  82. +48 −0 test/prism_regression/call_with_receiver.parse-tree.exp
  83. +7 −0 test/prism_regression/call_with_receiver.rb
  84. +93 −0 test/prism_regression/case.parse-tree.exp
  85. +17 −0 test/prism_regression/case.rb
  86. +971 −0 test/prism_regression/case_match.parse-tree.exp
  87. +83 −0 test/prism_regression/case_match.rb
  88. +284 −0 test/prism_regression/case_match_variable_binding.parse-tree.exp
  89. +26 −0 test/prism_regression/case_match_variable_binding.rb
  90. +110 −0 test/prism_regression/case_match_variable_pinning.parse-tree.exp
  91. +23 −0 test/prism_regression/case_match_variable_pinning.rb
  92. +59 −0 test/prism_regression/class.parse-tree.exp
  93. +17 −0 test/prism_regression/class.rb
  94. +28 −0 test/prism_regression/constants.parse-tree.exp
  95. +13 −0 test/prism_regression/constants.rb
  96. +45 −0 test/prism_regression/def.parse-tree.exp
  97. +17 −0 test/prism_regression/def.rb
  98. +56 −0 test/prism_regression/def_all_params.parse-tree.exp
  99. +5 −0 test/prism_regression/def_all_params.rb
  100. +45 −0 test/prism_regression/def_block_param.parse-tree.exp
  101. +9 −0 test/prism_regression/def_block_param.rb
  102. +10 −0 test/prism_regression/def_forwarding_param.parse-tree.exp
  103. +4 −0 test/prism_regression/def_forwarding_param.rb
  104. +36 −0 test/prism_regression/def_kw_rest_params.parse-tree.exp
  105. +7 −0 test/prism_regression/def_kw_rest_params.rb
  106. +38 −0 test/prism_regression/def_optional_kw_params.parse-tree.exp
  107. +4 −0 test/prism_regression/def_optional_kw_params.rb
  108. +38 −0 test/prism_regression/def_optional_params.parse-tree.exp
  109. +4 −0 test/prism_regression/def_optional_params.rb
  110. +29 −0 test/prism_regression/def_required_kw_params.parse-tree.exp
  111. +4 −0 test/prism_regression/def_required_kw_params.rb
  112. +29 −0 test/prism_regression/def_required_params.parse-tree.exp
  113. +4 −0 test/prism_regression/def_required_params.rb
  114. +43 −0 test/prism_regression/def_rest_params.parse-tree.exp
  115. +8 −0 test/prism_regression/def_rest_params.rb
  116. +43 −0 test/prism_regression/def_singleton.parse-tree.exp
  117. +8 −0 test/prism_regression/def_singleton.rb
  118. +7 −0 test/prism_regression/def_with_body.parse-tree.exp
  119. +5 −0 test/prism_regression/def_with_body.rb
  120. +72 −0 test/prism_regression/defined.parse-tree.exp
  121. +12 −0 test/prism_regression/defined.rb
  122. +133 −0 test/prism_regression/ensure.parse-tree.exp
  123. +53 −0 test/prism_regression/ensure.rb
  124. +19 −0 test/prism_regression/error_recovery/assign.parse-tree.exp
  125. +6 −0 test/prism_regression/error_recovery/assign.rb
  126. +96 −0 test/prism_regression/flip_flop.parse-tree.exp
  127. +26 −0 test/prism_regression/flip_flop.rb
  128. +76 −0 test/prism_regression/float.parse-tree.exp
  129. +26 −0 test/prism_regression/float.rb
  130. +89 −0 test/prism_regression/for_loop.parse-tree.exp
  131. +19 −0 test/prism_regression/for_loop.rb
  132. +26 −0 test/prism_regression/if.parse-tree.exp
  133. +5 −0 test/prism_regression/if.rb
  134. +10 −0 test/prism_regression/if_else.parse-tree.exp
  135. +7 −0 test/prism_regression/if_else.rb
  136. +17 −0 test/prism_regression/if_elsif.parse-tree.exp
  137. +9 −0 test/prism_regression/if_elsif.rb
  138. +8 −0 test/prism_regression/if_empty_else.parse-tree.exp
  139. +6 −0 test/prism_regression/if_empty_else.rb
  140. +8 −0 test/prism_regression/if_with_stmt.parse-tree.exp
  141. +5 −0 test/prism_regression/if_with_stmt.rb
  142. +17 −0 test/prism_regression/if_with_stmts.parse-tree.exp
  143. +7 −0 test/prism_regression/if_with_stmts.rb
  144. +2 −0 test/prism_regression/keyword_ENCODING.parse-tree.exp
  145. +3 −0 test/prism_regression/keyword_ENCODING.rb
  146. +2 −0 test/prism_regression/keyword_FILE.parse-tree.exp
  147. +3 −0 test/prism_regression/keyword_FILE.rb
  148. +2 −0 test/prism_regression/keyword_LINE.parse-tree.exp
  149. +3 −0 test/prism_regression/keyword_LINE.rb
  150. +2 −0 test/prism_regression/keyword_false.parse-tree.exp
  151. +3 −0 test/prism_regression/keyword_false.rb
  152. +49 −0 test/prism_regression/keyword_it.parse-tree.exp
  153. +15 −0 test/prism_regression/keyword_it.rb
  154. +2 −0 test/prism_regression/keyword_nil.parse-tree.exp
  155. +3 −0 test/prism_regression/keyword_nil.rb
  156. +2 −0 test/prism_regression/keyword_redo.parse-tree.exp
  157. +4 −0 test/prism_regression/keyword_redo.rb
  158. +2 −0 test/prism_regression/keyword_retry.parse-tree.exp
  159. +3 −0 test/prism_regression/keyword_retry.rb
  160. +2 −0 test/prism_regression/keyword_self.parse-tree.exp
  161. +3 −0 test/prism_regression/keyword_self.rb
  162. +2 −0 test/prism_regression/keyword_true.parse-tree.exp
  163. +3 −0 test/prism_regression/keyword_true.rb
  164. +222 −0 test/prism_regression/lambda.parse-tree.exp
  165. +24 −0 test/prism_regression/lambda.rb
  166. +115 −0 test/prism_regression/literal_array.parse-tree.exp
  167. +23 −0 test/prism_regression/literal_array.rb
  168. +55 −0 test/prism_regression/literal_complex.parse-tree.exp
  169. +10 −0 test/prism_regression/literal_complex.rb
  170. +138 −0 test/prism_regression/literal_hash.parse-tree.exp
  171. +18 −0 test/prism_regression/literal_hash.rb
  172. +3 −0 test/prism_regression/literal_imaginary.parse-tree.exp
  173. +3 −0 test/prism_regression/literal_imaginary.rb
  174. +109 −0 test/prism_regression/literal_integer.parse-tree.exp
  175. +50 −0 test/prism_regression/literal_integer.rb
  176. +60 −0 test/prism_regression/literal_range.parse-tree.exp
  177. +13 −0 test/prism_regression/literal_range.rb
  178. +10 −0 test/prism_regression/literal_rational.parse-tree.exp
  179. +8 −0 test/prism_regression/literal_rational.rb
  180. +91 −0 test/prism_regression/literal_regexp.parse-tree.exp
  181. +18 −0 test/prism_regression/literal_regexp.rb
  182. +3 −0 test/prism_regression/literal_string.parse-tree.exp
  183. +3 −0 test/prism_regression/literal_string.rb
  184. +81 −0 test/prism_regression/literal_string_interpolation.parse-tree.exp
  185. +11 −0 test/prism_regression/literal_string_interpolation.rb
  186. +44 −0 test/prism_regression/literal_symbol.parse-tree.exp
  187. +15 −0 test/prism_regression/literal_symbol.rb
  188. +31 −0 test/prism_regression/local_variables.parse-tree.exp
  189. +7 −0 test/prism_regression/local_variables.rb
  190. +126 −0 test/prism_regression/match_last_line.parse-tree.exp
  191. +22 −0 test/prism_regression/match_last_line.rb
  192. +46 −0 test/prism_regression/match_write.parse-tree.exp
  193. +12 −0 test/prism_regression/match_write.rb
  194. +19 −0 test/prism_regression/module.parse-tree.exp
  195. +7 −0 test/prism_regression/module.rb
  196. +13 −0 test/prism_regression/multi_statements.parse-tree.exp
  197. +5 −0 test/prism_regression/multi_statements.rb
  198. +341 −0 test/prism_regression/multi_target.parse-tree.exp
  199. +25 −0 test/prism_regression/multi_target.rb
  200. +230 −0 test/prism_regression/multi_write.parse-tree.exp
  201. +24 −0 test/prism_regression/multi_write.rb
  202. +23 −0 test/prism_regression/next.parse-tree.exp
  203. +12 −0 test/prism_regression/next.rb
  204. +65 −0 test/prism_regression/numbered_params.parse-tree.exp
  205. +4 −0 test/prism_regression/numbered_params.rb
  206. +51 −0 test/prism_regression/numbered_reference_read.parse-tree.exp
  207. +12 −0 test/prism_regression/numbered_reference_read.rb
  208. +16 −0 test/prism_regression/operator_and.parse-tree.exp
  209. +5 −0 test/prism_regression/operator_and.rb
  210. +16 −0 test/prism_regression/operator_or.parse-tree.exp
  211. +5 −0 test/prism_regression/operator_or.rb
  212. +34 −0 test/prism_regression/parentheses.parse-tree.exp
  213. +7 −0 test/prism_regression/parentheses.rb
  214. +266 −0 test/prism_regression/pattern_matching_operators.parse-tree.exp
  215. +28 −0 test/prism_regression/pattern_matching_operators.rb
  216. +47 −0 test/prism_regression/post_execution.parse-tree.exp
  217. +20 −0 test/prism_regression/post_execution.rb
  218. +47 −0 test/prism_regression/pre_execution.parse-tree.exp
  219. +20 −0 test/prism_regression/pre_execution.rb
  220. +67 −0 test/prism_regression/range.parse-tree.exp
  221. +11 −0 test/prism_regression/range.rb
  222. +550 −0 test/prism_regression/rescue.parse-tree.exp
  223. +109 −0 test/prism_regression/rescue.rb
  224. +25 −0 test/prism_regression/return.parse-tree.exp
  225. +7 −0 test/prism_regression/return.rb
  226. +19 −0 test/prism_regression/shareable_constant.parse-tree.exp
  227. +4 −0 test/prism_regression/shareable_constant.rb
  228. +47 −0 test/prism_regression/super.parse-tree.exp
  229. +19 −0 test/prism_regression/super.rb
  230. +28 −0 test/prism_regression/undef.parse-tree.exp
  231. +7 −0 test/prism_regression/undef.rb
  232. +24 −0 test/prism_regression/unless.parse-tree.exp
  233. +7 −0 test/prism_regression/unless.rb
  234. +10 −0 test/prism_regression/unless_else.parse-tree.exp
  235. +7 −0 test/prism_regression/unless_else.rb
  236. +78 −0 test/prism_regression/until.parse-tree.exp
  237. +27 −0 test/prism_regression/until.rb
  238. +15 −0 test/prism_regression/variables_class.parse-tree.exp
  239. +5 −0 test/prism_regression/variables_class.rb
  240. +15 −0 test/prism_regression/variables_global.parse-tree.exp
  241. +5 −0 test/prism_regression/variables_global.rb
  242. +15 −0 test/prism_regression/variables_instance.parse-tree.exp
  243. +5 −0 test/prism_regression/variables_instance.rb
  244. +115 −0 test/prism_regression/while.parse-tree.exp
  245. +33 −0 test/prism_regression/while.rb
  246. +111 −0 test/prism_regression/xstring.parse-tree.exp
  247. +13 −0 test/prism_regression/xstring.rb
  248. +25 −0 test/prism_regression/yield.parse-tree.exp
  249. +7 −0 test/prism_regression/yield.rb
  250. +8 −0 third_party/externals.bzl
  251. +8 −0 third_party/prism.BUILD
  252. +1 −1 tools/scripts/build_compilation_db.sh
  253. +22 −0 tools/scripts/inspect_parsing_errors.rb
  254. +45 −0 tools/scripts/verify_prism_regression_tests.sh
34 changes: 34 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: CI

on:
push:
branches: [prism, prism-squashed]
pull_request:
branches: [prism, prism-squashed]
jobs:
tests:
runs-on: ubuntu-latest
name: Prism Parser Regression Tests
steps:
- uses: actions/checkout@v4
- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: 3.3.0
- name: Set up Bazel
uses: bazel-contrib/setup-bazel@0.9.0
with:
# Avoid downloading Bazel every time.
bazelisk-cache: true
# Store build cache per workflow.
disk-cache: ${{ github.workflow }}
# Share repository cache between workflows.
repository-cache: true
- name: Verify parse trees
run: ./tools/scripts/verify_prism_regression_tests.sh
- name: Run tests
run: ./bazel test //test:prism_regression --config=dbg --test_output=errors
- name: Run location tests
run: ./bazel test //test:prism_location_tests --config=dbg --test_output=errors
- name: Run corpus tests
run: ./bazel test //test:test_corpus_prism --config=dbg --test_output=errors
4 changes: 3 additions & 1 deletion .vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
{
"recommendations": [
// C++ intellisense with clangd.
"llvm-vs-code-extensions.vscode-clangd"
"llvm-vs-code-extensions.vscode-clangd",
// TODO: remove this before upstreaming
"vadimcn.vscode-lldb",
],
"unwantedRecommendations": [
// Microsoft C++ intellisense (doesn't work properly with Sorbet)
39 changes: 38 additions & 1 deletion .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -22,6 +22,43 @@
"ignoreFailures": false
}
]
},
{
"type": "lldb",
"request": "launch",
"name": "Debug Prism test in LLDB",
"program": "${workspaceFolder}/bazel-bin/test/test_PosTests/prism_regression/${input:prism_regression_test_name}_prism.runfiles/com_stripe_ruby_typer/test/pipeline_test_runner",
"args": ["--single_test=${workspaceFolder}/test/prism_regression/${input:prism_regression_test_name}.rb", "--parser=prism"],
// We need to run all tests to generate the pipeline_test_runner files, which are needed by the LLDB debugger to execute the tests.
"preLaunchTask": "Run all Prism regression tests",
"stopOnEntry": false,
"sourceMap": {
"": "${workspaceFolder}",
},
},
{
"type": "lldb",
"request": "launch",
"name": "Debug Sorbet pipeline in LLDB",
"program": "${workspaceFolder}/bazel-bin/main/sorbet",
"args": ["--parser=prism", "${input:file_path}"],
"preLaunchTask": "Build with Prism",
"stopOnEntry": false,
"sourceMap": {
"": "${workspaceFolder}",
},
},
],
"inputs": [
{
"id": "prism_regression_test_name",
"type": "promptString",
"description": "Enter the test name, e.g. case for running test/prism_regression/case.rb",
},
{
"id": "file_path",
"type": "promptString",
"description": "Enter the path to the file to typecheck, e.g. test/prism_regression/case.rb",
}
]
}
}
136 changes: 136 additions & 0 deletions .vscode/tasks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
{
// See https://go.microsoft.com/fwlink/?LinkId=733558
// for the documentation about the tasks.json format
"version": "2.0.0",
"tasks": [
{
"label": "Build with Prism",
"type": "shell",
"command": "./bazel build //main:sorbet --config=dbg",
"group": {
"kind": "build",
"isDefault": true
},
"presentation": {
"reveal": "always"
}
},
{
"label": "(Re)Generate Prism test",
"type": "shell",
"command": "TEST_NAME=${input:prism_regression_test_name}; $EDITOR -w test/prism_regression/$TEST_NAME.rb; ./bazel-bin/main/sorbet --stop-after=parser --print=parse-tree test/prism_regression/$TEST_NAME.rb > test/prism_regression/$TEST_NAME.parse-tree.exp; $EDITOR test/prism_regression/$TEST_NAME.parse-tree.exp test/prism_regression/$TEST_NAME.rb",
"group": {
"kind": "test",
"isDefault": true
},
"presentation": {
"reveal": "always"
}
},
{
"label": "Run a single Prism regression test",
"type": "shell",
"command": "./bazel test //test:test_PosTests/prism_regression/${input:prism_regression_test_name}_prism //test:prism_regression/${input:prism_regression_test_name}_location_test --config=dbg --test_output=all",
"group": {
"kind": "test",
"isDefault": true
},
"presentation": {
"reveal": "always"
},
// With this option, when running this task with the `Tasks: Rerun Last Test` command, it'll
// reuse the previous run's input, rather than prompting for it again
"runOptions": {
"reevaluateOnRerun": false
}
},
{
"label": "Run all Prism regression tests",
"type": "shell",
"command": "./bazel test //test:prism_regression //test:prism_location_tests --config=dbg --test_output=all",
"group": {
"kind": "test",
"isDefault": true
},
"presentation": {
"reveal": "always"
}
},
{
"label": "Run all Prism corpus tests",
"type": "shell",
"command": "./bazel test //test:test_corpus_prism --config=dbg --test_output=all",
"group": {
"kind": "test",
"isDefault": true
},
"presentation": {
"reveal": "always"
}
},
{
"label": "Run a single Prism corpus test",
"type": "shell",
"command": "./bazel test //test:test_PosTests/testdata/${input:prism_corpus_test_name}_prism --config=dbg --test_output=all",
"group": {
"kind": "test",
"isDefault": true
},
"presentation": {
"reveal": "always"
},
// With this option, when running this task with the `Tasks: Rerun Last Test` command, it'll
// reuse the previous run's input, rather than prompting for it again
"runOptions": {
"reevaluateOnRerun": false
}
},
{
"label": "Typecheck with Prism",
"type": "shell",
"command": "bazel-bin/main/sorbet --parser=prism ${input:file_path}",
"group": {
"kind": "build",
"isDefault": true
},
"dependsOn": ["Build with Prism"],
"presentation": {
"reveal": "always"
}
},
{
"label": "Display Prism parse tree & errors for Ruby file",
"type": "shell",
"command": "tools/scripts/inspect_parsing_errors.rb ${file}",
"presentation": {
"reveal": "always"
},
"options": {
// Makes sure the shell can find the correct Ruby version
"shell": {
"executable": "${env:SHELL}",
"args": ["-c"]
}
},
"problemMatcher": []
}
],
"inputs": [
{
"id": "prism_corpus_test_name",
"type": "promptString",
"description": "Enter the test name, e.g. parser/and_and_bug for running test/testdata/parser/and_and_bug.rb",
},
{
"id": "prism_regression_test_name",
"type": "promptString",
"description": "Enter the test name, e.g. case for running test/prism_regression/case.rb",
},
{
"id": "file_path",
"type": "promptString",
"default": "test.rb",
"description": "Enter the file path to typecheck, e.g. test.rb",
}
]
}
1 change: 1 addition & 0 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source "https://rubygems.org"
12 changes: 12 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
GEM
remote: https://rubygems.org/
specs:

PLATFORMS
arm64-darwin-23
ruby

DEPENDENCIES

BUNDLED WITH
2.5.17
Empty file added break.rb
Empty file.
21 changes: 21 additions & 0 deletions main/options/options.cc
Original file line number Diff line number Diff line change
@@ -725,6 +725,9 @@ buildOptions(const vector<pipeline::semantic_extension::SemanticExtensionProvide
cxxopts::value<vector<string>>()->implicit_value("all"), "SECTION");
// }}}

options.add_options("dev")("parser", "Which parser to use", cxxopts::value<string>()->default_value("sorbet"),
"{sorbet, prism}");

for (auto &provider : semanticExtensionProviders) {
provider->injectOptions(options);
}
@@ -793,6 +796,22 @@ Phase extractStopAfter(cxxopts::ParseResult &raw, shared_ptr<spdlog::logger> log
return Phase::INIT;
}

Parser extractParser(cxxopts::ParseResult &raw, shared_ptr<spdlog::logger> logger) {
string opt = raw["parser"].as<string>();
for (auto &known : parser_options) {
if (known.option == opt) {
return known.flag;
}
}
vector<string_view> allOptions;
for (auto &known : parser_options) {
allOptions.emplace_back(known.option);
}

logger->error("Unknown --parser option: {}\nValid values: {}", opt, fmt::join(allOptions, ", "));
return Parser::SORBET;
}

// Given a path, strips any trailing forward slashes (/) at the end of the path.
string_view stripTrailingSlashes(string_view path) {
while (path.back() == '/') {
@@ -956,6 +975,7 @@ void readOptions(Options &opts,
throw EarlyReturnWithCode(1);
}
opts.stopAfterPhase = extractStopAfter(raw, logger);
opts.parser = extractParser(raw, logger);

opts.silenceErrors = raw["quiet"].as<bool>();
opts.autocorrect = raw["autocorrect"].as<bool>();
@@ -1356,6 +1376,7 @@ void readOptions(Options &opts,
fmt::print("Sorbet typechecker {}\n", sorbet_full_version_string);
throw EarlyReturnWithCode(0);
}

} catch (cxxopts::OptionParseException &e) {
logger->info("{}. To see all available options pass `--help`.", e.what());
throw EarlyReturnWithCode(1);
16 changes: 16 additions & 0 deletions main/options/options.h
Original file line number Diff line number Diff line change
@@ -106,6 +106,21 @@ enum class Phase {
INFERENCER,
};

enum class Parser {
PRISM,
SORBET,
};

struct ParserOptions {
std::string option;
Parser flag;
};

const std::vector<ParserOptions> parser_options({
{"sorbet", Parser::SORBET},
{"prism", Parser::PRISM},
});

namespace {

#if !defined(EMSCRIPTEN)
@@ -120,6 +135,7 @@ constexpr size_t MAX_CACHE_SIZE_BYTES = 1L * 1024 * 1024 * 1024; // 1 GiB
struct Options {
Printers print;
Phase stopAfterPhase = Phase::INFERENCER;
Parser parser = Parser::SORBET;
bool noStdlib = false;

// Should we monitor STDOUT for HUP and exit if it hangs up. This is a
2 changes: 2 additions & 0 deletions main/pipeline/BUILD
Original file line number Diff line number Diff line change
@@ -32,6 +32,7 @@ cc_library(
"//namer",
"//packager",
"//parser",
"//parser/prism",
"//payload/binary",
"//payload/text",
"//resolver",
@@ -70,6 +71,7 @@ cc_library(
"//main/options",
"//namer",
"//parser",
"//parser/prism",
"//payload/binary",
"//payload/text",
"//resolver",
54 changes: 52 additions & 2 deletions main/pipeline/pipeline.cc
Original file line number Diff line number Diff line change
@@ -37,11 +37,14 @@
#include "main/pipeline/semantic_extension/SemanticExtension.h"
#include "namer/namer.h"
#include "parser/parser.h"
#include "parser/prism/Parser.h"
#include "parser/prism/Translator.h"
#include "pipeline.h"
#include "resolver/resolver.h"
#include "rewriter/rewriter.h"

using namespace std;
namespace Prism = sorbet::parser::Prism;

namespace sorbet::realmain::pipeline {

@@ -112,6 +115,7 @@ unique_ptr<parser::Node> runParser(core::GlobalState &gs, core::FileRef file, co
auto settings = parser::Parser::Settings{traceLexer, traceParser, indentationAware};
nodes = parser::Parser::run(gs, file, settings);
}

if (print.ParseTree.enabled) {
print.ParseTree.fmt("{}\n", nodes->toStringWithTabs(gs, 0));
}
@@ -127,6 +131,37 @@ unique_ptr<parser::Node> runParser(core::GlobalState &gs, core::FileRef file, co
return nodes;
}

unique_ptr<parser::Node> runPrismParser(core::GlobalState &gs, core::FileRef file, bool stopAfterParser,
const options::Printers &print) {
auto source = file.data(gs).source();

core::UnfreezeNameTable nameTableAccess(gs);

Prism::Parser parser{source};
Prism::ParseResult parseResult = parser.parse_root();

if (stopAfterParser) {
return std::unique_ptr<parser::Node>();
}

auto nodes = Prism::Translator(parser, gs, file).translate(std::move(parseResult));

if (print.ParseTree.enabled) {
print.ParseTree.fmt("{}\n", nodes->toStringWithTabs(gs, 0));
}
if (print.ParseTreeJson.enabled) {
print.ParseTreeJson.fmt("{}\n", nodes->toJSON(gs, 0));
}
if (print.ParseTreeJsonWithLocs.enabled) {
print.ParseTreeJson.fmt("{}\n", nodes->toJSONWithLocs(gs, file, 0));
}
if (print.ParseTreeWhitequark.enabled) {
print.ParseTreeWhitequark.fmt("{}\n", nodes->toWhitequark(gs, 0));
}

return nodes;
}

ast::ExpressionPtr runDesugar(core::GlobalState &gs, core::FileRef file, unique_ptr<parser::Node> parseTree,
const options::Printers &print, bool preserveConcreteSyntax = false) {
Timer timeit(gs.tracer(), "runDesugar", {{"file", string(file.data(gs).path())}});
@@ -186,6 +221,8 @@ ast::ExpressionPtr desugarOne(const options::Options &opts, core::GlobalState &g
ast::ParsedFile indexOne(const options::Options &opts, core::GlobalState &lgs, core::FileRef file,
ast::ExpressionPtr tree) {
auto &print = opts.print;
auto &parser = opts.parser;

ast::ParsedFile rewritten{nullptr, file};

Timer timeit(lgs.tracer(), "indexOne", {{"file", string(file.data(lgs).path())}});
@@ -195,8 +232,21 @@ ast::ParsedFile indexOne(const options::Options &opts, core::GlobalState &lgs, c
if (file.data(lgs).strictLevel == core::StrictLevel::Ignore) {
return emptyParsedFile(file);
}
auto parseTree = runParser(lgs, file, print, opts.traceLexer, opts.traceParser);
if (opts.stopAfterPhase == options::Phase::PARSER) {

unique_ptr<parser::Node> parseTree;

bool stopAfterParser = opts.stopAfterPhase == options::Phase::PARSER;

switch (parser) {
case options::Parser::SORBET:
parseTree = runParser(lgs, file, print, opts.traceLexer, opts.traceParser);
break;
case options::Parser::PRISM:
parseTree = runPrismParser(lgs, file, stopAfterParser, print);
break;
}

if (stopAfterParser) {
return emptyParsedFile(file);
}
tree = runDesugar(lgs, file, move(parseTree), print);
3 changes: 3 additions & 0 deletions main/pipeline/pipeline.h
Original file line number Diff line number Diff line change
@@ -7,6 +7,7 @@
#include "common/kvstore/KeyValueStore.h"
#include "core/FileHash.h"
#include "main/options/options.h"
#include "parser/parser.h"

namespace sorbet::core::lsp {
class PreemptionTaskManager;
@@ -25,6 +26,8 @@ std::vector<core::FileRef> reserveFiles(std::unique_ptr<core::GlobalState> &gs,
std::vector<ast::ParsedFile> index(core::GlobalState &gs, absl::Span<core::FileRef> files, const options::Options &opts,
WorkerPool &workers, const std::unique_ptr<const OwnedKeyValueStore> &kvstore);

std::unique_ptr<parser::Node> runPrismParser(core::GlobalState &gs, core::FileRef file, bool stopAfterParser,
const options::Printers &print);
size_t partitionPackageFiles(const core::GlobalState &gs, absl::Span<core::FileRef> files);
void unpartitionPackageFiles(std::vector<ast::ParsedFile> &packageFiles,
std::vector<ast::ParsedFile> &&nonPackageFiles);
22 changes: 22 additions & 0 deletions parser/prism/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
cc_library(
name = "prism",
srcs = [
"Parser.cc",
"Translator.cc",
],
hdrs = [
"Parser.h",
"Translator.h",
"Helpers.h",
],
linkstatic = select({
"//tools/config:linkshared": 0,
"//conditions:default": 1,
}),
visibility = ["//visibility:public"],
deps = [
"//core",
"//parser", # Legacy parser, needed for translating to its nodes defined in `parser/Node_gen.h`
"@prism",
],
)
202 changes: 202 additions & 0 deletions parser/prism/Helpers.h

Large diffs are not rendered by default.

49 changes: 49 additions & 0 deletions parser/prism/Parser.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
#include "parser/prism/Parser.h"

namespace sorbet::parser::Prism {

pm_parser_t *Parser::getRawParserPointer() {
return &storage->parser;
}

ParseResult Parser::parse_root() {
pm_node_t *root = pm_parse(getRawParserPointer());
return ParseResult{*this, root, collectErrors()};
};

core::LocOffsets Parser::translateLocation(pm_location_t location) {
uint32_t start = static_cast<uint32_t>(location.start - storage->parser.start);
uint32_t end = static_cast<uint32_t>(location.end - storage->parser.start);

return core::LocOffsets{start, end};
}

std::string_view Parser::resolveConstant(pm_constant_id_t constant_id) {
pm_constant_t *constant = pm_constant_pool_id_to_constant(&storage->parser.constant_pool, constant_id);

return std::string_view(reinterpret_cast<const char *>(constant->start), constant->length);
}

std::string_view Parser::extractString(pm_string_t *string) {
return std::string_view(reinterpret_cast<const char *>(pm_string_source(string)), pm_string_length(string));
}

std::vector<ParseError> Parser::collectErrors() {
std::vector<ParseError> parseErrors;
parseErrors.reserve(storage->parser.error_list.size);

auto error_list = storage->parser.error_list;

for (auto *node = error_list.head; node != nullptr; node = node->next) {
auto *error = reinterpret_cast<pm_diagnostic_t *>(node);
auto level = static_cast<pm_error_level_t>(error->level);

ParseError parseError(error->diag_id, std::string(reinterpret_cast<const char *>(error->message)),
error->location, level);

parseErrors.push_back(parseError);
}

return parseErrors;
}
}; // namespace sorbet::parser::Prism
103 changes: 103 additions & 0 deletions parser/prism/Parser.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
#ifndef SORBET_PARSER_PRISM_PARSER_H
#define SORBET_PARSER_PRISM_PARSER_H

#include <memory>
#include <string>
#include <vector>

extern "C" {
#include "prism.h"
}

#include "core/LocOffsets.h"

namespace sorbet::parser::Prism {

class ParseResult;

class ParseError {
public:
ParseError(pm_diagnostic_id_t id, const std::string &message, pm_location_t location, pm_error_level_t level)
: id(id), message(message), location(location), level(level) {}

pm_diagnostic_id_t id;
std::string message;
pm_location_t location;
pm_error_level_t level;
};

// A backing implemenation detail of `Parser`, which stores a Prism parser and its options in a single allocation.
struct ParserStorage {
// The version of Ruby syntax that we're parsing with Prism. This determines what syntax is supported or not.
static constexpr std::string_view ParsedRubyVersion = "3.3.0";
pm_parser_t parser;
pm_options_t options;

ParserStorage(std::string_view source_code) : parser{}, options{} {
pm_options_version_set(&options, ParsedRubyVersion.data(), ParsedRubyVersion.size());

pm_parser_init(&parser, reinterpret_cast<const uint8_t *>(source_code.data()), source_code.size(), &options);
}

~ParserStorage() {
pm_parser_free(&parser);
pm_options_free(&options);
}

ParserStorage(const ParserStorage &) = delete;
ParserStorage &operator=(const ParserStorage &) = delete;
ParserStorage(ParserStorage &&) = delete;
ParserStorage &operator=(ParserStorage &&) = delete;
};

class Parser final {
friend class ParseResult;
friend struct NodeDeleter;

std::shared_ptr<ParserStorage> storage;

public:
Parser(std::string_view source_code) : storage(std::make_shared<ParserStorage>(source_code)) {}

Parser(const Parser &) = default;
Parser &operator=(const Parser &) = default;

ParseResult parse_root();
core::LocOffsets translateLocation(pm_location_t location);
std::string_view resolveConstant(pm_constant_id_t constant_id);
std::string_view extractString(pm_string_t *string);

private:
std::vector<ParseError> collectErrors();
pm_parser_t *getRawParserPointer();
};

class ParseResult final {
struct NodeDeleter {
Parser parser;

void operator()(pm_node_t *node) {
pm_node_destroy(parser.getRawParserPointer(), node);
}
};

friend class Parser;
friend class Translator;

Parser parser;
std::unique_ptr<pm_node_t, NodeDeleter> node;
std::vector<ParseError> parseErrors;

ParseResult(Parser parser, pm_node_t *node, std::vector<ParseError> parseErrors)
: parser{parser}, node{node, NodeDeleter{parser}}, parseErrors{parseErrors} {}

ParseResult(const ParseResult &) = delete; // Copy constructor
ParseResult &operator=(const ParseResult &) = delete; // Copy assignment

pm_node_t *getRawNodePointer() const {
return node.get();
}
};

} // namespace sorbet::parser::Prism
#endif // SORBET_PARSER_PRISM_PARSER_H
1,982 changes: 1,982 additions & 0 deletions parser/prism/Translator.cc

Large diffs are not rendered by default.

110 changes: 110 additions & 0 deletions parser/prism/Translator.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
#ifndef SORBET_PARSER_PRISM_TRANSLATOR_H
#define SORBET_PARSER_PRISM_TRANSLATOR_H

#include "core/errors/parser.h"
#include "parser/Node.h" // To clarify: these are Sorbet Parser nodes, not Prism ones.
#include "parser/prism/Parser.h"
#include <memory>

extern "C" {
#include "prism.h"
}

namespace sorbet::parser::Prism {

class Translator final {
Parser parser;

// The functions in Pipeline.cc pass around a reference to the global state as a parameter,
// but don't have explicit ownership over it. We take a temporary reference to it, but we can't
// escape that scope, which is why Translator objects can't be copied, or even moved.
core::GlobalState &gs;

// Needed for reporting diagnostics
core::FileRef file;

// The parse errors that occurred while parsing the root node
std::vector<ParseError> parseErrors;

// Context variables
bool isInMethodDef = false;

// Keep track of the unique ID counter
// uniqueCounterStorage is the source of truth maintained by the parent Translator
// uniqueCounter is a pointer to uniqueCounterStorage and is passed down to child Translators
int uniqueCounterStorage;
int *uniqueCounter;

Translator(Translator &&) = delete; // Move constructor
Translator(const Translator &) = delete; // Copy constructor
Translator &operator=(Translator &&) = delete; // Move assignment
Translator &operator=(const Translator &) = delete; // Copy assignment
public:
Translator(Parser parser, core::GlobalState &gs, core::FileRef file)
: parser(std::move(parser)), gs(gs), file(file), uniqueCounterStorage(1),
uniqueCounter(&this->uniqueCounterStorage) {}

int nextUniqueID() {
return *uniqueCounter += 1;
}

// Translates the given AST from Prism's node types into the equivalent AST in Sorbet's legacy parser node types.
std::unique_ptr<parser::Node> translate(pm_node_t *node);
std::unique_ptr<parser::Node> translate(const ParseResult &parseResult);

private:
// Private constructor used only for creating child translators
// uniqueCounterStorage is passed as the minimum integer value and is never used
Translator(Parser parser, core::GlobalState &gs, core::FileRef file, std::vector<ParseError> parseErrors,
bool isInMethodDef, int *uniqueCounter)
: parser(parser), gs(gs), file(file), parseErrors(parseErrors), isInMethodDef(isInMethodDef),
uniqueCounterStorage(std::numeric_limits<int>::min()), uniqueCounter(uniqueCounter) {}
void reportError(core::LocOffsets loc, const std::string &message);

core::LocOffsets translateLoc(pm_location_t loc);

parser::NodeVec translateMulti(pm_node_list prismNodes);
void translateMultiInto(NodeVec &sorbetNodes, absl::Span<pm_node_t *> prismNodes);

NodeVec translateArguments(pm_arguments_node *node, pm_node *blockArgumentNode = nullptr);
parser::NodeVec translateKeyValuePairs(pm_node_list_t elements);
static bool isKeywordHashElement(sorbet::parser::Node *nd);
std::unique_ptr<parser::Node> translateCallWithBlock(pm_node_t *prismBlockOrLambdaNode,
std::unique_ptr<parser::Node> sendNode);
std::unique_ptr<parser::Node> translateRescue(pm_rescue_node *prismRescueNode,
std::unique_ptr<parser::Node> beginNode,
std::unique_ptr<parser::Node> elseNode);
std::unique_ptr<parser::Node> translateStatements(pm_statements_node *stmtsNode, bool inlineIfSingle);

template <typename SorbetNode> std::unique_ptr<SorbetNode> translateSimpleKeyword(pm_node_t *untypedNode);

std::unique_ptr<parser::Regopt> translateRegexpOptions(pm_location_t closingLoc);
std::unique_ptr<parser::Regexp> translateRegexp(pm_string_t unescaped, core::LocOffsets location,
pm_location_t closingLoc);

template <typename PrismNode> std::unique_ptr<parser::Mlhs> translateMultiTargetLhs(PrismNode *);

template <typename PrismAssignmentNode, typename SorbetLHSNode>
std::unique_ptr<parser::Assign> translateAssignment(pm_node_t *node);

template <typename PrismAssignmentNode, typename SorbetAssignmentNode, typename SorbetLHSNode>
std::unique_ptr<SorbetAssignmentNode> translateOpAssignment(pm_node_t *node);

template <typename PrismLhsNode, typename SorbetLHSNode>
std::unique_ptr<parser::Node> translateConst(PrismLhsNode *node, bool replaceWithDynamicConstAssign = false);
core::NameRef translateConstantName(pm_constant_id_t constant_id);

// Pattern-matching
// ... variations of the main translation functions for pattern-matching related nodes.
std::unique_ptr<parser::Node> patternTranslate(pm_node_t *node);
parser::NodeVec patternTranslateMulti(pm_node_list prismNodes);
void patternTranslateMultiInto(NodeVec &sorbetNodes, absl::Span<pm_node_t *> prismNodes);

std::string_view sliceLocation(pm_location_t loc);

// Context management helpers. These return a copy of `this` with some change to the context.
Translator enterMethodDef();
};

} // namespace sorbet::parser::Prism
#endif // SORBET_PARSER_PRISM_TRANSLATOR_H
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
Command being timed: "bazel-bin/main/sorbet --parser=prism --stop-after=parser ../../Shopify/shopify"
User time (seconds): 308.82
System time (seconds): 10.04
Percent of CPU this job got: 113%
Elapsed (wall clock) time (h:mm:ss or m:ss): 4:40.27
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 881064
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 226461
Voluntary context switches: 14702
Involuntary context switches: 80406
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 0

Command exited with non-zero status 100
Command being timed: "bazel-bin/main/sorbet --parser=sorbet --stop-after=parser ../../Shopify/shopify"
User time (seconds): 476.12
System time (seconds): 9.49
Percent of CPU this job got: 160%
Elapsed (wall clock) time (h:mm:ss or m:ss): 5:02.08
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 1355112
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 343679
Voluntary context switches: 231
Involuntary context switches: 77417
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 100

Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
Command being timed: "bazel-bin/main/sorbet --parser=prism --stop-after=parser ../../Shopify/shopify"
User time (seconds): 311.65
System time (seconds): 30.33
Percent of CPU this job got: 123%
Elapsed (wall clock) time (h:mm:ss or m:ss): 4:36.99
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 877296
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 225456
Voluntary context switches: 89208
Involuntary context switches: 354071
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 0

Command exited with non-zero status 100
Command being timed: "bazel-bin/main/sorbet --parser=sorbet --stop-after=parser ../../Shopify/shopify"
User time (seconds): 771.97
System time (seconds): 27.98
Percent of CPU this job got: 239%
Elapsed (wall clock) time (h:mm:ss or m:ss): 5:34.40
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 1349328
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 342514
Voluntary context switches: 20
Involuntary context switches: 147120
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 100

Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
Command being timed: "bazel-bin/main/sorbet --parser=prism --stop-after=parser ../yjit-bench/benchmarks"
User time (seconds): 1.41
System time (seconds): 0.11
Percent of CPU this job got: 115%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.31
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 137440
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 24
Minor (reclaiming a frame) page faults: 38722
Voluntary context switches: 406
Involuntary context switches: 2968
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 0

Command being timed: "bazel-bin/main/sorbet --parser=sorbet --stop-after=parser ../yjit-bench/benchmarks"
User time (seconds): 4.07
System time (seconds): 0.08
Percent of CPU this job got: 198%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.09
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 197452
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 51716
Voluntary context switches: 0
Involuntary context switches: 1299
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 0

Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
Command being timed: "bazel-bin/main/sorbet --parser=prism --stop-after=parser ../yjit-bench/benchmarks"
User time (seconds): 1.81
System time (seconds): 0.18
Percent of CPU this job got: 35%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.69
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 135876
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 38705
Voluntary context switches: 383
Involuntary context switches: 2705
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 0

Command being timed: "bazel-bin/main/sorbet --parser=sorbet --stop-after=parser ../yjit-bench/benchmarks"
User time (seconds): 10.03
System time (seconds): 0.09
Percent of CPU this job got: 181%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.57
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 205288
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 53938
Voluntary context switches: 0
Involuntary context switches: 1746
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 0

Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
Command exited with non-zero status 100
Command being timed: "bazel-bin/main/sorbet --parser=prism test/prism_regression"
User time (seconds): 1.07
System time (seconds): 0.03
Percent of CPU this job got: 117%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.93
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 64996
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 18361
Voluntary context switches: 0
Involuntary context switches: 893
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 100

Command exited with non-zero status 100
Command being timed: "bazel-bin/main/sorbet --parser=sorbet test/prism_regression"
User time (seconds): 1.06
System time (seconds): 0.03
Percent of CPU this job got: 119%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.91
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 65492
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 18334
Voluntary context switches: 0
Involuntary context switches: 997
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 100

Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
Command exited with non-zero status 100
Command being timed: "bazel-bin/main/sorbet --parser=prism test/prism_regression"
User time (seconds): 1.65
System time (seconds): 0.03
Percent of CPU this job got: 121%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.39
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 69820
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 19420
Voluntary context switches: 0
Involuntary context switches: 1327
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 100

Command exited with non-zero status 100
Command being timed: "bazel-bin/main/sorbet --parser=sorbet test/prism_regression"
User time (seconds): 1.66
System time (seconds): 0.03
Percent of CPU this job got: 122%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.38
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 70564
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 19407
Voluntary context switches: 0
Involuntary context switches: 939
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 100

49 changes: 49 additions & 0 deletions prism_benchmarks/memory/data/pipeline/rbi/2024-09-18-5df44846d.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
Command exited with non-zero status 100
Command being timed: "bazel-bin/main/sorbet --parser=prism ../../Shopify/rbi"
User time (seconds): 22.53
System time (seconds): 0.14
Percent of CPU this job got: 349%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:06.48
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 125744
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 33529
Voluntary context switches: 164
Involuntary context switches: 6882
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 100

Command being timed: "bazel-bin/main/sorbet --parser=sorbet ../../Shopify/rbi"
User time (seconds): 24.78
System time (seconds): 0.17
Percent of CPU this job got: 364%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:06.85
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 134156
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 35155
Voluntary context switches: 0
Involuntary context switches: 5745
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 16384
Exit status: 0

67 changes: 67 additions & 0 deletions prism_benchmarks/memory/run_benchmarks.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
#!/bin/bash

# This script runs memory comparisons for the Prism parser in the Sorbet type checker.
# It assumes the presence of a few directories and tools:
# - The yjit-bench directory, which contains the benchmarks for YJIT (https://github.com/Shopify/yjit-bench)
# - The Shopify directory, which contains the Shopify codebase (https://github.com/Shopify/shopify)
# - gtime, a command-line time and memory analysis tool (https://www.gnu.org/software/time/)

# ----- Setup -----

YJIT_BENCH_DIR="../yjit-bench"
SHOPIFY_DIR="../../Shopify/shopify"
RBI_DIR="../../Shopify/rbi"

export SORBET_SILENCE_DEV_MESSAGE=1

# Check if required directories exist
for dir in "$YJIT_BENCH_DIR" "$SHOPIFY_DIR" "$RBI_DIR"; do
if [ ! -d "$dir" ]; then
echo "Please clone the required directories before running this script."
exit 1
fi
done

# Check and install gtime if necessary
if ! command -v gtime &> /dev/null; then
echo "Installing gtime..."
if [[ "$OSTYPE" == "darwin"* ]]; then
brew install gnu-time
else
echo "Please install gtime: https://www.gnu.org/software/time/"
exit 1
fi
fi

output_file_name="$(date '+%Y-%m-%d')-$(git rev-parse --short HEAD).txt"
parsers=("prism" "sorbet")

# ----- Run Benchmarks -----

run_benchmark() {
local title="$1"
local output_dir="$2"
local command="$3"

echo "#### $title ####"
output_file="prism_benchmarks/memory/data/$output_dir/${output_file_name}"

for parser in "${parsers[@]}"; do
gtime --verbose --output="$output_file" --append \
bazel-bin/main/sorbet --parser=$parser $command &> /dev/null
echo "" >> "$output_file"
# Calculate peak memory usage:
# - Search for lines that contain peak memory usage info
# - Take the last one
# - Extract the memory usage value
# - Convert it to MB
peak_memory=$(grep "Maximum resident set size (kbytes):" "$output_file" | tail -n 1 | awk '{print $6/1024 " MB"}')
echo "-> Peak Memory Usage with $parser parser: $peak_memory"
done
echo ""
}

run_benchmark "Memory Check 1: yjit-bench, parser only" "parser/yjit-bench" "--stop-after=parser $YJIT_BENCH_DIR/benchmarks"
run_benchmark "Memory Check 2: shopify, parser only" "parser/shopify" "--stop-after=parser $SHOPIFY_DIR"
run_benchmark "Memory Check 3: prism regression tests, whole pipeline" "pipeline/prism_regression" "test/prism_regression"
run_benchmark "Memory Check 4: RBI gem, whole pipeline" "pipeline/rbi" $RBI_DIR
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
{
"results": [
{
"command": "bazel-bin/main/sorbet --parser=sorbet --stop-after=parser test/testdata",
"mean": 10.0427811339,
"stddev": 0.025126222846626647,
"median": 10.0368188785,
"user": 15.975400439999998,
"system": 1.0591082200000002,
"min": 10.0078213055,
"max": 10.0817527725,
"times": [
10.0276461275,
10.0627562915,
10.0649185675,
10.0174194365,
10.0078213055,
10.0817527725,
10.0454906105,
10.0242698005,
10.0281471465,
10.0675892805
],
"exit_codes": [
100,
100,
100,
100,
100,
100,
100,
100,
100,
100
],
"parameters": {
"parser": "sorbet"
}
},
{
"command": "bazel-bin/main/sorbet --parser=prism --stop-after=parser test/testdata",
"mean": 9.028913599900001,
"stddev": 0.1174086729884074,
"median": 8.991058692,
"user": 12.66185554,
"system": 1.9508827199999998,
"min": 8.9338502675,
"max": 9.2513346015,
"times": [
8.9816991245,
9.041188655500001,
9.0004182595,
9.0065693535,
9.2513346015,
8.9599665045,
8.9354991625,
8.9459703115,
9.2326397585,
8.9338502675
],
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"parameters": {
"parser": "prism"
}
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
{
"results": [
{
"command": "bazel-bin/main/sorbet --parser=sorbet --stop-after=parser test/testdata",
"mean": 5.30816793542,
"stddev": 0.008554918806646472,
"median": 5.30728540082,
"user": 11.338379319999998,
"system": 0.9336967399999999,
"min": 5.29577648882,
"max": 5.32004709582,
"times": [
5.30104430882,
5.30434597482,
5.29893825782,
5.31593376782,
5.31993879882,
5.30577622782,
5.29577648882,
5.32004709582,
5.30879457382,
5.31108385982
],
"exit_codes": [
100,
100,
100,
100,
100,
100,
100,
100,
100,
100
],
"parameters": {
"parser": "sorbet"
}
},
{
"command": "bazel-bin/main/sorbet --parser=prism --stop-after=parser test/testdata",
"mean": 4.86590450502,
"stddev": 0.011452274879975811,
"median": 4.86448913132,
"user": 8.30477872,
"system": 1.13498544,
"min": 4.85319106082,
"max": 4.89339179082,
"times": [
4.86646737282,
4.86645828082,
4.85319106082,
4.86291949182,
4.89339179082,
4.85785923482,
4.85783726082,
4.87515050282,
4.8660587708200005,
4.85971128382
],
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"parameters": {
"parser": "prism"
}
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
{
"results": [
{
"command": "bazel-bin/main/sorbet --parser=sorbet --stop-after=parser ../yjit-bench/benchmarks",
"mean": 13.983836266140003,
"stddev": 0.3753695726271287,
"median": 13.88645245584,
"user": 22.976926319999997,
"system": 0.73527856,
"min": 13.451897627840001,
"max": 14.83432713384,
"times": [
13.451897627840001,
13.78587580584,
13.92110683084,
13.83593312984,
13.85179808084,
14.83432713384,
14.35947390984,
14.01668414084,
13.99022805084,
13.79103795084
],
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"parameters": {
"parser": "sorbet"
}
},
{
"command": "bazel-bin/main/sorbet --parser=prism --stop-after=parser ../yjit-bench/benchmarks",
"mean": 8.63029390654,
"stddev": 0.1018242999685836,
"median": 8.59375904684,
"user": 11.002803519999997,
"system": 0.9380242599999999,
"min": 8.504474396840001,
"max": 8.85641065884,
"times": [
8.57078804984,
8.85641065884,
8.56780114184,
8.69831273484,
8.57807582284,
8.70291158584,
8.60944227084,
8.504474396840001,
8.65632465784,
8.55839774584
],
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"parameters": {
"parser": "prism"
}
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
{
"results": [
{
"command": "bazel-bin/main/sorbet --parser=sorbet --stop-after=parser ../yjit-bench/benchmarks",
"mean": 6.25072342584,
"stddev": 0.1073974255052388,
"median": 6.19560604254,
"user": 16.251893579999994,
"system": 0.69604118,
"min": 6.16291310254,
"max": 6.44517373554,
"times": [
6.16291310254,
6.2626671075399996,
6.18448813654,
6.18419261454,
6.18531723254,
6.44517373554,
6.1749147535399995,
6.25796427854,
6.4437084445399995,
6.20589485254
],
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"parameters": {
"parser": "sorbet"
}
},
{
"command": "bazel-bin/main/sorbet --parser=prism --stop-after=parser ../yjit-bench/benchmarks",
"mean": 4.478415396640001,
"stddev": 0.006773926630539733,
"median": 4.47716277004,
"user": 6.40941108,
"system": 0.7607843799999999,
"min": 4.47000373854,
"max": 4.49072622954,
"times": [
4.47028990954,
4.48113160654,
4.4784504395399996,
4.47585923954,
4.48715721954,
4.49072622954,
4.47000373854,
4.47375967154,
4.47587510054,
4.48090081154
],
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"parameters": {
"parser": "prism"
}
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
{
"results": [
{
"command": "bazel-bin/main/sorbet --parser=sorbet test/prism_regression",
"mean": 0.84178919584,
"stddev": 0.008118653707744158,
"median": 0.83854312314,
"user": 1.59382048,
"system": 0.11926071999999999,
"min": 0.83163896114,
"max": 0.85557230614,
"times": [
0.83945907614,
0.83682301814,
0.85210590814,
0.83473545814,
0.85557230614,
0.83762717014,
0.83645661814,
0.84983627314,
0.84363716914,
0.83163896114
],
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"parameters": {
"parser": "sorbet"
}
},
{
"command": "bazel-bin/main/sorbet --parser=prism test/prism_regression",
"mean": 0.8412279621400002,
"stddev": 0.007882902400181078,
"median": 0.83984855864,
"user": 1.61844638,
"system": 0.10229632,
"min": 0.8312792881400001,
"max": 0.85482773714,
"times": [
0.84042653414,
0.8312792881400001,
0.83306462914,
0.85482773714,
0.85185022614,
0.84023890714,
0.83664591814,
0.8480770171400001,
0.83945821014,
0.83641115414
],
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"parameters": {
"parser": "prism"
}
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
{
"results": [
{
"command": "bazel-bin/main/sorbet --parser=sorbet test/prism_regression",
"mean": 1.74419526482,
"stddev": 0.069048868040311,
"median": 1.73331423512,
"user": 2.73193046,
"system": 0.13378163999999998,
"min": 1.6378026806200001,
"max": 1.84798413862,
"times": [
1.72125983862,
1.84798413862,
1.64443633762,
1.79690505062,
1.72480685162,
1.80412130462,
1.7226981556199998,
1.80011667162,
1.74182161862,
1.6378026806200001
],
"exit_codes": [
100,
100,
100,
100,
100,
100,
100,
100,
100,
100
],
"parameters": {
"parser": "sorbet"
}
},
{
"command": "bazel-bin/main/sorbet --parser=prism test/prism_regression",
"mean": 1.6491571758199997,
"stddev": 0.2075739536981031,
"median": 1.7204806501199998,
"user": 2.5753856599999994,
"system": 0.11331144,
"min": 1.07299557962,
"max": 1.80577517662,
"times": [
1.80577517662,
1.07299557962,
1.7247325946199998,
1.71462417062,
1.64513017262,
1.7240551876199999,
1.72276719362,
1.64053038262,
1.72107253662,
1.71988876362
],
"exit_codes": [
100,
100,
100,
100,
100,
100,
100,
100,
100,
100
],
"parameters": {
"parser": "prism"
}
}
]
}
78 changes: 78 additions & 0 deletions prism_benchmarks/time/data/pipeline/rbi/2024-09-18-5df44846d.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
{
"results": [
{
"command": "bazel-bin/main/sorbet --parser=sorbet ../rbi",
"mean": 6.16736409352,
"stddev": 0.06774821106573646,
"median": 6.15982815322,
"user": 21.81549374,
"system": 0.20507438000000003,
"min": 6.083021132720001,
"max": 6.28290752972,
"times": [
6.16744417972,
6.12050154872,
6.28290752972,
6.251253995720001,
6.11700055272,
6.15221212672,
6.083021132720001,
6.23237357272,
6.16752112172,
6.09940517472
],
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"parameters": {
"parser": "sorbet"
}
},
{
"command": "bazel-bin/main/sorbet --parser=prism ../rbi",
"mean": 6.18529064662,
"stddev": 0.04775460831234028,
"median": 6.17915380222,
"user": 19.401574840000002,
"system": 0.18767527999999994,
"min": 6.12177066272,
"max": 6.282041295720001,
"times": [
6.12177066272,
6.13718851672,
6.282041295720001,
6.153297766720001,
6.15942091572,
6.22498076072,
6.19488676172,
6.221012181720001,
6.171952952720001,
6.18635465172
],
"exit_codes": [
100,
100,
100,
100,
100,
100,
100,
100,
100,
100
],
"parameters": {
"parser": "prism"
}
}
]
}
60 changes: 60 additions & 0 deletions prism_benchmarks/time/run_benchmarks.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
#!/bin/bash

# This script runs the benchmarks for the Prism parser in the Sorbet type checker.
# It assumes the presence of a few directories and tools:
# - The yjit-bench directory, which contains the benchmarks for YJIT (https://github.com/Shopify/yjit-bench)
# - The rbi directory, which contains the RBI gem (https://github.com/Shopify/rbi)
# - Hyperfine, a command-line benchmarking tool (https://github.com/sharkdp/hyperfine)

# ----- Setup -----

YJIT_BENCH_DIR="../yjit-bench"
RBI_DIR="../rbi"

export SORBET_SILENCE_DEV_MESSAGE=1

# Check if required directories exist
for dir in "$YJIT_BENCH_DIR" "$RBI_DIR"; do
if [ ! -d "$dir" ]; then
echo "Please clone the required directories before running this script."
exit 1
fi
done

# Check if hyperfine is installed
if [! command -v hyperfine >/dev/null 2>&1 ]; then
echo "Please install hyperfine before running this script: https://github.com/sharkdp/hyperfine"
exit 1
fi

# Enable globstar for recursive directory listing
# This is for Bash 4.0 and later, which does not have recursive globbing enabled by default
shopt -s globstar

# Read all files in the benchmarks directory to warm up the file system cache
for i in {1..10}; do
cat ../yjit-bench/benchmarks/**/*.rb > /dev/null
cat test/testdata/**/*.rb > /dev/null
done

output_file_name="$(date '+%Y-%m-%d')-$(git rev-parse --short HEAD).json"

# ----- Run Benchmarks -----

run_benchmark() {
local title="$1"
local output_dir="$2"
local command="$3"

echo "#### $title ####"
output_file="prism_benchmarks/time/data/$output_dir/${output_file_name}"

hyperfine \
--warmup=10 --export-json="$output_file" --parameter-list parser sorbet,prism \
"bazel-bin/main/sorbet --parser={parser} $command" --ignore-failure
}

run_benchmark "Benchmark 1: yjit-bench, parser only" "parser/yjit-bench" "--stop-after=parser $YJIT_BENCH_DIR/benchmarks"
run_benchmark "Benchmark 2: sorbet-test, parser only" "parser/sorbet-tests" "--stop-after=parser test/testdata"
run_benchmark "Benchmark 3: prism regression tests, whole pipeline" "pipeline/prism_regression" "test/prism_regression"
run_benchmark "Benchmark 4: RBI gem, whole pipeline" "pipeline/rbi" "$RBI_DIR"
78 changes: 78 additions & 0 deletions test/BUILD
Original file line number Diff line number Diff line change
@@ -121,6 +121,21 @@ sh_test(
],
)

# Load the macro that creates a test for every file in the prism_regression folder
load("//test:prism_location_test.bzl", "prism_test_suite")

# Run the macro
prism_test_suite(
name = "prism_location_tests",
srcs = glob(["prism_regression/**/*.rb"]),
)

# Create a test suite to run all the location tests
test_suite(
name = "prism_location_tests",
tags = ["prism_location_test"],
)

cc_test(
name = "hello-test",
size = "small",
@@ -240,6 +255,49 @@ pipeline_tests(
"PosTests",
)

pipeline_tests(
"test_corpus_prism",
glob(
[
# Replace [parser] with other phases to test Prism at that level
# Phases: https://github.com/Shopify/sorbet/blob/prism/docs/internals.md#phases
"testdata/**/*.rb",
"testdata/**/*.exp",
],
exclude = [
# Tests having to do with tree differences in invalid Ruby code; will address later
"testdata/parser/error_recovery/**",
"testdata/parser/crash_block_pass_suggestion.rb",
"testdata/parser/invalid_fatal.rb",
"testdata/parser/invalid_syntax_error.rb",
"testdata/parser/kwargs_missing_comma.rb",
"testdata/parser/kwnil_errors.rb", # See https://github.com/Shopify/sorbet/issues/393
"testdata/parser/misc.rb",
"testdata/parser/offset0.rb",
"testdata/parser/ruby_25.rb",
"testdata/parser/kwargs.rb",
"testdata/lsp/completion/bad_arguments.rb",
"testdata/lsp/completion/bad_list_elems.rb",
"testdata/lsp/completion/case_1.rb",
"testdata/lsp/completion/case_2.rb",
"testdata/lsp/completion/eof.rb",
"testdata/lsp/completion/missing_const_name.rb",
"testdata/lsp/completion/missing_fun.rb",
"testdata/lsp/completion/self_receiver.rb",
"testdata/lsp/duplicate_kwarg.rb",
"testdata/namer/fuzz_repeated_kwarg.rb",
"testdata/deviations/keyword_method_names.rb",
"testdata/rewriter/minitest_empty_test_each.rb",
"testdata/infer/crash_after_parse_errors.rb",

# Sorbets' parser incorrectly accepts invalid syntax
"testdata/desugar/pattern_matching_hash.rb", # See https://github.com/Shopify/sorbet/issues/362
],
),
"PosTests",
parser = "prism",
)

pipeline_tests(
"whitequark_parser_corpus",
glob([
@@ -262,16 +320,36 @@ pipeline_tests(
extra_files = ["testdata/lsp/rubyfmt-stub/rubyfmt"],
)

pipeline_tests(
"prism_regression_corpus",
glob([
"prism_regression/**/*.rb",
"prism_regression/**/*.exp",
]),
"PosTests",
parser = "prism",
)

test_suite(
name = "test",
tests = ["test_corpus"],
)

test_suite(
name = "test_prism",
tests = ["test_corpus_prism"],
)

test_suite(
name = "whitequark_parser_tests",
tests = ["whitequark_parser_corpus"],
)

test_suite(
name = "prism_regression",
tests = ["prism_regression_corpus"],
)

sh_binary(
name = "single_package_runner",
srcs = ["test_single_package_runner.sh"],
15 changes: 15 additions & 0 deletions test/helpers/position_assertions.cc
Original file line number Diff line number Diff line change
@@ -58,6 +58,11 @@ template <typename T> bool isDuplicateDiagnostic(string_view filename, T *assert
template <typename T>
void reportMissingError(const string &filename, const T &assertion, string_view sourceLine, string_view errorPrefix,
bool missingDuplicate = false) {
// Skip error message checking when running with Prism.
if (sorbet::test::parser == realmain::options::Parser::PRISM) {
return;
}

auto coreMessage = missingDuplicate ? "Error was not duplicated" : "Did not find expected error";
auto messagePostfix = missingDuplicate ? "\nYou can fix this error by changing the assertion to `error:`." : "";
ADD_FAIL_CHECK_AT(filename.c_str(), assertion.range->start->line + 1,
@@ -68,9 +73,15 @@ void reportMissingError(const string &filename, const T &assertion, string_view

void reportUnexpectedError(const string &filename, const Diagnostic &diagnostic, string_view sourceLine,
string_view errorPrefix) {
// Skip error message checking when running with Prism.
if (sorbet::test::parser == realmain::options::Parser::PRISM) {
return;
}

auto diagnosticMessage = (diagnostic.severity == DiagnosticSeverity::Information)
? fmt::format("untyped: {}", diagnostic.message)
: fmt::format("error: {}", diagnostic.message);

ADD_FAIL_CHECK_AT(
filename.c_str(), diagnostic.range->start->line + 1,
fmt::format(
@@ -482,6 +493,10 @@ string ErrorAssertion::toString() const {
}

bool ErrorAssertion::check(const Diagnostic &diagnostic, string_view sourceLine, string_view errorPrefix) {
// Skip error message checking when running with Prism.
if (sorbet::test::parser == realmain::options::Parser::PRISM) {
return true;
}
// The error message must contain `message`.
if (diagnostic.message.find(message) == string::npos) {
ADD_FAIL_CHECK_AT(filename.c_str(), range->start->line + 1,
4 changes: 4 additions & 0 deletions test/helpers/position_assertions.h
Original file line number Diff line number Diff line change
@@ -9,6 +9,10 @@
namespace sorbet::test {
using namespace sorbet::realmain::lsp;

// TODO: Remove this before upstreaming
// This is needed to skip error assertions when running under Prism
extern realmain::options::Parser parser;

class ErrorAssertion;
class UntypedAssertion;

20 changes: 18 additions & 2 deletions test/pipeline_test.bzl
Original file line number Diff line number Diff line change
@@ -15,7 +15,7 @@ def dropExtension(p):
_TEST_SCRIPT = """#!/usr/bin/env bash
export ASAN_SYMBOLIZER_PATH=`pwd`/external/llvm_toolchain_15_0_7/bin/llvm-symbolizer
set -x
exec {runner} --single_test="{test}"
exec {runner} --single_test="{test}" --parser="{parser}"
"""

def _exp_test_impl(ctx):
@@ -24,6 +24,7 @@ def _exp_test_impl(ctx):
content = _TEST_SCRIPT.format(
runner = ctx.executable.runner.short_path,
test = ctx.file.test.path,
parser = ctx.attr.parser,
),
)

@@ -50,6 +51,7 @@ exp_test = rule(
"_llvm_symbolizer": attr.label(
default = "//test:llvm-symbolizer",
),
"parser": attr.string(default="sorbet"),
},
)

@@ -112,7 +114,7 @@ _TEST_RUNNERS = {
"PackagerTests": ":pipeline_test_runner",
}

def pipeline_tests(suite_name, all_paths, test_name_prefix, extra_files = [], tags = []):
def pipeline_tests(suite_name, all_paths, test_name_prefix, extra_files = [], tags = [], parser = "sorbet"):
tests = {} # test_name-> {"path": String, "prefix": String, "sentinel": String, "isPackage": bool}

# The packager step needs folder-based steps since folder structure dictates package membership.
@@ -140,6 +142,14 @@ def pipeline_tests(suite_name, all_paths, test_name_prefix, extra_files = [], ta
"disabled": "disabled" in test_name,
"isPackage": True,
}

# Tests that run with Prism parser need to have "_prism" appended to their name
# to differentiate them from the tests that run with Sorbet parser.
# The condition here is only for the packager tests
# Other tests are handled below.
if parser == "prism":
test_name = test_name + "_prism"

tests[test_name] = data
continue

@@ -148,6 +158,11 @@ def pipeline_tests(suite_name, all_paths, test_name_prefix, extra_files = [], ta

test_name = dirname(path) + "/" + prefix

# Tests that run with Prism parser need to have "_prism" appended to their name
# to differentiate them from the tests that run with Sorbet parser.
if parser == "prism":
test_name = test_name + "_prism"

current = tests.get(test_name)
if None == current:
data = {
@@ -205,6 +220,7 @@ def pipeline_tests(suite_name, all_paths, test_name_prefix, extra_files = [], ta
test = sentinel,
size = "small",
tags = tags + extra_tags,
parser = parser,
)

native.test_suite(
77 changes: 62 additions & 15 deletions test/pipeline_test_runner.cc
Original file line number Diff line number Diff line change
@@ -59,6 +59,7 @@ namespace sorbet::test {
using namespace std;

string singleTest;
realmain::options::Parser parser;

constexpr string_view whitelistedTypedNoneTest = "missing_typed_sigil.rb"sv;
constexpr string_view packageFileName = "__package.rb"sv;
@@ -84,15 +85,34 @@ class CFGCollectorAndTyper {
}
};

UnorderedSet<string> knownExpectations = {"parse-tree", "parse-tree-json", "parse-tree-whitequark",
"desugar-tree", "desugar-tree-raw", "rewrite-tree",
"rewrite-tree-raw", "index-tree", "index-tree-raw",
"symbol-table", "symbol-table-raw", "name-tree",
"name-tree-raw", "resolve-tree", "resolve-tree-raw",
"flatten-tree", "flatten-tree-raw", "cfg",
"cfg-raw", "cfg-text", "autogen",
"document-symbols", "package-tree", "document-formatting-rubyfmt",
"autocorrects", "minimized-rbi", "rbi-gen"};
UnorderedSet<string> knownExpectations = {"parse-tree",
"parse-tree-json",
"parse-tree-json-with-locs",
"parse-tree-whitequark",
"desugar-tree",
"desugar-tree-raw",
"rewrite-tree",
"rewrite-tree-raw",
"index-tree",
"index-tree-raw",
"symbol-table",
"symbol-table-raw",
"name-tree",
"name-tree-raw",
"resolve-tree",
"resolve-tree-raw",
"flatten-tree",
"flatten-tree-raw",
"cfg",
"cfg-raw",
"cfg-text",
"autogen",
"document-symbols",
"package-tree",
"document-formatting-rubyfmt",
"autocorrects",
"minimized-rbi",
"rbi-gen"};

ast::ParsedFile testSerialize(core::GlobalState &gs, ast::ParsedFile expr) {
auto &savedFile = expr.file.data(gs);
@@ -202,17 +222,26 @@ vector<ast::ParsedFile> index(unique_ptr<core::GlobalState> &gs, absl::Span<core
}

unique_ptr<parser::Node> nodes;
{
core::UnfreezeNameTable nameTableAccess(*gs); // enters original strings
switch (parser) {
case realmain::options::Parser::SORBET: {
std::cout << "Parsing with sorbet" << std::endl;
core::UnfreezeNameTable nameTableAccess(*gs); // enters original strings

auto settings = parser::Parser::Settings{};
nodes = parser::Parser::run(*gs, file, settings);
auto settings = parser::Parser::Settings{};
nodes = parser::Parser::run(*gs, file, settings);
break;
}
case realmain::options::Parser::PRISM:
std::cout << "Parsing with prism" << std::endl;
nodes = realmain::pipeline::runPrismParser(*gs, file, false, {});
break;
}

handler.drainErrors(*gs);
handler.addObserved(*gs, "parse-tree", [&]() { return nodes->toString(*gs); });
handler.addObserved(*gs, "parse-tree-whitequark", [&]() { return nodes->toWhitequark(*gs); });
handler.addObserved(*gs, "parse-tree-json", [&]() { return nodes->toJSON(*gs); });
handler.addObserved(*gs, "parse-tree-json-with-locs", [&]() { return nodes->toJSONWithLocs(*gs, file); });

// Desugarer
ast::ParsedFile desugared;
@@ -778,8 +807,15 @@ TEST_CASE("PerPhaseTest") { // NOLINT
gs->replaceFile(f.file, move(newFile));

// this replicates the logic of pipeline::indexOne
auto settings = parser::Parser::Settings{};
auto nodes = parser::Parser::run(*gs, f.file, settings);
unique_ptr<parser::Node> nodes;

if (parser == realmain::options::Parser::SORBET) {
auto settings = parser::Parser::Settings{};
nodes = parser::Parser::run(*gs, f.file, settings);
} else if (parser == realmain::options::Parser::PRISM) {
nodes = realmain::pipeline::runPrismParser(*gs, f.file, false, {});
}

handler.addObserved(*gs, "parse-tree", [&]() { return nodes->toString(*gs); });
handler.addObserved(*gs, "parse-tree-json", [&]() { return nodes->toJSON(*gs); });

@@ -870,6 +906,9 @@ int main(int argc, char *argv[]) {
cxxopts::Options options("test_corpus", "Test corpus for Sorbet typechecker");
options.allow_unrecognised_options().add_options()("single_test", "run over single test.",
cxxopts::value<std::string>()->default_value(""), "testpath");
options.allow_unrecognised_options().add_options()("parser", "The parser to use while testing.",
cxxopts::value<std::string>()->default_value("sorbet"),
"{prism, sorbet}");
auto res = options.parse(argc, argv);

if (res.count("single_test") != 1) {
@@ -879,6 +918,14 @@ int main(int argc, char *argv[]) {

sorbet::test::singleTest = res["single_test"].as<std::string>();

std::string parserOpt = res["parser"].as<std::string>();
for (auto &known : sorbet::realmain::options::parser_options) {
if (known.option == parserOpt) {
sorbet::test::parser = known.flag;
break;
}
}

doctest::Context context(argc, argv);
return context.run();
}
14 changes: 14 additions & 0 deletions test/prism_location_test.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
def prism_test_suite(name, srcs):
for src in srcs:
test_name = src.replace(".rb", "") + "_location_test"
native.sh_test(
name = test_name,
srcs = ["prism_location_test.sh"],
args = [src],
tags = ["prism_location_test"],
data = [
"//main:sorbet",
"@bazel_tools//tools/bash/runfiles",
src,
],
)
42 changes: 42 additions & 0 deletions test/prism_location_test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#!/bin/bash

set -euo pipefail

# Set up the runfiles
# shellcheck disable=SC1090
source "${RUNFILES_DIR:-/dev/null}/bazel_tools/tools/bash/runfiles/runfiles.bash"
if [[ ! -d "${RUNFILES_DIR:-/dev/null}" ]]; then
echo >&2 "ERROR: could not find runfiles directory"
exit 1
fi

# Locate the Sorbet binary
SORBET_BIN="$(rlocation com_stripe_ruby_typer/main/sorbet)"
TEST_DIR="$(rlocation com_stripe_ruby_typer/test)"

# The Ruby file to test is passed as the first argument
TEST_FILE="$1"

# Temporary files to hold parser outputs
sorbet_output_prism=$(mktemp)
sorbet_output_sorbet=$(mktemp)

set +e # Temporarily disable exit on error
# Run Sorbet with default parser
"$SORBET_BIN" --print=parse-tree-json-with-locs --parser=sorbet "$TEST_DIR/$TEST_FILE" > "$sorbet_output_sorbet" 2>/dev/null
# Run Sorbet with Prism parser
"$SORBET_BIN" --print=parse-tree-json-with-locs --parser=prism "$TEST_DIR/$TEST_FILE" > "$sorbet_output_prism" 2>/dev/null
set -e # Re-enable exit on error

# Compare outputs
if ! diff -q "$sorbet_output_prism" "$sorbet_output_sorbet" >/dev/null 2>&1; then
echo "Test failed for $TEST_FILE"
echo "Differences:"
diff -u "$sorbet_output_sorbet" "$sorbet_output_prism"
rm "$sorbet_output_prism" "$sorbet_output_sorbet"
exit 1
fi

# Clean up temporary files
rm "$sorbet_output_prism" "$sorbet_output_sorbet"
echo "Test passed for $TEST_FILE"
28 changes: 28 additions & 0 deletions test/prism_regression/alias.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
Begin {
stmts = [
Alias {
from = Symbol {
val = <U new_method1>
}
to = Symbol {
val = <U new_method1>
}
}
Alias {
from = Symbol {
val = <U new_method2>
}
to = Symbol {
val = <U new_method2>
}
}
Alias {
from = GVar {
name = <U $new_global>
}
to = GVar {
name = <U $old_global>
}
}
]
}
10 changes: 10 additions & 0 deletions test/prism_regression/alias.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# typed: false

alias new_method1 new_method1 # Works with bare references

alias :new_method2 :new_method2 # Works with Symbols

# `alias` can't be used with instance and class variables
# alias @new_ivar @old_ivar
# alias @@new_cvar @@old_cvar
alias $new_global $old_global # Works with global variables
140 changes: 140 additions & 0 deletions test/prism_regression/assign_to_class_variable.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
Begin {
stmts = [
Assign {
lhs = CVarLhs {
name = <U @@regular>
}
rhs = Integer {
val = "1"
}
}
OpAsgn {
left = CVarLhs {
name = <U @@bitwise_and>
}
op = <U &>
right = Integer {
val = "2"
}
}
OpAsgn {
left = CVarLhs {
name = <U @@bitwise_xor>
}
op = <U ^>
right = Integer {
val = "4"
}
}
OpAsgn {
left = CVarLhs {
name = <U @@shift_right>
}
op = <U >>>
right = Integer {
val = "5"
}
}
OpAsgn {
left = CVarLhs {
name = <U @@shift_left>
}
op = <U <<>
right = Integer {
val = "6"
}
}
OpAsgn {
left = CVarLhs {
name = <U @@subtract_assign>
}
op = <U ->
right = Integer {
val = "7"
}
}
OpAsgn {
left = CVarLhs {
name = <U @@module_assign>
}
op = <U %>
right = Integer {
val = "8"
}
}
OpAsgn {
left = CVarLhs {
name = <U @@bitwise_or>
}
op = <U |>
right = Integer {
val = "9"
}
}
OpAsgn {
left = CVarLhs {
name = <U @@divide_assign>
}
op = <U />
right = Integer {
val = "10"
}
}
OpAsgn {
left = CVarLhs {
name = <U @@multiply_assign>
}
op = <U *>
right = Integer {
val = "11"
}
}
OpAsgn {
left = CVarLhs {
name = <U @@exponentiate_assign>
}
op = <U **>
right = Integer {
val = "12"
}
}
AndAsgn {
left = CVarLhs {
name = <U @@lazy_and_assign>
}
right = Integer {
val = "13"
}
}
OrAsgn {
left = CVarLhs {
name = <U @@lazy_or_assgin>
}
right = Integer {
val = "14"
}
}
Masgn {
lhs = Mlhs {
exprs = [
CVarLhs {
name = <U @@target1>
}
CVarLhs {
name = <U @@target2>
}
]
}
rhs = Array {
elts = [
Integer {
val = "15"
}
Integer {
val = "16"
}
]
}
}
]
}
23 changes: 23 additions & 0 deletions test/prism_regression/assign_to_class_variable.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# typed: false

# Regular assignment
@@regular = 1

# Compound assignment operators
@@bitwise_and &= 2
@@bitwise_xor ^= 4
@@shift_right >>= 5
@@shift_left <<= 6
@@subtract_assign -= 7
@@module_assign %= 8
@@bitwise_or |= 9
@@divide_assign /= 10
@@multiply_assign *= 11
@@exponentiate_assign **= 12

# Special cases
@@lazy_and_assign &&= 13
@@lazy_or_assgin ||= 14

# Multi-target assignment
@@target1, @@target2 = 15, 16
691 changes: 691 additions & 0 deletions test/prism_regression/assign_to_constant.parse-tree.exp

Large diffs are not rendered by default.

96 changes: 96 additions & 0 deletions test/prism_regression/assign_to_constant.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# typed: false

# Regular assignment
REGULAR = 1

# Compound assignment operators
BITWISE_AND &= 2
BITWISE_XOR ^= 4
SHIFT_RIGHT >>= 5
SHIFT_LEFT <<= 6
SUBTRACT_ASSIGN -= 7
MODULE_ASSIGN %= 8
BITWISE_OR |= 9
DIVIDE_ASSIGN /= 10
MULTIPLY_ASSIGN *= 11
EXPONENTIATE_ASSIGN **= 12

# Special cases
LAZY_AND_ASSIGN &&= 13
LAZY_OR_ASSGIN ||= 14

# Multi-target assignment
TARGET1, TARGET2 = 29, 30

### Constant paths

# Regular assignment
ConstantPath::REGULAR = 15
::FullyQualified::ConstantPath::REGULAR = 16

# # Compound assignment operators
ConstantPath::BITWISE_AND &= 17
ConstantPath::BITWISE_XOR ^= 18
ConstantPath::SHIFT_RIGHT >>= 19
ConstantPath::SHIFT_LEFT <<= 20
ConstantPath::SUBTRACT_ASSIGN -= 21
ConstantPath::MODULE_ASSIGN %= 22
ConstantPath::BITWISE_OR |= 23
ConstantPath::DIVIDE_ASSIGN /= 24
ConstantPath::MULTIPLY_ASSIGN *= 25
ConstantPath::EXPONENTIATE_ASSIGN **= 26

# Special cases
ConstantPath::LAZY_AND_ASSIGN &&= 27
ConstantPath::LAZY_OR_ASSGIN ||= 28

# Multi-target assignment
ConstantPath::TARGET1, ConstantPath::TARGET2 = 31, 32
::ConstantPath::TARGET1, ::ConstantPath::TARGET2 = 33, 34

# Dynamic constant assignments

def method1
DynamicConstant = "These should all raise SyntaxErrors at runtime"

DynamicConstantBitwiseAnd &= 2
DynamicConstantBitwiseXor ^= 4
DynamicConstantShiftRight >>= 5
DynamicConstantShiftLeft <<= 6
DynamicConstantSubtractAssign -= 7
DynamicConstantModuleAssign %= 8
DynamicConstantBitwiseOr |= 9
DynamicConstantDivideAssign /= 10
DynamicConstantMultiplyAssign *= 11
DynamicConstantExponentiateAssign **= 12

# Special cases
DynamicConstantLazyAndAssign &&= 13
DynamicConstantLazyOrAssgin ||= 14

# Sorbet doesn't do the dynamic constant workaround for multi-target assignments
# DynamicConstantTarget1, DynamicConstantTarget2 = 35, 36
end

def method2
ConstantPath::DynamicConstant2 = "This should raise a SyntaxError at runtime"

# These should *NOT* raise a SyntaxError at runtime
ConstantPath::DynamicConstantBitwiseAnd &= 2
ConstantPath::DynamicConstantBitwiseXor ^= 4
ConstantPath::DynamicConstantShiftRight >>= 5
ConstantPath::DynamicConstantShiftLeft <<= 6
ConstantPath::DynamicConstantSubtractAssign -= 7
ConstantPath::DynamicConstantModuleAssign %= 8
ConstantPath::DynamicConstantBitwiseOr |= 9
ConstantPath::DynamicConstantDivideAssign /= 10
ConstantPath::DynamicConstantMultiplyAssign *= 11
ConstantPath::DynamicConstantExponentiateAssign **= 12

# And / Or assignment; still should not raise a syntax error
ConstantPath::DynamicConstantLazyAndAssign &&= 13
ConstantPath::DynamicConstantLazyOrAssgin ||= 14

# Sorbet doesn't do the dynamic constant workaround for multi-target assignments
# ConstantPath::DynamicConstantTarget1, ConstantPath::DynamicConstantTarget2 = 35, 36
end
140 changes: 140 additions & 0 deletions test/prism_regression/assign_to_global_variable.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
Begin {
stmts = [
Assign {
lhs = GVarLhs {
name = <U $regular>
}
rhs = Integer {
val = "1"
}
}
OpAsgn {
left = GVarLhs {
name = <U $bitwise_and>
}
op = <U &>
right = Integer {
val = "2"
}
}
OpAsgn {
left = GVarLhs {
name = <U $bitwise_xor>
}
op = <U ^>
right = Integer {
val = "4"
}
}
OpAsgn {
left = GVarLhs {
name = <U $shift_right>
}
op = <U >>>
right = Integer {
val = "5"
}
}
OpAsgn {
left = GVarLhs {
name = <U $shift_left>
}
op = <U <<>
right = Integer {
val = "6"
}
}
OpAsgn {
left = GVarLhs {
name = <U $subtract_assign>
}
op = <U ->
right = Integer {
val = "7"
}
}
OpAsgn {
left = GVarLhs {
name = <U $module_assign>
}
op = <U %>
right = Integer {
val = "8"
}
}
OpAsgn {
left = GVarLhs {
name = <U $bitwise_or>
}
op = <U |>
right = Integer {
val = "9"
}
}
OpAsgn {
left = GVarLhs {
name = <U $divide_assign>
}
op = <U />
right = Integer {
val = "10"
}
}
OpAsgn {
left = GVarLhs {
name = <U $multiply_assign>
}
op = <U *>
right = Integer {
val = "11"
}
}
OpAsgn {
left = GVarLhs {
name = <U $exponentiate_assign>
}
op = <U **>
right = Integer {
val = "12"
}
}
AndAsgn {
left = GVarLhs {
name = <U $lazy_and_assign>
}
right = Integer {
val = "13"
}
}
OrAsgn {
left = GVarLhs {
name = <U $lazy_or_assgin>
}
right = Integer {
val = "14"
}
}
Masgn {
lhs = Mlhs {
exprs = [
GVarLhs {
name = <U $target1>
}
GVarLhs {
name = <U $target2>
}
]
}
rhs = Array {
elts = [
Integer {
val = "15"
}
Integer {
val = "16"
}
]
}
}
]
}
23 changes: 23 additions & 0 deletions test/prism_regression/assign_to_global_variable.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# typed: false

# Regular assignment
$regular = 1

# Compound assignment operators
$bitwise_and &= 2
$bitwise_xor ^= 4
$shift_right >>= 5
$shift_left <<= 6
$subtract_assign -= 7
$module_assign %= 8
$bitwise_or |= 9
$divide_assign /= 10
$multiply_assign *= 11
$exponentiate_assign **= 12

# Special cases
$lazy_and_assign &&= 13
$lazy_or_assgin ||= 14

# Multi-target assignment
$target1, $target2 = 15, 16
713 changes: 713 additions & 0 deletions test/prism_regression/assign_to_index.parse-tree.exp

Large diffs are not rendered by default.

45 changes: 45 additions & 0 deletions test/prism_regression/assign_to_index.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# typed: false

# Regular assignment
regular[0] = 1

# Compound assignment operators
bitwise_and[0] &= 2
bitwise_xor[0] ^= 4
shift_right[0] >>= 5
shift_left[0] <<= 6
subtract_assign[0] -= 7
module_assign[0] %= 8
bitwise_or[0] |= 9
divide_assign[0] /= 10
multiply_assign[0] *= 11
exponentiate_assign[0] **= 12

# Special cases
lazy_and_assign[0] &&= 13
lazy_or_assign[0] ||= 14

# Multi-target assignment
target[0], target[1] = 15, 16
target[2, 3], target[4, 5] = 17, 18, 19, 20
target[] = 21 # Yes, this is valid. You can have `def []=(only_one_param)`.

# Assign to index with a block
# Note: this is technically not valid Ruby, but Prism allows it.

bitwise_and[&blk] &= 2
bitwise_xor[&blk] ^= 4
shift_right[&blk] >>= 5
shift_left[&blk] <<= 6
subtract_assign[&blk] -= 7
module_assign[&blk] %= 8
bitwise_or[&blk] |= 9
divide_assign[&blk] /= 10
multiply_assign[&blk] *= 11
exponentiate_assign[&blk] **= 12

lazy_and_assign[&blk] &&= 13
lazy_or_assign[&blk] ||= 14


target[&blk], target[1] = 4
140 changes: 140 additions & 0 deletions test/prism_regression/assign_to_instance_variable.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
Begin {
stmts = [
Assign {
lhs = IVarLhs {
name = <U @regular>
}
rhs = Integer {
val = "1"
}
}
OpAsgn {
left = IVarLhs {
name = <U @bitwise_and>
}
op = <U &>
right = Integer {
val = "2"
}
}
OpAsgn {
left = IVarLhs {
name = <U @bitwise_xor>
}
op = <U ^>
right = Integer {
val = "4"
}
}
OpAsgn {
left = IVarLhs {
name = <U @shift_right>
}
op = <U >>>
right = Integer {
val = "5"
}
}
OpAsgn {
left = IVarLhs {
name = <U @shift_left>
}
op = <U <<>
right = Integer {
val = "6"
}
}
OpAsgn {
left = IVarLhs {
name = <U @subtract_assign>
}
op = <U ->
right = Integer {
val = "7"
}
}
OpAsgn {
left = IVarLhs {
name = <U @module_assign>
}
op = <U %>
right = Integer {
val = "8"
}
}
OpAsgn {
left = IVarLhs {
name = <U @bitwise_or>
}
op = <U |>
right = Integer {
val = "9"
}
}
OpAsgn {
left = IVarLhs {
name = <U @divide_assign>
}
op = <U />
right = Integer {
val = "10"
}
}
OpAsgn {
left = IVarLhs {
name = <U @multiply_assign>
}
op = <U *>
right = Integer {
val = "11"
}
}
OpAsgn {
left = IVarLhs {
name = <U @exponentiate_assign>
}
op = <U **>
right = Integer {
val = "12"
}
}
AndAsgn {
left = IVarLhs {
name = <U @lazy_and_assign>
}
right = Integer {
val = "13"
}
}
OrAsgn {
left = IVarLhs {
name = <U @lazy_or_assgin>
}
right = Integer {
val = "14"
}
}
Masgn {
lhs = Mlhs {
exprs = [
IVarLhs {
name = <U @target1>
}
IVarLhs {
name = <U @target2>
}
]
}
rhs = Array {
elts = [
Integer {
val = "15"
}
Integer {
val = "16"
}
]
}
}
]
}
23 changes: 23 additions & 0 deletions test/prism_regression/assign_to_instance_variable.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# typed: false

# Regular assignment
@regular = 1

# Compound assignment operators
@bitwise_and &= 2
@bitwise_xor ^= 4
@shift_right >>= 5
@shift_left <<= 6
@subtract_assign -= 7
@module_assign %= 8
@bitwise_or |= 9
@divide_assign /= 10
@multiply_assign *= 11
@exponentiate_assign **= 12

# Special cases
@lazy_and_assign &&= 13
@lazy_or_assgin ||= 14

# Multi-target assignment
@target1, @target2 = 15, 16
140 changes: 140 additions & 0 deletions test/prism_regression/assign_to_local_variable.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
Begin {
stmts = [
Assign {
lhs = LVarLhs {
name = <U regular>
}
rhs = Integer {
val = "1"
}
}
OpAsgn {
left = LVarLhs {
name = <U bitwise_and>
}
op = <U &>
right = Integer {
val = "2"
}
}
OpAsgn {
left = LVarLhs {
name = <U bitwise_xor>
}
op = <U ^>
right = Integer {
val = "4"
}
}
OpAsgn {
left = LVarLhs {
name = <U shift_right>
}
op = <U >>>
right = Integer {
val = "5"
}
}
OpAsgn {
left = LVarLhs {
name = <U shift_left>
}
op = <U <<>
right = Integer {
val = "6"
}
}
OpAsgn {
left = LVarLhs {
name = <U subtract_assign>
}
op = <U ->
right = Integer {
val = "7"
}
}
OpAsgn {
left = LVarLhs {
name = <U module_assign>
}
op = <U %>
right = Integer {
val = "8"
}
}
OpAsgn {
left = LVarLhs {
name = <U bitwise_or>
}
op = <U |>
right = Integer {
val = "9"
}
}
OpAsgn {
left = LVarLhs {
name = <U divide_assign>
}
op = <U />
right = Integer {
val = "10"
}
}
OpAsgn {
left = LVarLhs {
name = <U multiply_assign>
}
op = <U *>
right = Integer {
val = "11"
}
}
OpAsgn {
left = LVarLhs {
name = <U exponentiate_assign>
}
op = <U **>
right = Integer {
val = "12"
}
}
AndAsgn {
left = LVarLhs {
name = <U lazy_and_assign>
}
right = Integer {
val = "13"
}
}
OrAsgn {
left = LVarLhs {
name = <U lazy_or_assgin>
}
right = Integer {
val = "14"
}
}
Masgn {
lhs = Mlhs {
exprs = [
LVarLhs {
name = <U target1>
}
LVarLhs {
name = <U target2>
}
]
}
rhs = Array {
elts = [
Integer {
val = "15"
}
Integer {
val = "16"
}
]
}
}
]
}
23 changes: 23 additions & 0 deletions test/prism_regression/assign_to_local_variable.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# typed: false

# Regular assignment
regular = 1

# Compound assignment operators
bitwise_and &= 2
bitwise_xor ^= 4
shift_right >>= 5
shift_left <<= 6
subtract_assign -= 7
module_assign %= 8
bitwise_or |= 9
divide_assign /= 10
multiply_assign *= 11
exponentiate_assign **= 12

# Special cases
lazy_and_assign &&= 13
lazy_or_assgin ||= 14

# Multi-target assignment
target1, target2 = 15, 16
228 changes: 228 additions & 0 deletions test/prism_regression/assign_to_method_result.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,228 @@
Begin {
stmts = [
Send {
receiver = Self {
}
method = <U m=>
args = [
Integer {
val = "1"
}
]
}
OpAsgn {
left = Send {
receiver = Self {
}
method = <U m>
args = [
]
}
op = <U &>
right = Integer {
val = "2"
}
}
OpAsgn {
left = Send {
receiver = Self {
}
method = <U m>
args = [
]
}
op = <U ^>
right = Integer {
val = "4"
}
}
OpAsgn {
left = Send {
receiver = Self {
}
method = <U m>
args = [
]
}
op = <U >>>
right = Integer {
val = "5"
}
}
OpAsgn {
left = Send {
receiver = Self {
}
method = <U m>
args = [
]
}
op = <U <<>
right = Integer {
val = "6"
}
}
OpAsgn {
left = Send {
receiver = Self {
}
method = <U m>
args = [
]
}
op = <U ->
right = Integer {
val = "7"
}
}
OpAsgn {
left = Send {
receiver = Self {
}
method = <U m>
args = [
]
}
op = <U %>
right = Integer {
val = "8"
}
}
OpAsgn {
left = Send {
receiver = Self {
}
method = <U m>
args = [
]
}
op = <U |>
right = Integer {
val = "9"
}
}
OpAsgn {
left = Send {
receiver = Self {
}
method = <U m>
args = [
]
}
op = <U />
right = Integer {
val = "10"
}
}
OpAsgn {
left = Send {
receiver = Self {
}
method = <U m>
args = [
]
}
op = <U *>
right = Integer {
val = "11"
}
}
OpAsgn {
left = Send {
receiver = Self {
}
method = <U m>
args = [
]
}
op = <U **>
right = Integer {
val = "12"
}
}
AndAsgn {
left = Send {
receiver = Self {
}
method = <U m>
args = [
]
}
right = Integer {
val = "13"
}
}
OrAsgn {
left = Send {
receiver = Self {
}
method = <U m>
args = [
]
}
right = Integer {
val = "14"
}
}
Masgn {
lhs = Mlhs {
exprs = [
Send {
receiver = Self {
}
method = <U target1=>
args = [
]
}
Send {
receiver = Self {
}
method = <U target2=>
args = [
]
}
]
}
rhs = Array {
elts = [
Integer {
val = "15"
}
Integer {
val = "16"
}
]
}
}
Masgn {
lhs = Mlhs {
exprs = [
CSend {
receiver = Self {
}
method = <U target1=>
args = [
]
}
CSend {
receiver = Self {
}
method = <U target2=>
args = [
]
}
]
}
rhs = Array {
elts = [
Integer {
val = "17"
}
Integer {
val = "18"
}
]
}
}
]
}
24 changes: 24 additions & 0 deletions test/prism_regression/assign_to_method_result.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# typed: false

# Regular assignment
self.m = 1

# Compound assignment operators
self.m &= 2
self.m ^= 4
self.m >>= 5
self.m <<= 6
self.m -= 7
self.m %= 8
self.m |= 9
self.m /= 10
self.m *= 11
self.m **= 12

# Special cases
self.m &&= 13
self.m ||= 14

# Multi-target assignment
self.target1, self.target2 = 15, 16
self&.target1, self&.target2 = 17, 18 # Not valid Ruby, but the parser needs to support it for the diagnostics to work
59 changes: 59 additions & 0 deletions test/prism_regression/assoc_splat.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
Begin {
stmts = [
Hash {
kwargs = false
pairs = [
Kwsplat {
expr = Send {
receiver = NULL
method = <U foo>
args = [
]
}
}
]
}
Hash {
kwargs = false
pairs = [
Pair {
key = Symbol {
val = <U a>
}
value = Integer {
val = "1"
}
}
Kwsplat {
expr = Send {
receiver = NULL
method = <U foo>
args = [
]
}
}
]
}
Hash {
kwargs = false
pairs = [
Kwsplat {
expr = Send {
receiver = NULL
method = <U foo>
args = [
]
}
}
Pair {
key = Symbol {
val = <U a>
}
value = Integer {
val = "1"
}
}
]
}
]
}
7 changes: 7 additions & 0 deletions test/prism_regression/assoc_splat.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# typed: false

{ **foo }

# with other hash keys
{ a: 1, **foo }
{ **foo, a: 1 }
57 changes: 57 additions & 0 deletions test/prism_regression/back_reference_read.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
Begin {
stmts = [
Backref {
name = <U $&>
}
Backref {
name = <U $`>
}
Backref {
name = <U $'>
}
Backref {
name = <U $+>
}
If {
condition = Send {
receiver = String {
val = <U foobar>
}
method = <U =~>
args = [
Regexp {
regex = [
String {
val = <U foo(.*)>
}
]
opts = Regopt {
opts = ""
}
}
]
}
then_ = Send {
receiver = NULL
method = <U puts>
args = [
DString {
nodes = [
String {
val = <U The last matching word was >
}
Begin {
stmts = [
Backref {
name = <U $+>
}
]
}
]
}
]
}
else_ = NULL
}
]
}
18 changes: 18 additions & 0 deletions test/prism_regression/back_reference_read.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# typed: false

# The result of the last match
$&

# The string preceding the match in the last match
$`

# The string following the match in the last match
$'

# The results of the highest-numbered group matched
$+

# In context
if "foobar" =~ /foo(.*)/ then
puts "The last matching word was #{$+}"
end
21 changes: 21 additions & 0 deletions test/prism_regression/begin.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
Begin {
stmts = [
Kwbegin {
stmts = [
Integer {
val = "1"
}
Integer {
val = "2"
}
Integer {
val = "3"
}
]
}
Kwbegin {
stmts = [
]
}
]
}
10 changes: 10 additions & 0 deletions test/prism_regression/begin.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# typed: false

begin
1
2
3
end

begin
end
26 changes: 26 additions & 0 deletions test/prism_regression/break.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Begin {
stmts = [
While {
cond = True {
}
body = Break {
exprs = [
Send {
receiver = NULL
method = <U foo>
args = [
]
}
]
}
}
While {
cond = True {
}
body = Break {
exprs = [
]
}
}
]
}
11 changes: 11 additions & 0 deletions test/prism_regression/break.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# typed: false

# with arguments
while true
break foo
end

# without arguments
while true
break
end
58 changes: 58 additions & 0 deletions test/prism_regression/call.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
Begin {
stmts = [
Send {
receiver = NULL
method = <U foo>
args = [
]
}
Send {
receiver = NULL
method = <U foo>
args = [
]
}
Send {
receiver = Send {
receiver = NULL
method = <U foo>
args = [
]
}
method = <U call>
args = [
]
}
Send {
receiver = Send {
receiver = NULL
method = <U foo>
args = [
]
}
method = <U []>
args = [
Symbol {
val = <U bar>
}
]
}
Send {
receiver = Send {
receiver = NULL
method = <U foo>
args = [
]
}
method = <U []=>
args = [
Symbol {
val = <U baz>
}
Integer {
val = "1"
}
]
}
]
}
7 changes: 7 additions & 0 deletions test/prism_regression/call.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# typed: false

foo
foo()
foo.()
foo[:bar]
foo[:baz] = 1
237 changes: 237 additions & 0 deletions test/prism_regression/call_all_params.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
Begin {
stmts = [
Send {
receiver = Send {
receiver = NULL
method = <U receiver>
args = [
]
}
method = <U foo>
args = [
Integer {
val = "1"
}
Integer {
val = "2"
}
Splat {
var = Array {
elts = [
Integer {
val = "3"
}
]
}
}
Hash {
kwargs = true
pairs = [
Pair {
key = Symbol {
val = <U d>
}
value = Integer {
val = "4"
}
}
Pair {
key = Symbol {
val = <U e>
}
value = Integer {
val = "5"
}
}
Pair {
key = Symbol {
val = <U f>
}
value = Integer {
val = "6"
}
}
]
}
BlockPass {
block = Send {
receiver = NULL
method = <U forwarded_block>
args = [
]
}
}
]
}
Block {
send = Send {
receiver = Send {
receiver = NULL
method = <U receiver>
args = [
]
}
method = <U foo>
args = [
Integer {
val = "1"
}
Integer {
val = "2"
}
Splat {
var = Array {
elts = [
Integer {
val = "3"
}
]
}
}
Hash {
kwargs = true
pairs = [
Pair {
key = Symbol {
val = <U d>
}
value = Integer {
val = "4"
}
}
Pair {
key = Symbol {
val = <U e>
}
value = Integer {
val = "5"
}
}
Pair {
key = Symbol {
val = <U f>
}
value = Integer {
val = "6"
}
}
]
}
]
}
args = Args {
args = [
Arg {
name = <U a>
}
Arg {
name = <U b>
}
Arg {
name = <U c>
}
Kwarg {
name = <U d>
}
Kwarg {
name = <U e>
}
Kwarg {
name = <U f>
}
Blockarg {
name = <U block>
}
]
}
body = String {
val = <U inline block>
}
}
Block {
send = Send {
receiver = Send {
receiver = NULL
method = <U receiver>
args = [
]
}
method = <U foo>
args = [
Integer {
val = "1"
}
Integer {
val = "2"
}
Splat {
var = Array {
elts = [
Integer {
val = "3"
}
]
}
}
Hash {
kwargs = true
pairs = [
Pair {
key = Symbol {
val = <U d>
}
value = Integer {
val = "4"
}
}
Pair {
key = Symbol {
val = <U e>
}
value = Integer {
val = "5"
}
}
Pair {
key = Symbol {
val = <U f>
}
value = Integer {
val = "6"
}
}
]
}
]
}
args = Args {
args = [
Arg {
name = <U a>
}
Arg {
name = <U b>
}
Arg {
name = <U c>
}
Kwarg {
name = <U d>
}
Kwarg {
name = <U e>
}
Kwarg {
name = <U f>
}
Blockarg {
name = <U block>
}
]
}
body = String {
val = <U do-end block>
}
}
]
}
9 changes: 9 additions & 0 deletions test/prism_regression/call_all_params.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# typed: false

receiver.foo(1, 2, *[3], d: 4, e: 5, f: 6, &forwarded_block)

receiver.foo(1, 2, *[3], d: 4, e: 5, f: 6) { |a, b, c, d:, e:, f:, &block| "inline block" }

receiver.foo(1, 2, *[3], d: 4, e: 5, f: 6) do |a, b, c, d:, e:, f:, &block|
"do-end block"
end
296 changes: 296 additions & 0 deletions test/prism_regression/call_block_param.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,296 @@
Begin {
stmts = [
Block {
send = Send {
receiver = NULL
method = <U foo>
args = [
]
}
args = NULL
body = NULL
}
Block {
send = Send {
receiver = NULL
method = <U foo>
args = [
]
}
args = NULL
body = NULL
}
Block {
send = Send {
receiver = NULL
method = <U foo>
args = [
]
}
args = NULL
body = String {
val = <U inline block>
}
}
Block {
send = Send {
receiver = NULL
method = <U foo>
args = [
]
}
args = NULL
body = String {
val = <U do-end block>
}
}
Block {
send = Send {
receiver = NULL
method = <U foo>
args = [
]
}
args = Args {
args = [
Arg {
name = <U positional>
}
Kwarg {
name = <U kwarg>
}
Blockarg {
name = <U block>
}
]
}
body = String {
val = <U inline block with params>
}
}
Block {
send = Send {
receiver = NULL
method = <U foo>
args = [
]
}
args = Args {
args = [
Arg {
name = <U positional>
}
Kwarg {
name = <U kwarg>
}
Blockarg {
name = <U block>
}
]
}
body = String {
val = <U inline block with params>
}
}
Block {
send = Send {
receiver = NULL
method = <U foo>
args = [
]
}
args = Args {
args = [
Arg {
name = <U positional>
}
Mlhs {
exprs = [
Arg {
name = <U multi>
}
Arg {
name = <U target>
}
]
}
]
}
body = String {
val = <U block with multi-target node in parameter list>
}
}
Block {
send = Send {
receiver = NULL
method = <U foo>
args = [
]
}
args = Args {
args = [
Arg {
name = <U bar>
}
Shadowarg {
name = <U baz>
}
Shadowarg {
name = <U qux>
}
]
}
body = NULL
}
Block {
send = Send {
receiver = NULL
method = <U foo>
args = [
]
}
args = Args {
args = [
Arg {
name = <U positional>
}
Mlhs {
exprs = [
Arg {
name = <U multi>
}
Arg {
name = <U target>
}
]
}
Shadowarg {
name = <U baz>
}
Shadowarg {
name = <U qux>
}
]
}
body = String {
val = <U block with multi-target node in parameter list and block locals>
}
}
Block {
send = Send {
receiver = NULL
method = <U foo>
args = [
]
}
args = Args {
args = [
Arg {
name = <U bar>
}
Shadowarg {
name = <U baz>
}
Shadowarg {
name = <U qux>
}
]
}
body = NULL
}
Send {
receiver = NULL
method = <U foo>
args = [
BlockPass {
block = Send {
receiver = NULL
method = <U forwarded_block>
args = [
]
}
}
]
}
Block {
send = CSend {
receiver = Send {
receiver = NULL
method = <U foo>
args = [
]
}
method = <U bar>
args = [
]
}
args = NULL
body = NULL
}
Block {
send = Send {
receiver = NULL
method = <U foo>
args = [
]
}
args = Args {
args = [
Mlhs {
exprs = [
Restarg {
name = <U args>
}
]
}
]
}
body = String {
val = <U block with multi-target rest args>
}
}
Block {
send = Send {
receiver = NULL
method = <U foo>
args = [
]
}
args = Args {
args = [
Restarg {
name = <U args>
}
]
}
body = String {
val = <U block with rest args>
}
}
Send {
receiver = NULL
method = <U def_delegators>
args = [
Symbol {
val = <U foo>
}
Hash {
kwargs = false
pairs = [
Pair {
key = Send {
receiver = NULL
method = <U local>
args = [
]
}
value = Symbol {
val = <U thing>
}
}
]
}
]
}
]
}
45 changes: 45 additions & 0 deletions test/prism_regression/call_block_param.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# typed: false

foo {} # empty inline block

foo do # empty do-end block
end

foo { "inline block" }

foo do
"do-end block"
end

foo { |positional, kwarg:, &block| "inline block with params" }

foo do |positional, kwarg:, &block|
"inline block with params"
end

foo do |positional, (multi, target)|
"block with multi-target node in parameter list"
end

# block with parameter `bar` and block locals `baz` and `qux`
foo { |bar; baz, qux| }

foo do |positional, (multi, target); baz, qux|
"block with multi-target node in parameter list and block locals"
end

foo { |bar; baz, qux| }

foo(&forwarded_block)

foo&.bar {}

foo do |(*args)|
"block with multi-target rest args"
end

foo do |*args|
"block with rest args"
end

def_delegators :foo, local => :thing
17 changes: 17 additions & 0 deletions test/prism_regression/call_forwarding_param.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
DefMethod {
name = <U foo>
args = Args {
args = [
ForwardArg {
}
]
}
body = Send {
receiver = NULL
method = <U bar>
args = [
ForwardedArgs {
}
]
}
}
6 changes: 6 additions & 0 deletions test/prism_regression/call_forwarding_param.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@

# typed: false

def foo(...)
bar(...)
end
53 changes: 53 additions & 0 deletions test/prism_regression/call_kw_rest_params.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
Begin {
stmts = [
DefMethod {
name = <U has_named_kwargs>
args = Args {
args = [
Kwrestarg {
name = <U kwargs>
}
]
}
body = Send {
receiver = NULL
method = <U delegate>
args = [
Hash {
kwargs = true
pairs = [
Kwsplat {
expr = LVar {
name = <U kwargs>
}
}
]
}
]
}
}
DefMethod {
name = <U has_anonymous_kwargs>
args = Args {
args = [
Kwrestarg {
name = <P <U **> $2>
}
]
}
body = Send {
receiver = NULL
method = <U delegate>
args = [
Hash {
kwargs = true
pairs = [
ForwardedKwrestArg {
}
]
}
]
}
}
]
}
9 changes: 9 additions & 0 deletions test/prism_regression/call_kw_rest_params.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# typed: false

def has_named_kwargs(**kwargs)
delegate(**kwargs)
end

def has_anonymous_kwargs(**)
delegate(**)
end
96 changes: 96 additions & 0 deletions test/prism_regression/call_required_kw_params.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
Begin {
stmts = [
Send {
receiver = NULL
method = <U foo>
args = [
Hash {
kwargs = true
pairs = [
Pair {
key = Symbol {
val = <U a>
}
value = Integer {
val = "1"
}
}
]
}
]
}
Send {
receiver = NULL
method = <U foo>
args = [
Hash {
kwargs = true
pairs = [
Pair {
key = Symbol {
val = <U a>
}
value = Integer {
val = "1"
}
}
]
}
]
}
Send {
receiver = NULL
method = <U bar>
args = [
Hash {
kwargs = true
pairs = [
Pair {
key = Symbol {
val = <U a>
}
value = Integer {
val = "2"
}
}
Pair {
key = Symbol {
val = <U b>
}
value = Integer {
val = "3"
}
}
]
}
]
}
Send {
receiver = NULL
method = <U bar>
args = [
Hash {
kwargs = true
pairs = [
Pair {
key = Symbol {
val = <U a>
}
value = Integer {
val = "2"
}
}
Pair {
key = Symbol {
val = <U b>
}
value = Integer {
val = "3"
}
}
]
}
]
}
]
}
9 changes: 9 additions & 0 deletions test/prism_regression/call_required_kw_params.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# typed: false

foo(a: 1)

foo a: 1

bar(a: 2, b: 3)

bar a: 2, b: 3
46 changes: 46 additions & 0 deletions test/prism_regression/call_required_params.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
Begin {
stmts = [
Send {
receiver = NULL
method = <U foo>
args = [
Integer {
val = "1"
}
]
}
Send {
receiver = NULL
method = <U foo>
args = [
Integer {
val = "1"
}
]
}
Send {
receiver = NULL
method = <U bar>
args = [
Integer {
val = "2"
}
Integer {
val = "3"
}
]
}
Send {
receiver = NULL
method = <U bar>
args = [
Integer {
val = "2"
}
Integer {
val = "3"
}
]
}
]
}
9 changes: 9 additions & 0 deletions test/prism_regression/call_required_params.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# typed: false

foo(1)

foo 1

bar(2, 3)

bar 2, 3
43 changes: 43 additions & 0 deletions test/prism_regression/call_rest_params.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
Begin {
stmts = [
DefMethod {
name = <U has_named_rest_args>
args = Args {
args = [
Restarg {
name = <U args>
}
]
}
body = Send {
receiver = NULL
method = <U delegate>
args = [
Splat {
var = LVar {
name = <U args>
}
}
]
}
}
DefMethod {
name = <U has_anonymous_rest_args>
args = Args {
args = [
Restarg {
name = <U *>
}
]
}
body = Send {
receiver = NULL
method = <U delegate>
args = [
ForwardedRestArg {
}
]
}
}
]
}
10 changes: 10 additions & 0 deletions test/prism_regression/call_rest_params.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@

# typed: false

def has_named_rest_args(*args)
delegate(*args)
end

def has_anonymous_rest_args(*)
delegate(*)
end
48 changes: 48 additions & 0 deletions test/prism_regression/call_with_receiver.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
Begin {
stmts = [
Send {
receiver = Send {
receiver = NULL
method = <U receiver>
args = [
]
}
method = <U foo>
args = [
]
}
Send {
receiver = Send {
receiver = NULL
method = <U receiver>
args = [
]
}
method = <U foo>
args = [
]
}
CSend {
receiver = Send {
receiver = NULL
method = <U receiver>
args = [
]
}
method = <U foo>
args = [
]
}
CSend {
receiver = Send {
receiver = NULL
method = <U receiver>
args = [
]
}
method = <U foo>
args = [
]
}
]
}
7 changes: 7 additions & 0 deletions test/prism_regression/call_with_receiver.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# typed: false

receiver.foo
receiver.foo()

receiver&.foo
receiver&.foo()
93 changes: 93 additions & 0 deletions test/prism_regression/case.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
Begin {
stmts = [
Case {
condition = Send {
receiver = NULL
method = <U foo>
args = [
]
}
whens = [
When {
patterns = [
Integer {
val = "1"
}
]
body = Send {
receiver = NULL
method = <U puts>
args = [
String {
val = <U one!>
}
]
}
}
When {
patterns = [
Integer {
val = "2"
}
Integer {
val = "3"
}
]
body = Send {
receiver = NULL
method = <U puts>
args = [
String {
val = <U two or three!>
}
]
}
}
]
else_ = Send {
receiver = NULL
method = <U puts>
args = [
String {
val = <U Who knows!>
}
]
}
}
Case {
condition = Send {
receiver = NULL
method = <U foo>
args = [
]
}
whens = [
When {
patterns = [
Const {
scope = NULL
name = <C <U Integer>>
}
]
body = Begin {
stmts = [
Integer {
val = "4"
}
Send {
receiver = NULL
method = <U puts>
args = [
String {
val = <U surprise, multi-line!>
}
]
}
]
}
}
]
else_ = NULL
}
]
}
17 changes: 17 additions & 0 deletions test/prism_regression/case.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# typed: false

case foo
when 1
puts "one!"
when 2, 3
puts "two or three!"
else
puts "Who knows!"
end

# no else
case foo
when Integer
4
puts "surprise, multi-line!"
end
971 changes: 971 additions & 0 deletions test/prism_regression/case_match.parse-tree.exp

Large diffs are not rendered by default.

83 changes: 83 additions & 0 deletions test/prism_regression/case_match.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# typed: false

case foo
in 1
puts "one!"
in 2
puts "two!"
in 3 | 4
puts "three or four!"
else
puts "Who knows!"
end

case array_like_thing
in [] => a
puts "empty!"
in [1, 2]
puts "one and two!"
in 3, 4 # An array pattern without [], but otherwise similar to the one above
puts "three and four!"
in [5, *]
puts "starts with five!"
in [*, 6]
puts "ends with six!"
in [*, 7, *] # A "find pattern"
puts "contains a seven!"
in Array[first, second] # Requires the `array_like_thing` to be an `Array` specifically
puts "An Array with first: #{first} and second: #{second}"
in Point[x, y] # Requires the `array_like_thing` to be a `Point` specifically
puts "A Point with x: #{x} and y: #{y}"
in [i,]
puts "An array with an element and maybe other stuff"
end

case hash_like_thing
in {}
puts "empty!"
in { a: 1, b: 2 } => h
puts "#{h} contains a and b, and maybe other stuff!"
in { c: 3, ** } => h
puts "#{h} has c, and maybe other stuff!"
in { d: 4, **nil } => h
puts "#{h} has d and nothing else!"
in {"kj": j} | {"kh": l} => m
puts "#{m} has j or l!"
in {"n1":, n2:, "n3":} => n4
puts "#{n4} has n1, n2, and n3!"
in Hash[e: 5 => e] # Requires the `hash_like_thing` to be a `Hash` specifically
puts "A Hash with e: #{e}"
in Point[x: 6 => x, y: 7 => y] # Requires the `hash_like_thing` to be a `Point` specifically
puts "A Point with x: #{x} and y: #{y}"
in **o
puts "splat!"
in **nil
puts "splat nil!"
end

# no else
case foo
in 1
"one!"
puts "surprise, multi-line!"
end

# pattern matching with if guards
case bar
in x if x == 1
"in with if"
in a, b if b == 2
"in with 2 args and if"
in c, d; c if c == 3
"in with 2 args, semicolon, and if"
end

# pattern matching with unless guards
case baz
in x unless x == 1
"in with unless"
in a, b unless b == 2
"in with 2 args and unless"
in c, d; c unless c == 3
"in with 2 args, semicolon, and unless"
end
284 changes: 284 additions & 0 deletions test/prism_regression/case_match_variable_binding.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,284 @@
Begin {
stmts = [
Assign {
lhs = LVarLhs {
name = <U lvar>
}
rhs = Integer {
val = "1"
}
}
CaseMatch {
expr = Send {
receiver = NULL
method = <U foo>
args = [
]
}
inBodies = [
InPattern {
pattern = ArrayPattern {
elts = [
MatchVar {
name = <U x>
}
]
}
guard = NULL
body = DString {
nodes = [
String {
val = <U An Array-like thing that only contains >
}
Begin {
stmts = [
LVar {
name = <U x>
}
]
}
]
}
}
InPattern {
pattern = HashPattern {
pairs = [
Pair {
key = Symbol {
val = <U k>
}
value = MatchVar {
name = <U x>
}
}
]
}
guard = NULL
body = DString {
nodes = [
String {
val = <U A Hash-like whose key `:k` has value >
}
Begin {
stmts = [
LVar {
name = <U x>
}
]
}
]
}
}
InPattern {
pattern = ArrayPattern {
elts = [
ArrayPattern {
elts = [
MatchVar {
name = <U value>
}
]
}
MatchRest {
var = MatchVar {
name = <U tail>
}
}
]
}
guard = NULL
body = DString {
nodes = [
String {
val = <U An array-like thing that starts with a one-element Array containing >
}
Begin {
stmts = [
LVar {
name = <U value>
}
]
}
String {
val = <U , and ends with >
}
Begin {
stmts = [
LVar {
name = <U tail>
}
]
}
]
}
}
InPattern {
pattern = HashPattern {
pairs = [
Pair {
key = Symbol {
val = <U k>
}
value = ArrayPattern {
elts = [
MatchVar {
name = <U value>
}
]
}
}
]
}
guard = NULL
body = DString {
nodes = [
String {
val = <U A hash-like whose key `:k` has a one-element Array value containing >
}
Begin {
stmts = [
LVar {
name = <U value>
}
]
}
]
}
}
InPattern {
pattern = ArrayPattern {
elts = [
HashPattern {
pairs = [
Pair {
key = Symbol {
val = <U k>
}
value = MatchVar {
name = <U value>
}
}
]
}
MatchRest {
var = MatchVar {
name = <U tail>
}
}
]
}
guard = NULL
body = DString {
nodes = [
String {
val = <U An array-like thing that starts with a one-element Hash containing >
}
Begin {
stmts = [
LVar {
name = <U value>
}
]
}
String {
val = <U , and ends with >
}
Begin {
stmts = [
LVar {
name = <U tail>
}
]
}
]
}
}
InPattern {
pattern = HashPattern {
pairs = [
Pair {
key = Symbol {
val = <U k>
}
value = HashPattern {
pairs = [
Pair {
key = Symbol {
val = <U k2>
}
value = MatchVar {
name = <U value>
}
}
]
}
}
]
}
guard = NULL
body = DString {
nodes = [
String {
val = <U A hash-like whose key `:k` has a one-element Hash value containing k2: >
}
Begin {
stmts = [
LVar {
name = <U value>
}
]
}
]
}
}
InPattern {
pattern = MatchAs {
value = Const {
scope = NULL
name = <C <U Integer>>
}
as = MatchVar {
name = <U i>
}
}
guard = NULL
body = DString {
nodes = [
String {
val = <U An Integer: >
}
Begin {
stmts = [
LVar {
name = <U i>
}
]
}
]
}
}
InPattern {
pattern = MatchVar {
name = <U x>
}
guard = NULL
body = DString {
nodes = [
String {
val = <U Some other value: >
}
Begin {
stmts = [
LVar {
name = <U x>
}
]
}
]
}
}
]
elseBody = NULL
}
]
}
26 changes: 26 additions & 0 deletions test/prism_regression/case_match_variable_binding.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# typed: false

lvar = 1

case foo
in [x] # Variable binding nested in an Array pattern
"An Array-like thing that only contains #{x}"
in { k: x } # Variable binding nested in a Hash pattern
"A Hash-like whose key `:k` has value #{x}"

in [[value], *tail] # Array pattern inside an Array pattern
"An array-like thing that starts with a one-element Array containing #{value}, and ends with #{tail}"
in { k: [value] } # Array pattern inside a Hash pattern
"A hash-like whose key `:k` has a one-element Array value containing #{value}"

in [{ k: value }, *tail] # A Hash pattern inside an Array pattern
"An array-like thing that starts with a one-element Hash containing #{value}, and ends with #{tail}"
in { k: { k2: value } } # A Hash pattern inside a Hash pattern
"A hash-like whose key `:k` has a one-element Hash value containing k2: #{value}"

in Integer => i # Pattern that also binds a variable
"An Integer: #{i}"

in x # Binds any value to a new variable `x`.
"Some other value: #{x}"
end
110 changes: 110 additions & 0 deletions test/prism_regression/case_match_variable_pinning.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
Begin {
stmts = [
Assign {
lhs = LVarLhs {
name = <U lvar>
}
rhs = Integer {
val = "1"
}
}
CaseMatch {
expr = Send {
receiver = NULL
method = <U foo>
args = [
]
}
inBodies = [
InPattern {
pattern = Pin {
var = LVar {
name = <U lvar>
}
}
guard = NULL
body = String {
val = <U Has the same value as the preexisting variable `x`>
}
}
InPattern {
pattern = Pin {
var = IVar {
name = <U @ivar>
}
}
guard = NULL
body = String {
val = <U Has the same value as `@ivar`>
}
}
InPattern {
pattern = Pin {
var = CVar {
name = <U @@cvar>
}
}
guard = NULL
body = String {
val = <U Has the same value as `@@cvar`>
}
}
InPattern {
pattern = Pin {
var = GVar {
name = <U $global>
}
}
guard = NULL
body = String {
val = <U Has the same value as `$global`>
}
}
InPattern {
pattern = Pin {
var = Begin {
stmts = [
Send {
receiver = Integer {
val = "1"
}
method = <U +>
args = [
Integer {
val = "2"
}
]
}
]
}
}
guard = NULL
body = String {
val = <U Has the same value as `1 + 2`>
}
}
InPattern {
pattern = MatchVar {
name = <U x>
}
guard = NULL
body = DString {
nodes = [
String {
val = <U Some other value: >
}
Begin {
stmts = [
LVar {
name = <U x>
}
]
}
]
}
}
]
elseBody = NULL
}
]
}
23 changes: 23 additions & 0 deletions test/prism_regression/case_match_variable_pinning.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# typed: false

lvar = 1

case foo

# "Variable pinning", which matches against the value of that variable
in ^lvar # `lvar` must already exist. Does *not* bind a new variable `lvar`, like `in x` below.
"Has the same value as the preexisting variable `x`"
in ^@ivar
"Has the same value as `@ivar`"
in ^@@cvar
"Has the same value as `@@cvar`"
in ^$global
"Has the same value as `$global`"

# "Expression pinning", which match the result of the expression
in ^(1 + 2)
"Has the same value as `1 + 2`"

in x # Binds any value to a new variable `x`.
"Some other value: #{x}"
end
59 changes: 59 additions & 0 deletions test/prism_regression/class.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
Begin {
stmts = [
Class {
name = Const {
scope = NULL
name = <C <U Parent>>
}
superclass = NULL
body = Begin {
stmts = [
Integer {
val = "1"
}
Integer {
val = "2"
}
Integer {
val = "3"
}
]
}
}
Class {
name = Const {
scope = NULL
name = <C <U Child>>
}
superclass = Const {
scope = NULL
name = <C <U Parent>>
}
body = NULL
}
SClass {
expr = Self {
}
body = Begin {
stmts = [
Integer {
val = "4"
}
Integer {
val = "5"
}
Integer {
val = "6"
}
]
}
}
SClass {
expr = Const {
scope = NULL
name = <C <U Parent>>
}
body = NULL
}
]
}
17 changes: 17 additions & 0 deletions test/prism_regression/class.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# typed: false

class Parent
1
2
3
end

class Child < Parent; end

class << self
4
5
6
end

class << Parent; end
28 changes: 28 additions & 0 deletions test/prism_regression/constants.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
Begin {
stmts = [
Const {
scope = NULL
name = <C <U SimpleConstant>>
}
Const {
scope = Const {
scope = NULL
name = <C <U Nested>>
}
name = <C <U Constant>>
}
Const {
scope = Cbase {
}
name = <C <U FullyQualifiedConstant>>
}
Const {
scope = Const {
scope = Cbase {
}
name = <C <U FullyQualified>>
}
name = <C <U NestedConstant>>
}
]
}
13 changes: 13 additions & 0 deletions test/prism_regression/constants.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# typed: false

SimpleConstant
#^^^^^^^^^^^^^^ error: Unable to resolve constant `SimpleConstant`

Nested::Constant
#^^^^^^ error: Unable to resolve constant `Nested`

::FullyQualifiedConstant
#^^^^^^^^^^^^^^^^^^^^^^^^ error: Unable to resolve constant `FullyQualifiedConstant`

::FullyQualified::NestedConstant
#^^^^^^^^^^^^^^^^ error: Unable to resolve constant `FullyQualified`
45 changes: 45 additions & 0 deletions test/prism_regression/def.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
Begin {
stmts = [
DefMethod {
name = <U foo>
args = NULL
body = NULL
}
DefMethod {
name = <U bar>
args = NULL
body = Begin {
stmts = [
String {
val = <U string1>
}
String {
val = <U string2>
}
]
}
}
DefMethod {
name = <U baz>
args = NULL
body = Kwbegin {
stmts = [
String {
val = <U string1>
}
String {
val = <U string2>
}
]
}
}
DefMethod {
name = <U qux>
args = Args {
args = [
]
}
body = NULL
}
]
}
17 changes: 17 additions & 0 deletions test/prism_regression/def.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# typed: false

def foo; end

def bar
"string1"
"string2"
end

def baz
begin
"string1"
"string2"
end
end

def qux(); end
56 changes: 56 additions & 0 deletions test/prism_regression/def_all_params.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
Begin {
stmts = [
DefMethod {
name = <U foo>
args = Args {
args = [
Arg {
name = <U a>
}
Optarg {
name = <U b>
default_ = Integer {
val = "2"
}
}
Restarg {
name = <U c>
}
Kwarg {
name = <U d>
}
Kwoptarg {
name = <U e>
default_ = Integer {
val = "5"
}
}
Kwrestarg {
name = <U f>
}
Blockarg {
name = <U blk>
}
]
}
body = NULL
}
DefMethod {
name = <U foo>
args = Args {
args = [
Restarg {
name = <U *>
}
Kwrestarg {
name = <P <U **> $2>
}
Blockarg {
name = <P <U &> $3>
}
]
}
body = NULL
}
]
}
5 changes: 5 additions & 0 deletions test/prism_regression/def_all_params.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# typed: false

def foo(a, b = 2, *c, d:, e: 5, **f, &blk); end

def foo(*, **, &); end
45 changes: 45 additions & 0 deletions test/prism_regression/def_block_param.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
Begin {
stmts = [
DefMethod {
name = <U foo>
args = Args {
args = [
Blockarg {
name = <U blk>
}
]
}
body = NULL
}
DefMethod {
name = <U foo>
args = Args {
args = [
Blockarg {
name = <P <U &> $2>
}
]
}
body = NULL
}
DefMethod {
name = <U foo>
args = Args {
args = [
Blockarg {
name = <U blk>
}
]
}
body = Super {
args = [
BlockPass {
block = LVar {
name = <U blk>
}
}
]
}
}
]
}
9 changes: 9 additions & 0 deletions test/prism_regression/def_block_param.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# typed: false

def foo(&blk); end

def foo(&); end

def foo(&blk)
super(&blk)
end
10 changes: 10 additions & 0 deletions test/prism_regression/def_forwarding_param.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
DefMethod {
name = <U foo>
args = Args {
args = [
ForwardArg {
}
]
}
body = NULL
}
4 changes: 4 additions & 0 deletions test/prism_regression/def_forwarding_param.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# typed: false

def foo(...)
end
36 changes: 36 additions & 0 deletions test/prism_regression/def_kw_rest_params.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
Begin {
stmts = [
DefMethod {
name = <U foo>
args = Args {
args = [
Kwrestarg {
name = <U a>
}
]
}
body = NULL
}
DefMethod {
name = <U foo>
args = Args {
args = [
Kwrestarg {
name = <P <U **> $2>
}
]
}
body = NULL
}
DefMethod {
name = <U foo>
args = Args {
args = [
Kwnilarg {
}
]
}
body = NULL
}
]
}
7 changes: 7 additions & 0 deletions test/prism_regression/def_kw_rest_params.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# typed: false

def foo(**a); end

def foo(**); end

def foo(**nil); end # Disallows keyword params, which is now the default in Ruby 3. Ruby 2 could might still have this.
38 changes: 38 additions & 0 deletions test/prism_regression/def_optional_kw_params.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
Begin {
stmts = [
DefMethod {
name = <U foo>
args = Args {
args = [
Kwoptarg {
name = <U a>
default_ = Integer {
val = "1"
}
}
]
}
body = NULL
}
DefMethod {
name = <U bar>
args = Args {
args = [
Kwoptarg {
name = <U a>
default_ = Integer {
val = "1"
}
}
Kwoptarg {
name = <U b>
default_ = Integer {
val = "2"
}
}
]
}
body = NULL
}
]
}
4 changes: 4 additions & 0 deletions test/prism_regression/def_optional_kw_params.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# typed: false

def foo(a: 1); end
def bar(a: 1, b: 2); end
38 changes: 38 additions & 0 deletions test/prism_regression/def_optional_params.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
Begin {
stmts = [
DefMethod {
name = <U foo>
args = Args {
args = [
Optarg {
name = <U a>
default_ = Integer {
val = "1"
}
}
]
}
body = NULL
}
DefMethod {
name = <U bar>
args = Args {
args = [
Optarg {
name = <U a>
default_ = Integer {
val = "1"
}
}
Optarg {
name = <U b>
default_ = Integer {
val = "2"
}
}
]
}
body = NULL
}
]
}
4 changes: 4 additions & 0 deletions test/prism_regression/def_optional_params.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# typed: false

def foo(a = 1); end
def bar(a = 1, b = 2); end
29 changes: 29 additions & 0 deletions test/prism_regression/def_required_kw_params.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
Begin {
stmts = [
DefMethod {
name = <U foo>
args = Args {
args = [
Kwarg {
name = <U a>
}
]
}
body = NULL
}
DefMethod {
name = <U bar>
args = Args {
args = [
Kwarg {
name = <U a>
}
Kwarg {
name = <U b>
}
]
}
body = NULL
}
]
}
4 changes: 4 additions & 0 deletions test/prism_regression/def_required_kw_params.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# typed: false

def foo(a:); end
def bar(a:, b:); end
29 changes: 29 additions & 0 deletions test/prism_regression/def_required_params.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
Begin {
stmts = [
DefMethod {
name = <U foo>
args = Args {
args = [
Arg {
name = <U a>
}
]
}
body = NULL
}
DefMethod {
name = <U bar>
args = Args {
args = [
Arg {
name = <U a>
}
Arg {
name = <U b>
}
]
}
body = NULL
}
]
}
4 changes: 4 additions & 0 deletions test/prism_regression/def_required_params.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# typed: false

def foo(a); end
def bar(a, b); end
43 changes: 43 additions & 0 deletions test/prism_regression/def_rest_params.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
Begin {
stmts = [
DefMethod {
name = <U foo>
args = Args {
args = [
Restarg {
name = <U a>
}
]
}
body = NULL
}
DefMethod {
name = <U foo>
args = Args {
args = [
Restarg {
name = <U *>
}
]
}
body = NULL
}
DefMethod {
name = <U foo>
args = Args {
args = [
Restarg {
name = <U a>
}
Arg {
name = <U b>
}
Arg {
name = <U c>
}
]
}
body = NULL
}
]
}
8 changes: 8 additions & 0 deletions test/prism_regression/def_rest_params.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# typed: false

def foo(*a); end

def foo(*); end

# Rest with posts
def foo(*a, b, c); end
43 changes: 43 additions & 0 deletions test/prism_regression/def_singleton.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
Begin {
stmts = [
DefS {
singleton = Self {
}
name = <U foo>
args = NULL
body = NULL
}
DefS {
singleton = Send {
receiver = NULL
method = <U x>
args = [
]
}
name = <U foo>
args = NULL
body = NULL
}
DefS {
singleton = Self {
}
name = <U foo>
args = Args {
args = [
Blockarg {
name = <U blk>
}
]
}
body = Super {
args = [
BlockPass {
block = LVar {
name = <U blk>
}
}
]
}
}
]
}
8 changes: 8 additions & 0 deletions test/prism_regression/def_singleton.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# typed: false

def self.foo; end
def x.foo; end # This is invalid

def self.foo(&blk)
super(&blk)
end
7 changes: 7 additions & 0 deletions test/prism_regression/def_with_body.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
DefMethod {
name = <U foo>
args = NULL
body = Integer {
val = "5"
}
}
5 changes: 5 additions & 0 deletions test/prism_regression/def_with_body.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# typed: false

def foo
5
end
72 changes: 72 additions & 0 deletions test/prism_regression/defined.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
Begin {
stmts = [
Assign {
lhs = LVarLhs {
name = <U local_var>
}
rhs = Integer {
val = "1"
}
}
Defined {
value = Const {
scope = Const {
scope = NULL
name = <C <U Constant>>
}
name = <C <U Path>>
}
}
Defined {
value = Const {
scope = Const {
scope = Const {
scope = Cbase {
}
name = <C <U FullyQualified>>
}
name = <C <U Constant>>
}
name = <C <U Path>>
}
}
Defined {
value = Send {
receiver = NULL
method = <U method_or_local_var>
args = [
]
}
}
Defined {
value = LVar {
name = <U local_var>
}
}
Defined {
value = IVar {
name = <U @ivar>
}
}
Defined {
value = CVar {
name = <U @@cvar>
}
}
Defined {
value = GVar {
name = <U $gvar>
}
}
Defined {
value = Yield {
exprs = [
]
}
}
Defined {
value = ZSuper {
}
}
]
}
12 changes: 12 additions & 0 deletions test/prism_regression/defined.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# typed: false
local_var = 1

defined?(Constant::Path)
defined?(::FullyQualified::Constant::Path)
defined?(method_or_local_var)
defined?(local_var)
defined?(@ivar)
defined?(@@cvar)
defined?($gvar)
defined?(yield)
defined?(super)
133 changes: 133 additions & 0 deletions test/prism_regression/ensure.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
Begin {
stmts = [
Kwbegin {
stmts = [
Ensure {
body = Send {
receiver = NULL
method = <U foo>
args = [
]
}
ensure = Send {
receiver = NULL
method = <U bar>
args = [
]
}
}
]
}
DefMethod {
name = <U method_with_ensure>
args = NULL
body = Ensure {
body = Begin {
stmts = [
String {
val = <U string1>
}
String {
val = <U string2>
}
]
}
ensure = String {
val = <U ensured>
}
}
}
DefMethod {
name = <U empty_method_with_ensure>
args = NULL
body = Ensure {
body = NULL
ensure = String {
val = <U ensured>
}
}
}
DefMethod {
name = <U method_with_begin_and_ensure>
args = NULL
body = Kwbegin {
stmts = [
Ensure {
body = Begin {
stmts = [
String {
val = <U string1>
}
String {
val = <U string2>
}
]
}
ensure = String {
val = <U ensured>
}
}
]
}
}
Kwbegin {
stmts = [
Ensure {
body = Begin {
stmts = [
String {
val = <U string1>
}
String {
val = <U string2>
}
]
}
ensure = Begin {
stmts = [
String {
val = <U ensured1>
}
String {
val = <U ensured2>
}
]
}
}
]
}
Kwbegin {
stmts = [
Ensure {
body = Rescue {
body = Begin {
stmts = [
String {
val = <U string1>
}
String {
val = <U string2>
}
]
}
rescue = [
Resbody {
exception = NULL
var = NULL
body = String {
val = <U rescued rescue>
}
}
]
else_ = String {
val = <U rescued else>
}
}
ensure = String {
val = <U ensure>
}
}
]
}
]
}
53 changes: 53 additions & 0 deletions test/prism_regression/ensure.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# typed: false

# Testing an ensure clause without a rescue or else clause
begin
foo
ensure
bar
end

# Testing an ensure clause in a method definition
def method_with_ensure
"string1"
"string2"
ensure
"ensured"
end

# Testing an ensure clause in an empty method definition
def empty_method_with_ensure
ensure
"ensured"
end

# Testing an ensure clause in a begin block in a method definition
def method_with_begin_and_ensure
begin
"string1"
"string2"
ensure
"ensured"
end
end

# Testing an ensure clause with multiple methods
begin
"string1"
"string2"
ensure
"ensured1"
"ensured2"
end

# Testing a rescue clause with an else and ensure clause
begin
"string1"
"string2"
rescue
"rescued rescue"
else
"rescued else"
ensure
"ensure"
end
19 changes: 19 additions & 0 deletions test/prism_regression/error_recovery/assign.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
DefMethod {
name = <U test_bad_assign>
args = Args {
args = [
Arg {
name = <U x>
}
]
}
body = Assign {
lhs = LVarLhs {
name = <U x>
}
rhs = Const {
scope = NULL
name = <C <U <ErrorNode>>>
}
}
}
6 changes: 6 additions & 0 deletions test/prism_regression/error_recovery/assign.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# typed: false
# Should still see at least method def (not body)
def test_bad_assign(x)
x =
# ^ error: expected an expression after `=`
end
96 changes: 96 additions & 0 deletions test/prism_regression/flip_flop.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
Begin {
stmts = [
If {
condition = IFlipflop {
left = String {
val = <U flip>
}
right = String {
val = <U flop>
}
}
then_ = String {
val = <U body 1>
}
else_ = NULL
}
If {
condition = EFlipflop {
left = String {
val = <U flip>
}
right = String {
val = <U flop>
}
}
then_ = String {
val = <U body 2>
}
else_ = NULL
}
If {
condition = IFlipflop {
left = String {
val = <U flip>
}
right = String {
val = <U flop>
}
}
then_ = String {
val = <U body 3>
}
else_ = NULL
}
If {
condition = IFlipflop {
left = String {
val = <U flip>
}
right = String {
val = <U flop>
}
}
then_ = NULL
else_ = String {
val = <U body 4>
}
}
If {
condition = IFlipflop {
left = String {
val = <U flip>
}
right = String {
val = <U flop>
}
}
then_ = NULL
else_ = String {
val = <U body 5>
}
}
Assign {
lhs = LVarLhs {
name = <U b>
}
rhs = Send {
receiver = Begin {
stmts = [
IFlipflop {
left = String {
val = <U flip>
}
right = String {
val = <U flop>
}
}
]
}
method = <U !>
args = [
]
}
}
]
}
26 changes: 26 additions & 0 deletions test/prism_regression/flip_flop.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# typed: false

if "flip".."flop" # 2 dots
#^^^^^^^^^^^^^^ error: Unsupported node type `IFlipflop`
"body 1"
end

if "flip"..."flop" # 3 dots
#^^^^^^^^^^^^^^^ error: Unsupported node type `EFlipflop`
"body 2"
end

"body 3" if "flip".."flop" # In a modifier
# ^^^^^^^^^^^^^^ error: Unsupported node type `IFlipflop`

unless "flip".."flop"
# ^^^^^^^^^^^^^^ error: Unsupported node type `IFlipflop`
"body 4"
end

"body 5" unless "flip".."flop" # In a modifier
# ^^^^^^^^^^^^^^ error: Unsupported node type `IFlipflop`

# In a boolean expression
b = !("flip".."flop")
# ^^^^^^^^^^^^^^ error: Unsupported node type `IFlipflop`
76 changes: 76 additions & 0 deletions test/prism_regression/float.parse-tree.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
Begin {
stmts = [
Float {
val = "-0.001"
}
Float {
val = "-1.602176634e-19"
}
Float {
val = "-1.732"
}
Float {
val = "-100.0"
}
Float {
val = "-2.71"
}
Float {
val = "-2.99792458e8"
}
Float {
val = "-42.01"
}
Float {
val = "-6.02"
}
Float {
val = "-9.81"
}
Float {
val = "-987.654"
}
Float {
val = "+0.577"
}
Float {
val = "+1.602176634e-19"
}
Float {
val = "+123.456"
}
Float {
val = "+3.14"
}
Float {
val = "+42.01"
}
Float {
val = "+42.123"
}
Float {
val = "+42.195"
}
Float {
val = "+6.67430e-11"
}
Float {
val = "0.0"
}
Float {
val = "1.618"
}
Float {
val = "2.71828"
}
Float {
val = "3.14159265359"
}
Float {
val = "3.333"
}
Float {
val = "5.333333"
}
]
}
26 changes: 26 additions & 0 deletions test/prism_regression/float.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# typed: false

-0.001
-1.602176634e-19
-1.732
-100.0
-2.71
-2.99792458e8
-42.01
-6.02
-9.81
-987.654
+0.577
+1.602176634e-19
+123.456
+3.14
+42.01
+42.123
+42.195
+6.67430e-11
0.0
1.618
2.71828
3.14159265359
3.333
5.333333
89 changes: 89 additions & 0 deletions test/prism_regression/for_loop.parse-tree.exp
19 changes: 19 additions & 0 deletions test/prism_regression/for_loop.rb
26 changes: 26 additions & 0 deletions test/prism_regression/if.parse-tree.exp
5 changes: 5 additions & 0 deletions test/prism_regression/if.rb
10 changes: 10 additions & 0 deletions test/prism_regression/if_else.parse-tree.exp
7 changes: 7 additions & 0 deletions test/prism_regression/if_else.rb
17 changes: 17 additions & 0 deletions test/prism_regression/if_elsif.parse-tree.exp
9 changes: 9 additions & 0 deletions test/prism_regression/if_elsif.rb
8 changes: 8 additions & 0 deletions test/prism_regression/if_empty_else.parse-tree.exp
6 changes: 6 additions & 0 deletions test/prism_regression/if_empty_else.rb
8 changes: 8 additions & 0 deletions test/prism_regression/if_with_stmt.parse-tree.exp
5 changes: 5 additions & 0 deletions test/prism_regression/if_with_stmt.rb
17 changes: 17 additions & 0 deletions test/prism_regression/if_with_stmts.parse-tree.exp
7 changes: 7 additions & 0 deletions test/prism_regression/if_with_stmts.rb
2 changes: 2 additions & 0 deletions test/prism_regression/keyword_ENCODING.parse-tree.exp
3 changes: 3 additions & 0 deletions test/prism_regression/keyword_ENCODING.rb
2 changes: 2 additions & 0 deletions test/prism_regression/keyword_FILE.parse-tree.exp
3 changes: 3 additions & 0 deletions test/prism_regression/keyword_FILE.rb
2 changes: 2 additions & 0 deletions test/prism_regression/keyword_LINE.parse-tree.exp
3 changes: 3 additions & 0 deletions test/prism_regression/keyword_LINE.rb
2 changes: 2 additions & 0 deletions test/prism_regression/keyword_false.parse-tree.exp
3 changes: 3 additions & 0 deletions test/prism_regression/keyword_false.rb
49 changes: 49 additions & 0 deletions test/prism_regression/keyword_it.parse-tree.exp
15 changes: 15 additions & 0 deletions test/prism_regression/keyword_it.rb
2 changes: 2 additions & 0 deletions test/prism_regression/keyword_nil.parse-tree.exp
3 changes: 3 additions & 0 deletions test/prism_regression/keyword_nil.rb
2 changes: 2 additions & 0 deletions test/prism_regression/keyword_redo.parse-tree.exp
4 changes: 4 additions & 0 deletions test/prism_regression/keyword_redo.rb
2 changes: 2 additions & 0 deletions test/prism_regression/keyword_retry.parse-tree.exp
3 changes: 3 additions & 0 deletions test/prism_regression/keyword_retry.rb
2 changes: 2 additions & 0 deletions test/prism_regression/keyword_self.parse-tree.exp
3 changes: 3 additions & 0 deletions test/prism_regression/keyword_self.rb
2 changes: 2 additions & 0 deletions test/prism_regression/keyword_true.parse-tree.exp
3 changes: 3 additions & 0 deletions test/prism_regression/keyword_true.rb
222 changes: 222 additions & 0 deletions test/prism_regression/lambda.parse-tree.exp
24 changes: 24 additions & 0 deletions test/prism_regression/lambda.rb
115 changes: 115 additions & 0 deletions test/prism_regression/literal_array.parse-tree.exp
23 changes: 23 additions & 0 deletions test/prism_regression/literal_array.rb
55 changes: 55 additions & 0 deletions test/prism_regression/literal_complex.parse-tree.exp
10 changes: 10 additions & 0 deletions test/prism_regression/literal_complex.rb
138 changes: 138 additions & 0 deletions test/prism_regression/literal_hash.parse-tree.exp
18 changes: 18 additions & 0 deletions test/prism_regression/literal_hash.rb
3 changes: 3 additions & 0 deletions test/prism_regression/literal_imaginary.parse-tree.exp
3 changes: 3 additions & 0 deletions test/prism_regression/literal_imaginary.rb
109 changes: 109 additions & 0 deletions test/prism_regression/literal_integer.parse-tree.exp
50 changes: 50 additions & 0 deletions test/prism_regression/literal_integer.rb
60 changes: 60 additions & 0 deletions test/prism_regression/literal_range.parse-tree.exp
13 changes: 13 additions & 0 deletions test/prism_regression/literal_range.rb
10 changes: 10 additions & 0 deletions test/prism_regression/literal_rational.parse-tree.exp
8 changes: 8 additions & 0 deletions test/prism_regression/literal_rational.rb
91 changes: 91 additions & 0 deletions test/prism_regression/literal_regexp.parse-tree.exp
18 changes: 18 additions & 0 deletions test/prism_regression/literal_regexp.rb
3 changes: 3 additions & 0 deletions test/prism_regression/literal_string.parse-tree.exp
3 changes: 3 additions & 0 deletions test/prism_regression/literal_string.rb
81 changes: 81 additions & 0 deletions test/prism_regression/literal_string_interpolation.parse-tree.exp
11 changes: 11 additions & 0 deletions test/prism_regression/literal_string_interpolation.rb
44 changes: 44 additions & 0 deletions test/prism_regression/literal_symbol.parse-tree.exp
15 changes: 15 additions & 0 deletions test/prism_regression/literal_symbol.rb
31 changes: 31 additions & 0 deletions test/prism_regression/local_variables.parse-tree.exp
7 changes: 7 additions & 0 deletions test/prism_regression/local_variables.rb
126 changes: 126 additions & 0 deletions test/prism_regression/match_last_line.parse-tree.exp
22 changes: 22 additions & 0 deletions test/prism_regression/match_last_line.rb
46 changes: 46 additions & 0 deletions test/prism_regression/match_write.parse-tree.exp
12 changes: 12 additions & 0 deletions test/prism_regression/match_write.rb
19 changes: 19 additions & 0 deletions test/prism_regression/module.parse-tree.exp
7 changes: 7 additions & 0 deletions test/prism_regression/module.rb
13 changes: 13 additions & 0 deletions test/prism_regression/multi_statements.parse-tree.exp
5 changes: 5 additions & 0 deletions test/prism_regression/multi_statements.rb
341 changes: 341 additions & 0 deletions test/prism_regression/multi_target.parse-tree.exp
25 changes: 25 additions & 0 deletions test/prism_regression/multi_target.rb
230 changes: 230 additions & 0 deletions test/prism_regression/multi_write.parse-tree.exp
24 changes: 24 additions & 0 deletions test/prism_regression/multi_write.rb
23 changes: 23 additions & 0 deletions test/prism_regression/next.parse-tree.exp
12 changes: 12 additions & 0 deletions test/prism_regression/next.rb
65 changes: 65 additions & 0 deletions test/prism_regression/numbered_params.parse-tree.exp
4 changes: 4 additions & 0 deletions test/prism_regression/numbered_params.rb
51 changes: 51 additions & 0 deletions test/prism_regression/numbered_reference_read.parse-tree.exp
12 changes: 12 additions & 0 deletions test/prism_regression/numbered_reference_read.rb
16 changes: 16 additions & 0 deletions test/prism_regression/operator_and.parse-tree.exp
5 changes: 5 additions & 0 deletions test/prism_regression/operator_and.rb
16 changes: 16 additions & 0 deletions test/prism_regression/operator_or.parse-tree.exp
5 changes: 5 additions & 0 deletions test/prism_regression/operator_or.rb
34 changes: 34 additions & 0 deletions test/prism_regression/parentheses.parse-tree.exp
7 changes: 7 additions & 0 deletions test/prism_regression/parentheses.rb
266 changes: 266 additions & 0 deletions test/prism_regression/pattern_matching_operators.parse-tree.exp
28 changes: 28 additions & 0 deletions test/prism_regression/pattern_matching_operators.rb
47 changes: 47 additions & 0 deletions test/prism_regression/post_execution.parse-tree.exp
20 changes: 20 additions & 0 deletions test/prism_regression/post_execution.rb
47 changes: 47 additions & 0 deletions test/prism_regression/pre_execution.parse-tree.exp
20 changes: 20 additions & 0 deletions test/prism_regression/pre_execution.rb
67 changes: 67 additions & 0 deletions test/prism_regression/range.parse-tree.exp
11 changes: 11 additions & 0 deletions test/prism_regression/range.rb
550 changes: 550 additions & 0 deletions test/prism_regression/rescue.parse-tree.exp
109 changes: 109 additions & 0 deletions test/prism_regression/rescue.rb
25 changes: 25 additions & 0 deletions test/prism_regression/return.parse-tree.exp
7 changes: 7 additions & 0 deletions test/prism_regression/return.rb
19 changes: 19 additions & 0 deletions test/prism_regression/shareable_constant.parse-tree.exp
4 changes: 4 additions & 0 deletions test/prism_regression/shareable_constant.rb
47 changes: 47 additions & 0 deletions test/prism_regression/super.parse-tree.exp
19 changes: 19 additions & 0 deletions test/prism_regression/super.rb
28 changes: 28 additions & 0 deletions test/prism_regression/undef.parse-tree.exp
7 changes: 7 additions & 0 deletions test/prism_regression/undef.rb
24 changes: 24 additions & 0 deletions test/prism_regression/unless.parse-tree.exp
7 changes: 7 additions & 0 deletions test/prism_regression/unless.rb
10 changes: 10 additions & 0 deletions test/prism_regression/unless_else.parse-tree.exp
7 changes: 7 additions & 0 deletions test/prism_regression/unless_else.rb
78 changes: 78 additions & 0 deletions test/prism_regression/until.parse-tree.exp
27 changes: 27 additions & 0 deletions test/prism_regression/until.rb
15 changes: 15 additions & 0 deletions test/prism_regression/variables_class.parse-tree.exp
5 changes: 5 additions & 0 deletions test/prism_regression/variables_class.rb
15 changes: 15 additions & 0 deletions test/prism_regression/variables_global.parse-tree.exp
5 changes: 5 additions & 0 deletions test/prism_regression/variables_global.rb
15 changes: 15 additions & 0 deletions test/prism_regression/variables_instance.parse-tree.exp
5 changes: 5 additions & 0 deletions test/prism_regression/variables_instance.rb
115 changes: 115 additions & 0 deletions test/prism_regression/while.parse-tree.exp
33 changes: 33 additions & 0 deletions test/prism_regression/while.rb
111 changes: 111 additions & 0 deletions test/prism_regression/xstring.parse-tree.exp
13 changes: 13 additions & 0 deletions test/prism_regression/xstring.rb
25 changes: 25 additions & 0 deletions test/prism_regression/yield.parse-tree.exp
7 changes: 7 additions & 0 deletions test/prism_regression/yield.rb
8 changes: 8 additions & 0 deletions third_party/externals.bzl
8 changes: 8 additions & 0 deletions third_party/prism.BUILD
2 changes: 1 addition & 1 deletion tools/scripts/build_compilation_db.sh
22 changes: 22 additions & 0 deletions tools/scripts/inspect_parsing_errors.rb
45 changes: 45 additions & 0 deletions tools/scripts/verify_prism_regression_tests.sh