Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Shopify/rbs's latest C parser implementation #426

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

st0012
Copy link
Member

@st0012 st0012 commented Feb 27, 2025

Motivation

This PR adopts the latest RBS parser implementation from Shopify/rbs with a major refactor.

Some key changes:

  1. Sorbet now needs to initialize a RBS global constant pool at the start of the process. Since Sorbet doesn't invoke its main in tests, we need to do it in its realmain module and the beginning of the testing pipeline
  2. The new SignatureTranslator now performs both RBS parsing + translation in its translateType and translateSignature functions
    • With this, rbs::Type and rbs::MethodType structs become redundant as we don't need an intermediate data type
  3. TypeTranslator and MethodTypeTranslator now store states like context and constant pool to reduce the number of parameters its functions need to keep passing down

Before

Unable to render rich display

Could not find a suitable point for the given distance

For more information, see https://docs.github.com/get-started/writing-on-github/working-with-advanced-formatting/creating-diagrams#creating-mermaid-diagrams

graph TD
    RBSSignaturesWalk -->|passes Comment| RBSParser
    RBSParser -->|returns MethodType or Type| RBSSignaturesWalk
    RBSParser -->|returns ParseError| RBSSignaturesWalk
    RBSSignaturesWalk -->|passes MethodType or Type| MethodTypeTranslator
    MethodTypeTranslator -->|uses Type| TypeTranslator

    subgraph Structs
        Comment
        ParseError
        MethodType
        Type
    end

    RBSSignaturesWalk --> Comment
    RBSParser --> Comment
    RBSParser --> ParseError
    RBSParser --> MethodType
    RBSParser --> Type
    MethodTypeTranslator --> MethodType
    MethodTypeTranslator --> Type
    TypeTranslator --> Type

After

Unable to render rich display

Could not find a suitable point for the given distance

For more information, see https://docs.github.com/get-started/writing-on-github/working-with-advanced-formatting/creating-diagrams#creating-mermaid-diagrams

graph TD
    RBSSignaturesWalk -->|creates| Comment
    RBSSignaturesWalk -->|passes Comment| SignatureTranslator
    SignatureTranslator -->|creates| Parser
    SignatureTranslator -->|uses| Comment
    SignatureTranslator -->|calls parseMethodType| Parser
    SignatureTranslator -->|calls parseType| Parser
    SignatureTranslator -->|passes Parser| MethodTypeTranslator
    MethodTypeTranslator -->|passes Parser| TypeTranslator

    subgraph Structs
        Comment
    end

    RBSSignaturesWalk --> Comment
    SignatureTranslator --> Comment

Test plan

See included automated tests.

@st0012 st0012 force-pushed the use-c-api-rbs branch 4 times, most recently from 50ab3a7 to e82cee7 Compare March 3, 2025 22:40
@st0012 st0012 marked this pull request as ready for review March 3, 2025 23:06
Copy link

@amomchilov amomchilov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That cleans up really nicely!

Comment on lines -110 to +124
rbs_constant_t *constant = rbs_constant_pool_id_to_constant(fake_constant_pool, node->name->constant_id);
string_view str(constant->start, constant->length);
auto str = parser.resolveConstant(node->name);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice.

Copy link

@Morriar Morriar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, I'm mostly concerned about the changes to realmain.cc

@st0012 st0012 force-pushed the use-c-api-rbs branch 2 times, most recently from 5bdbdcd to 47e7d0d Compare March 5, 2025 19:17
@st0012 st0012 requested a review from Morriar March 5, 2025 19:18
@st0012 st0012 force-pushed the use-c-api-rbs branch 2 times, most recently from ed2dc23 to e6a5c3a Compare March 5, 2025 21:34
st0012 and others added 7 commits March 6, 2025 20:37
- Create SignatureTranslator to simplify the translation of RBS signatures
- Make TypeTranslator and MethodTypeTranslator store parser state
- Remove RBSParser
The global constant pool is initialized at process startup and freed at
process exit.

With this change, we don't need to manually manage them in realmain.cc
and pipeline_test_runner.cc.

Co-authored-by: Alexander Momchilov <alexander.momchilov@shopify.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants