Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Crossandra.tokenize return type #15

Open
trag1c opened this issue Aug 14, 2023 · 0 comments
Open

Improve Crossandra.tokenize return type #15

trag1c opened this issue Aug 14, 2023 · 0 comments
Labels
help wanted Contributions especially welcome improvement Code/documentation/DX improvements

Comments

@trag1c
Copy link
Owner

trag1c commented Aug 14, 2023

Right now the method is declared to return list[Enum | Any] where:

  • Enum refers to the token_source enum members (probably not specific to the one specified in the __init__?)
  • Any refers to anything that the rules supplied to the tokenizer return

It feels like both things could be improved with generics.

Here are some (hopefully relevant) highlights of their origin:

def invert_enum(enum: type[Enum]) -> dict[str, Enum]:
    ...


class Empty(Enum):
    pass


class Crossandra:

    def __init__(
        self,
        token_source: type[Enum] = Empty,
        *,
        ...
        rules: list[Rule[Any] | RuleGroup] | None = None,
        ...
    ) -> None:
        self.__rules: list[Rule[Any]] = []
        for r in rules or []:
            if isinstance(r, RuleGroup):
                self.__rules.extend(r)
            else:
                self.__rules.append(r)
        ...
        self.__tokens = invert_enum(token_source)
        ...

    def tokenize(self, code: str) -> list[Enum | Any]:
        ...

Solving #16 is likely to help here.

@trag1c trag1c added the feature New feature or request label Aug 14, 2023
@trag1c trag1c added this to the Crossandra 2.1.0 milestone Aug 14, 2023
@trag1c trag1c added help wanted Contributions especially welcome improvement Code/documentation/DX improvements and removed feature New feature or request labels May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Contributions especially welcome improvement Code/documentation/DX improvements
Projects
None yet
Development

No branches or pull requests

1 participant