Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logos incorrectly lexing comma-separated list of numbers #286

Closed
bigyihsuan opened this issue Mar 30, 2023 · 2 comments
Closed

Logos incorrectly lexing comma-separated list of numbers #286

bigyihsuan opened this issue Mar 30, 2023 · 2 comments

Comments

@bigyihsuan
Copy link

I have an input [1,2]. My Token enum is as follows (truncated):

#[derive(Logos, Debug, Clone, PartialEq)]
pub enum Token {
    // ignore whitespace
    #[error]
    #[regex(r"[ \t\r\n\f]+", logos::skip)]
    Error,

    // ...

    // simple literals
    #[regex("[0-9]+", |lex| parse_int::parse::<i64>(lex.slice()))]
    IntLit(i64),
    #[regex("[0-9]+.[0-9]+", |lex| parse_int::parse::<f64>(lex.slice()))]
    DecLit(f64),

    // ...

    // symbols
    #[token("[")]
    LeftBracket,
    #[token("]")]
    RightBracket,

    // ...
}

However, it returns an Token::Error, saying that 1,2 isn't a valid token (obviously).

I was expecting something like

LeftBracket IntLit(1) Comma IntLit(2) RightBracket

How can I get Logos to lex [1,2] as the above output? Is this possibly related to #279 (specifically "[settling] on the last correctly matched pattern")?

@maciejhirsz
Copy link
Owner

You need to escape the period in your decimal, it matches every character in regex rules.

@bigyihsuan
Copy link
Author

You need to escape the period in your decimal, it matches every character in regex rules.

You are 100% correct, thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants