Skip to content

DOT_INT,DOT_LIDENT,DOT_UIDENT These tokens do not match the source position of the source code #77

@illusory0x0

Description

@illusory0x0
fn only_col(tse : (@tokens.Token,@basic.Position,@basic.Position)) -> (@tokens.Token, Int,Int)  {
  (tse.0, tse.1.cnum, tse.2.cnum)
}

test {
  let tokens = @lexer.tokens_from_string("1",comment=false)
  inspect(tokens.tokens.map(only_col), content=(
    #|[(INT("1"), 0, 1), (EOF, 1, 1)]
  ))

  let tokens = @lexer.tokens_from_string(".1",comment=false)
  // range of DOT_INT is not corepsonding to the source code
  // here ignore `.` character. 
  inspect(tokens.tokens.map(only_col), content=(
    "[(DOT_INT(1), 1, 2), (EOF, 2, 2)]"
  ))
}

Metadata

Metadata

Assignees

Labels

wontfixThis will not be worked on

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions