New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should flatMap intersperse whitespace when using WhitespaceApi ? #88

Closed
denisrosset opened this Issue May 15, 2016 · 1 comment

Comments

Projects
None yet
2 participants
@denisrosset

denisrosset commented May 15, 2016

We want to parse the following: the first line contains the size, the second line an integer sequence of that size. We use WhitespaceApi to define spaces and tabs as whitespace; line endings are considered significant and have their own parser.

We first parse the size, then the repeated sequence of integers using flatMap. However, flatMap does not eat whitespace between the first parser and the second parser. Written without an explicit whitespace token (see Scala code below), the following will parse:

2
3 4

but not the following:

2
  3 4

Code below:

import fastparse.WhitespaceApi

object Test extends App {

  // whitespace contains spaces and tabs
  val White = WhitespaceApi.Wrapper{
    import fastparse.all._
    NoTrace(CharsWhile(" \t".contains(_)).?)
  }

  import White._
  import fastparse.noApi._

  // line endings

  val lineEnding: P[Unit] = P("\r".? ~ "\n")

  // non-negative integer
  val nnInt: P[Int] = P( CharIn('0'to'9').repX(1).!.map(_.toInt) )

  // sized sequence of integers, separated by whitespace
  def seqInt(n: Int): P[Seq[Int]] = nnInt.rep(min=n, max=n)

  // size followed by sized sequence
  val sizeAndSeqInt: P[Seq[Int]] = (nnInt ~ lineEnding).flatMap( n => seqInt(n) )

  val sizeAndSeqInt1: P[Seq[Int]] = (nnInt ~ lineEnding).flatMap( n => Pass ~ seqInt(n) )

  sizeAndSeqInt.parse("2\n3 4").get // Success

  sizeAndSeqInt1.parse("2\n 3 4").get // Success

  sizeAndSeqInt.parse("2\n 3 4").get // Failure

}

The behavior is different from the Scala parser combinators and should be documented (or the flatMap semantics changed, if it makes sense).

@lihaoyi

This comment has been minimized.

Show comment
Hide comment
@lihaoyi

lihaoyi Oct 18, 2018

Owner

fixed in 793e8eb

Owner

lihaoyi commented Oct 18, 2018

fixed in 793e8eb

@lihaoyi lihaoyi closed this Oct 18, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment