Choice between ignored and non-ignored parsers, when padded_by fails

[BinaryQuantumSoul]

I experience a recurring issue when trying to choice between some parsers that ouput () and others that do not.

Issue

For example, I have comments, that should be ignored and integers that should not.

let val = text::int::<_, extra::Err<Simple<char>>>(10);
let comment = just("//")
  .then(none_of('\n').repeated())
  .ignored();

Intuitively, if would use an or or a choice, but the types don't match:

let parser = val
  .or(comment)
  .padded()
  .repeated()
  .collect::<Vec<&str>>(); // compile error

It seems, from the examples, that the recommended way would be:

let parser = val
  .padded()
  .padded_by(comment.repeated())
  .repeated()
  .collect::<Vec<&str>>();

But, as you can see it breaks when we don't have any int at all, but only one comment.

assert_eq!(parser.parse("123 //abc").into_output(), Some(vec!["123"]));
assert!(parser.parse("//abc").has_errors()); // should not fail !

Workaround case 1

Right now, I have to use a workaround.
I map comments to a special Skip variant of my token enum and filter out the skips after parsing

#[derive(Debug, Clone, PartialEq)]
enum Token {
  Int(String),
  Skip,
}

let val = text::int::<_, extra::Err<Simple<char>>>(10)
  .from_str()
  .unwrapped()
  .map(Token::Int);
let comment = just("//")
  .then(none_of('\n').repeated())
  .to(Token::Skip);
let parser = val
  .or(comment)
  .padded()
  .repeated()
  .collect::<Vec<Token>>()
  .map(|v| {
    v.into_iter()
      .filter(|x| *x != Token::Skip)
      .collect::<Vec<Token>>()
  });

assert_eq!(parser.parse("123 //abc").into_output(), Some(vec![Token::Int("123".to_string())]));
assert_eq!(parser.parse("//abc").into_output(), Some(vec![]));

Workaround case 2

The workaround definitely won't work for everything, for example, let's say I want to replace, during parsing, occurrences of \\n with \n and remove occurrences of\\0.
This parser will not type check because first has char return type and second has ().

let parser = none_of::<_, _, extra::Err<Simple<char>>>('\\')
  .or(
    just('\\')
      .ignored()
      .then(just('n'))
      .to('\n')
      .or(just('0').ignored()), //compile error
  )
  .repeated()
  .collect::<String>();

The workaround here is even less elegant: convert everything to string and concatenate:

let parser = none_of::<_, _, extra::Err<Simple<char>>>('\\')
 .to_slice()
 .or(
   just('\\')
     .ignored()
     .then(just('n').to("\n").or(just('0').to("")))
     .map(|(_, s)| s),
 )
 .repeated()
 .collect::<Vec<_>>()
 .map(|v| v.concat());

assert_eq!(parser.parse("a\\n\\0b").into_output(), Some("a\nb".to_string()));

TL;DR

Basically I want a way to allow for choices or or to have ignored () parsers which are ignored in the case of a repeat().collect().
Is there a more idiomatic, less cumbersome way?

[BinaryQuantumSoul]

I found a way better workaround: add .map(some) for desired and .to(None) for ignored then map with flatten after collect.

For first example, we have:

let val = text::int::<_, extra::Err<Simple<char>>>(10);
let comment = just("//").then(none_of('\n').repeated());
let parser = val
  .map(Some)
  .or(comment.to(None))
  .padded()
  .repeated()
  .collect::<Vec<_>>()
  .map(|v| v.into_iter().flatten().collect());

assert_eq!(parser.parse("123 //abc").into_output(), Some(vec!["123"]));
assert_eq!(parser.parse("//abc").into_output(), Some(vec![]));

For second:

let parser = none_of::<_, _, extra::Err<Simple<char>>>('\\')
.map(Some)
.or(
  just('\\').ignore_then(
    just('n')
      .to('\n')
      .map(Some)
      .or(just('0').to(None)),
  ),
)
.repeated()
.collect::<Vec<_>>()
.map(|v| v.into_iter().flatten().collect());

assert_eq!(parser.parse("a\\n\\0b").into_output(), Some("a\nb".to_string()));

This is still quite verbose and I think chumsky would benefit form some sort of .skipped() and .collect_not_skipped() combinators to abstract this Option/flatten boilerplate.

[zesterer]

I think you're focussing in on one particular pattern too closely. Usually when you want to parse tokens in this way, it's as part of a group of tokens. So you might do something like the following:

let token = choice((
    ...
));

token
    // This will cause any comments to be ignored anywhere in the input
    .padded_by(comment.repeated())
    .repeated()
    .collect::<Vec<_>>()

[BinaryQuantumSoul]

I feel like you did not read my post ?
I have seen the padded_by method but it does not work, as shown in the post.
It created multiple errors on empty cases and that's exactly why I had to find an alternative.
I showed two scenarios where a better approach was needed.