toAST incorrectly makes some nodes intermediate nodes

[rrthomas]

Using Ohm 17.1.0, and the following grammar:

Demo {
  Sequence = ListOf<Exp, ";">

  PrimaryExp = number

  UnaryExp
    = "~" UnaryExp    -- bitwise_not
    | PrimaryExp

  Exp = UnaryExp

  number
    = digit* "." digit+  -- fract
    | digit+             -- whole
}

I run the following code:

import {toAST} from 'ohm-js/extras'

// eslint-disable-next-line import/extensions
import grammar from './demo.ohm-bundle.js'

const matchResult = grammar.match('2')
console.dir(toAST(matchResult))

const matchResult2 = grammar.match('~2')
console.dir(toAST(matchResult2))

and the output is:

[ '2' ]
[ '2' ]

In other words, there's no difference between the ASTs for ~2 and for 2.

It looks to me as though Ohm is first discarding the node for the literal string "~", and then incorrectly deciding that the rule UnaryExp_bitwise_not has only one node, and hence considering it an intermediate node.

[cristiano-belloni]

I'm seeing a similar behaviour, where something like this (simplified):

ComparisonExp = 
  "a" "IN" aValues
 |"b" "IN" bValues

Gives you only the value of aValues or bValues when toAST is called, while I expect to see if the node is called on "a" or "b", and with which operator. Using mapping doesn't work at all, because ComparisonExp only maps to the 0-th place, if I try to access the 1st or 2nd I get an exception.

To be fair, why can't .toAST() generate a complete AST, without discarding literals? Apart from being buggy, it's really hard to read.