Transformer (non-inline) and v_args(tree=True) does nothing

[Qix-]

Using @v_args(tree=True) on a subclass of Transformer doesn't have any effect. No errors, nothing - it simply doesn't change any behavior (the children are still being passed as normal).

This is also the case with @v_args(meta=True) though this incantation causes an error to be raised - something about not being able to pass meta information to internal visitors (very cryptic error).

How do you get a non-inline Transformer subclass to receive Tree or Meta information?

[erezsh]

Your title mentions Transformer, but you say nothing about it in the text.

Visitor passes the tree by default, which lets you access everything, including the meta.

[erezsh]

Ok, nice ninja edit. Please paste the exact code you're using, that shows that it doesn't work.

[Qix-]

Yeah, sorry about that - caught it right after I posted.

Here's a repro case. This is using lark-parser from pypi:

import sys

from lark import Lark, Transformer, v_args

GRAMMAR = """
?start: foo
foo: BAR?
BAR: "bar"
"""

@v_args(meta=True)
class FooTransformer(Transformer):
    def foo(self, t, m=None):
        print(t, file=sys.stderr)
        print(m, file=sys.stderr)

parser = Lark(
    GRAMMAR,
    parser="lalr",
    transformer=FooTransformer(),
    propagate_positions=True,
)

print(parser.parse("bar"))

For meta=True I get

Traceback (most recent call last):
  File "test.py", line 21, in <module>
    propagate_positions=True,
  File "/private/tmp/test-lark-bug/env3/lib/python3.7/site-packages/lark/lark.py", line 221, in __init__
    self.parser = self._build_parser()
  File "/private/tmp/test-lark-bug/env3/lib/python3.7/site-packages/lark/lark.py", line 239, in _build_parser
    self._prepare_callbacks()
  File "/private/tmp/test-lark-bug/env3/lib/python3.7/site-packages/lark/lark.py", line 236, in _prepare_callbacks
    self._callbacks = self._parse_tree_builder.create_callback(self.options.transformer)
  File "/private/tmp/test-lark-bug/env3/lib/python3.7/site-packages/lark/parse_tree_builder.py", line 230, in create_callback
    assert not getattr(f, 'meta', False), "Meta args not supported for internal transformer"
AssertionError: Meta args not supported for internal transformer

and for tree=True I get

[Token(BAR, 'bar')]
None
None

[erezsh]

Meta args not supported for internal transformer

It means that you can't access .meta at that stage, because it's not ready yet.

If you want access to it, you'll have to use the transformer outside of the parser, using FooTransformer().transform(tree)

and for tree=True I get

In latest version:

$ python issue408.py
Tree(foo, [Token(BAR, 'bar')])

Make sure you're upgraded to the latest.

[Qix-]

I have the latest on pypi, perhaps it's outdated? Version that pip pulled down is lark-parser==0.7.1. I'm using Python 3, if that makes any difference.

---

At any rate, the meta trick worked ^^ Thanks for the quick response.