Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GDScript 2.0 - multiline lambdas not supported #191

Closed
ngotoandev opened this issue Dec 25, 2022 · 31 comments
Closed

GDScript 2.0 - multiline lambdas not supported #191

ngotoandev opened this issue Dec 25, 2022 · 31 comments
Labels
bug Something isn't working formatter parser

Comments

@ngotoandev
Copy link

ngotoandev commented Dec 25, 2022

This is flagged as error on gd-format but completely legit syntax in GDScript 2.0:

func f():
  return func():
    pass

Error:

return func():
                      ^

Unexpected token Token('_NL', '\n\t\t') at line 11, column 16.
Expected one of:
        * NOT
        * HEX
        * BANG
        * BIN
        * VAR
        * AWAIT
        * PERCENT
        * NUMBER
        * LPAR
        * LBRACE
        * REGULAR_STRING
        * CIRCUMFLEX
        * TILDE
        * LSQB
        * RETURN
        * NAME
        * LONG_STRING
        * DOLLAR
        * MINUS
        * FUNC
        * PLUS
        * AMPERSAND
        * PASS
@Scony Scony changed the title Godot 4 Lambda functions returning lambda GDScript 2.0 - multiline lambdas not supported Dec 27, 2022
@Scony Scony added bug Something isn't working formatter parser labels Dec 27, 2022
@Scony
Copy link
Owner

Scony commented Dec 27, 2022

Multiline lambdas are currently not supported. They won't be anytime soon as it requires writing a custom lexer to avoid ambiguity on the parser level.

@halcaponey
Copy link

They won't be anytime soon as it requires writing a custom lexer to avoid ambiguity on the parser level.

Just out of curiosity, what is the ambiguity?

@Scony
Copy link
Owner

Scony commented Jan 30, 2023

As far as I remember the clash was on newlines/indent/dedent. The key thing is to get proper indent/dedent when in ([{.

@Scony Scony modified the milestones: 4.x, 4.1 Feb 7, 2023
@Scony Scony modified the milestones: 4.1, 4.x Mar 1, 2023
@ogrady
Copy link

ogrady commented Aug 6, 2023

So what is the current way to work around this? As this seems to be a lexer error, we can't really gdlint: disable, I assume.
Is there a way to exempt blocks of code from the whole linting process?

@Scony
Copy link
Owner

Scony commented Aug 7, 2023

So what is the current way to work around this? As this seems to be a lexer error, we can't really gdlint: disable, I assume. Is there a way to exempt blocks of code from the whole linting process?

You can workaround it by having lambda returning one bing expression, or you can use ; to separate statements. See: https://github.com/Scony/godot-gdscript-toolkit/blob/master/tests/formatter/input-output-pairs/long-inline-lambdas.out.gd

As for exempting blocks of code from the whole linting process - it's not possible atm.

@ogrady
Copy link

ogrady commented Aug 7, 2023

Thanks for letting me know, and great work providing this toolkit btw. 👍

@iandoesallthethings
Copy link

Any movement on this bug? I've got a working 2d grid iterator pattern going, but formatting breaks and throws errors any time I use it, which is a bummer.

func damage_all(attack):
	for element in attack.keys():
		var elemental_damage = func(space, enemy):
				if enemy == null: return
				enemy.damage(attack[element], element)
				if enemy.current_health <= 0:
					$enemies.destroy_entity_at(space)
		$enemies.for_each(elemental_damage)

@Scony
Copy link
Owner

Scony commented Oct 16, 2023

Not yet.

@m21-cerutti
Copy link

m21-cerutti commented Dec 10, 2023

Same problem here on a maybe more tricky problem

var l_add_and_print := func(root_path, name, is_file):
		if is_file:
			var path = root_path+"/"+name
			gut.p(path)
			paths.append(path)

Error:

print := func(root_path, name, is_file):
                                        ^

Unexpected token Token('_NL', '\n\t\t') at line 8, column 56.
Expected one of: 
	* DOLLAR
	* VAR
	* PERCENT
	* NAME
	* FUNC
	* NUMBER
	* PLUS
	* AMPERSAND
	* RETURN
	* HEX
	* NOT
	* PASS
	* LPAR
	* BIN
	* LONG_STRING
	* TILDE
	* LSQB
	* AWAIT
	* CIRCUMFLEX
	* REGULAR_STRING
	* LBRACE
	* BANG
	* MINUS

Disable the linter is ok but we can't disable the formatter for this problem.
Also if we have a "if", can't make the trick to do all in the same line
Unexpected token Token('COLON', ':') at line 11, column 66.

@tavurth
Copy link

tavurth commented Jan 9, 2024

	camera.tween.finished.connect(func():
		camera.tween.stop()

		# Switch to linear tween for a smoother follow
		camera.tween\
			.tween_method(set_pos, 0.0, 1.0, time)\
			.set_trans(Tween.TRANS_LINEAR)

		camera.tween.play()
	)

@Skyway666
Copy link

I can understand not wanting to support Multiline Lambdas if it implies too much work (I've never build a linter or formatter) since a simple workaround is creating a separate function for the callback.

However, would it be much harder to work on handling the exception? When parsing multiple files the user doesn't even know which one is causing the error, and it's hard to deduce why it is happening. An error message as the following would be helpful:

Error in "sample.gd" line 32: gdscript toolkit doesn't support Multiline Lambda functions, please create a separate function to assign as callback.

At least it will prevent users from writing more duplicate issues about the matter :)

@Scony
Copy link
Owner

Scony commented Jan 19, 2024

@Skyway666 that's a good point - I can try exploring that. It's definitely not as simple as writing a single if statement or so, but maybe with some more conditions, it will be possible to give such helpful error in, say, 80% of situations. That would help a lot, I agree.

@tavurth
Copy link

tavurth commented Jan 20, 2024

@Scony this would help so much!

Currently it can be hard to figure out even what type of error is going on.

You have to delete the file in halves to try and figure out where the error is coming from.

@MikeSchulze
Copy link

MikeSchulze commented Jan 23, 2024

Hi, i run into same issue today, but i got a more hard error.
Using Python 3.12.1 and gdlint latest master

$ gdlint example.gd
Traceback (most recent call last):
  File "D:\development\tools\pyton\Lib\site-packages\lark\lexer.py", line 590, in lex
    yield lexer.next_token(lexer_state, parser_state)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\lexer.py", line 528, in next_token
    raise UnexpectedCharacters(lex_state.text, line_ctr.char_pos, line_ctr.line, line_ctr.column,
lark.exceptions.UnexpectedCharacters: <exception str() failed>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\linter\__main__.py", line 134, in _lint_file
    problems = lint_code(content, config)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\linter\__init__.py", line 118, in lint_code
    parse_tree = parser.parse(gdscript_code, gather_metadata=True)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\parser\parser.py", line 62, in parse
    self._parser_with_metadata.parse(adjusted_code)
  File "D:\development\tools\pyton\Lib\site-packages\lark\lark.py", line 645, in parse
    return self.parser.parse(text, start=start, on_error=on_error)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\parser_frontends.py", line 96, in parse
    return self.parser.parse(stream, chosen_start, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\parsers\lalr_parser.py", line 41, in parse
    return self.parser.parse(lexer, start)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\parsers\lalr_parser.py", line 171, in parse
    return self.parse_from_state(parser_state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\parsers\lalr_parser.py", line 193, in parse_from_state
    raise e
  File "D:\development\tools\pyton\Lib\site-packages\lark\parsers\lalr_parser.py", line 183, in parse_from_state
    for token in state.lexer.lex(state):
  File "D:\development\tools\pyton\Lib\site-packages\lark\indenter.py", line 45, in _process
    for token in stream:
  File "D:\development\tools\pyton\Lib\site-packages\lark\lexer.py", line 599, in lex
    raise UnexpectedToken(token, e.allowed, state=parser_state, token_history=[last_token], terminals_by_name=self.root_lexer.terminals_by_name)
lark.exceptions.UnexpectedToken: <exception str() failed>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\common\exceptions.py", line 14, in lark_unexpected_token_to_str
    return f"{exception.get_context(code)}\n{exception}"
                                            ^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\exceptions.py", line 256, in __str__
    % (self.token, self.line, self.column, self._format_expected(self.accepts or self.expected)))
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\exceptions.py", line 142, in _format_expected
    expected = [d[t_name].user_repr() if t_name in d else t_name for t_name in expected]
                ^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\lexer.py", line 124, in user_repr
    return self.pattern.raw or self.name
           ^^^^^^^^^^^^^^^^
AttributeError: 'PatternStr' object has no attribute 'raw'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "D:\development\tools\pyton\Scripts\gdlint.exe\__main__.py", line 7, in <module>
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\linter\__main__.py", line 66, in main
    problems_total += _lint_file(file_path, config)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\linter\__main__.py", line 148, in _lint_file
    lark_unexpected_token_to_str(exception, content),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\common\exceptions.py", line 16, in lark_unexpected_token_to_str
    return f"{exception}".strip()
             ^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\exceptions.py", line 256, in __str__
    % (self.token, self.line, self.column, self._format_expected(self.accepts or self.expected)))
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\exceptions.py", line 142, in _format_expected
    expected = [d[t_name].user_repr() if t_name in d else t_name for t_name in expected]
                ^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\lexer.py", line 124, in user_repr
    return self.pattern.raw or self.name
           ^^^^^^^^^^^^^^^^
AttributeError: 'PatternStr' object has no attribute 'raw'

The example.gd


func contains_keys(expected :Array):
	var keys_not_found :Array = expected.filter(func(key):
		prints("key:", key)
		return true
	)

The only workaround is to move the inline lambda to a function and use it.
The question remains, will this bug be fixed or will there at least be a flag to exclude such code blocks from the linter like
# gdlint:disable=lambda

@Scony
Copy link
Owner

Scony commented Jan 23, 2024

@MikeSchulze since it does happen on the parser level, the # gdlint:disable= is not possible. I'll try to fix it at some point probably. Anyway, it's still a low prio as for fairly small lambdas it can be worked around by turning multiline lambda into single-line one: func(key):prints("key:", key);return true

@lethefrost
Copy link

So what is the current way to work around this? As this seems to be a lexer error, we can't really gdlint: disable, I assume. Is there a way to exempt blocks of code from the whole linting process?

You can workaround it by having lambda returning one bing expression, or you can use ; to separate statements. See: https://github.com/Scony/godot-gdscript-toolkit/blob/master/tests/formatter/input-output-pairs/long-inline-lambdas.out.gd

As for exempting blocks of code from the whole linting process - it's not possible atm.

Sorry, this page is gone now... Do you mind explaining a little more about how to workaround?

Also, I am wondering if it would be able to just add some try/catch block to this error, so that at least the formatter can continue and finish formatting the file regardless of the error, instead of crashing at the point it encounters a lambda function and doesn't format at all.

@Scony
Copy link
Owner

Scony commented Apr 18, 2024

So what is the current way to work around this? As this seems to be a lexer error, we can't really gdlint: disable, I assume. Is there a way to exempt blocks of code from the whole linting process?

You can workaround it by having lambda returning one bing expression, or you can use ; to separate statements. See: https://github.com/Scony/godot-gdscript-toolkit/blob/master/tests/formatter/input-output-pairs/long-inline-lambdas.out.gd
As for exempting blocks of code from the whole linting process - it's not possible atm.

Sorry, this page is gone now... Do you mind explaining a little more about how to workaround?

Also, I am wondering if it would be able to just add some try/catch block to this error, so that at least the formatter can continue and finish formatting the file regardless of the error, instead of crashing at the point it encounters a lambda function and doesn't format at all.

As for the workaround, lambda may be written in one line where statements are separated by a semicolon like this:

var f = func(key):print("key:", key);return true

also, if the lambda has some expression inside, it can be extended to multiple lines like:

var f = func(x): return (
    x is int
    and x > 0
    and x < 10
)

As for the try-catch - it's not possible without multiline lambdas support.

@lethefrost
Copy link

So what is the current way to work around this? As this seems to be a lexer error, we can't really gdlint: disable, I assume. Is there a way to exempt blocks of code from the whole linting process?

You can workaround it by having lambda returning one bing expression, or you can use ; to separate statements. See: https://github.com/Scony/godot-gdscript-toolkit/blob/master/tests/formatter/input-output-pairs/long-inline-lambdas.out.gd
As for exempting blocks of code from the whole linting process - it's not possible atm.

Sorry, this page is gone now... Do you mind explaining a little more about how to workaround?

Also, I am wondering if it would be able to just add some try/catch block to this error, so that at least the formatter can continue and finish formatting the file regardless of the error, instead of crashing at the point it encounters a lambda function and doesn't format at all.

As for the workaround, lambda may be written in one line where statements are separated by a semicolon like this:

var f = func(key):print("key:", key);return true

also, if the lambda has some expression inside, it can be extended to multiple lines like:

var f = func(x): return (
    x is int
    and x > 0
    and x < 10
)

As for the try-catch - it's not possible without multiline lambdas support.

Thanks for the clarification, and looking forward to a authentic solution. Just by the way thank you for all the efforts bringing this amazing toolkit to us.

@Bloodyaugust
Copy link

@Scony I wholly understand that this issue is very tricky to fix, but a callout in the main documentation that multiline lambdas are currently causing gdformat to error out would be much appreciated. Finding this issue was a little tricky, and for a while I was tearing my hair out.

@Scony
Copy link
Owner

Scony commented Apr 29, 2024

@Scony I wholly understand that this issue is very tricky to fix, but a callout in the main documentation that multiline lambdas are currently causing gdformat to error out would be much appreciated. Finding this issue was a little tricky, and for a while I was tearing my hair out.

That's a very good point - I'll address that, thanks.

@oxeron
Copy link

oxeron commented Jun 5, 2024

Any news on this subject ?

@Scony
Copy link
Owner

Scony commented Jun 5, 2024

Any news on this subject ?

It's WIP, you can track progress here: https://github.com/Scony/godot-gdscript-toolkit/commits/multiline-lambdas/

TL;DR the lexer and parser support is done, the formatter support is like 5% done.

@JohnDevlopment
Copy link

JohnDevlopment commented Aug 20, 2024

Hi, I wrote var cb := func(button: Button): button.release_focus() but still got an error. I'm using the latest version. This code is all on one line.

EDIT: Nevermind, there was a multiline lambda later on in my code.

@Jack-023
Copy link

Jack-023 commented Sep 2, 2024

TL;DR the lexer and parser support is done, the formatter support is like 5% done.

Does this mean that the linter works? Would it be possible to update gdlint to support multi-line lambdas without gdformat? I have some pretty long lambdas so it is difficult to work with them if they are in a single line but I like having a precommit hook run gdlint. I am working around it at the moment but by keeping the non-minified version in a comment but it is pretty hacky.

@Scony
Copy link
Owner

Scony commented Sep 2, 2024

TL;DR the lexer and parser support is done, the formatter support is like 5% done.

Does this mean that the linter works? Would it be possible to update gdlint to support multi-line lambdas without gdformat? I have some pretty long lambdas so it is difficult to work with them if they are in a single line but I like having a precommit hook run gdlint. I am working around it at the moment but by keeping the non-minified version in a comment but it is pretty hacky.

Yes, linter should work - but making gdformat (even broken) aligned to that requires effort, therefore I can only suggest you to try https://github.com/Scony/godot-gdscript-toolkit/tree/multiline-lambdas branch yourself.

P.S. I'm not sure but there may be some regression in the parser so if you get parser errors then it won't work for you anyway.

@Jack-023
Copy link

Jack-023 commented Sep 3, 2024

That makes sense, thanks!

@Scony
Copy link
Owner

Scony commented Sep 15, 2024

The support for multiline (multi-statement) lambdas has been merged into master now.

There are few not implemented problems, but IMO 99.9% of users should not notice them, so I decided to proceed.

Now, I'd like the change to sink in for few days/weeks before I release new gdtoolkit version.

If someone here would like to help, I'd appreciate it if someone could install gdtoolkit from master using either:

pip3 install git+https://github.com/Scony/godot-gdscript-toolkit.git
# or
pipx install git+https://github.com/Scony/godot-gdscript-toolkit.git

and write a comment here stating that "in my project it works!"

@tavurth
Copy link

tavurth commented Sep 15, 2024

Screenshot 2024-09-15 at 20 20 26

"In my project it works!"

@lethefrost
Copy link

The support for multiline (multi-statement) lambdas has been merged into master now.

There are few not implemented problems, but IMO 99.9% of users should not notice them, so I decided to proceed.

Now, I'd like the change to sink in for few days/weeks before I release new gdtoolkit version.

If someone here would like to help, I'd appreciate it if someone could install gdtoolkit from master using either:

pip3 install git+https://github.com/Scony/godot-gdscript-toolkit.git
# or
pipx install git+https://github.com/Scony/godot-gdscript-toolkit.git

and write a comment here stating that "in my project it works!"

Finally!! Thank you so much. It's my hugest problem with this formatter. ❤️

Would you mind elaborating what the few remaining issues are? So that people might help on it.

@plink-plonk-will
Copy link
Contributor

In my project it works! This is fantastic - thank you!

@Scony
Copy link
Owner

Scony commented Sep 16, 2024

The support for multiline (multi-statement) lambdas has been merged into master now.
There are few not implemented problems, but IMO 99.9% of users should not notice them, so I decided to proceed.
Now, I'd like the change to sink in for few days/weeks before I release new gdtoolkit version.
If someone here would like to help, I'd appreciate it if someone could install gdtoolkit from master using either:

pip3 install git+https://github.com/Scony/godot-gdscript-toolkit.git
# or
pipx install git+https://github.com/Scony/godot-gdscript-toolkit.git

and write a comment here stating that "in my project it works!"

Finally!! Thank you so much. It's my hugest problem with this formatter. ❤️

Would you mind elaborating what the few remaining issues are? So that people might help on it.

Problem one:
var foo = func(): if true: return 1 won't parse

Problem two:
89b8200#diff-deb96115f719c01f21e10dcc674db18d6be7d759a1a3badaf506c9c9f6625a0fR216-R221

Both are most likely a bug in the lexer, so hopefully, I can address those fairly easily.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working formatter parser
Projects
Status: Done
Development

No branches or pull requests