Fix parsing of python f-strings

[olejorgenb]

Double-quoted f-strings mostly worked, but single-quoted did not:

a = f'what'

Reported that f'what was mispelled.

I think this is due to:

1. the (strings) @string query give back in the whole node string node - including f'
2. the word-extraction in parser.rs (processor.extract_words()) does not consider ' a word separator (rightly so)

(2) explains why double-quote f-strings works - the word splitting split on ", resulting in f and what, and I assume single letter words is not checked.

(1) can be confirmed in the tree-sitter playground [1], where we also see that there is an string_content node type more suitable.

Screenshot from the playground

a = f'hello'
a = f"hello {foo} world"

Using the string_content node means that code inside interpolation units wont be spell-checked, but this aligns with the current behavior that only definitions, comments, and strings are checked.

[1] https://tree-sitter.github.io/tree-sitter/7-playground.html

---

Ref: #136 (comment)

[olejorgenb]

From here

https://github.com/tree-sitter-grammars/tree-sitter-query/blob/60e253d3c9d6b1131a0f75c85e4bdcc9a48d5b42/src/node-types.json#L398-L412

it seems like string_content perhaps is a general node type which exist in all grammars? (Though I don't find any mention of it in the docs)

I see now that the same node-type is used in rust.scm and many of the other languages (but not all).

[blopker]

Hey! Thanks for looking into this. I thought this was an issue, but was unable to reproduce when I looked into it. Oddly, this issue isn't triggered when the f-string is in a function call, like print().

If you have time, I'd love to see a test for this though. Something in test_python.rs would be good enough.

[blopker]

it seems like string_content perhaps is a general node type which exist in all grammars? (Though I don't find any mention of it in the docs)

Sadly, this isn't always the case, and even if a grammar has it, it can mean different things depending on the language. All we can do is test it per application. 😅

[olejorgenb]

I'll see if I can add a test later 👍

Oddly, this issue isn't triggered when the f-string is in a function call, like print().

It does here?

[blopker]

It does here?

Wow, yeah, my experiment was bad! I tried print(f'Wht'), but that works because of the capital W. Trying to do too many things at once...

[olejorgenb]

Ah, right - because capital letters splits words.

I pushed a test quikcly, but I admit that I let Claude write it. It's perhaps unnecessary long, but otherwise seems fine as a basic test.

Thanks for creating this language server btw. It works very well and it's nice that only definitions are checked ❤️

[blopker]

Nice, thanks! Claude didn't misspell any of the f-string variables, but I can fix that up :)

Glad you've enjoyed using Codebook!

[blopker]

v0.3.18 is out with the fixes!

[olejorgenb]

Nice, thanks! Claude didn't misspell any of the f-string variables, but I can fix that up :)

Ah, that was sloppy of me. Seems it just repeated the text I gave it.