Add README

[look]

rendered

[computermode]

Wow this is awesome, thank you Luke for taking the time to make these thoughtful additions to our docs 🙇🏻 I have a question and a suggestion but they're both minor. Thank you!

[computermode]

Question: Is an entity in this context the same as a symbol? We refer to classes/methods as symbols in our other public documentation such as Navigating code on GitHub and I'm wondering if we'd want to keep that consistent here or if there's a specific reason to call it an entity instead.

[look]

I copied this language from the first paragraph of Navigating code on GitHub, but I like "symbol" as a word to use here, it makes more sense to me than entity or "programming language construct" as I used below.

[computermode]

Oh that's funny that the same document has both terms used seemingly interchangeably 😆 I also prefer symbol over entity

[hendrikvanantwerpen]

I like the idea of being more explicit about this process and the write up as well! I left a few questions and suggestions on the text.

One high-level question I have is whether you envision this repo just as a means to document the process and interact with the community, or whether we also put artifacts here? More conretely, we could move tree-sitter-parsers here, plus custom query files if we accept those.

[hendrikvanantwerpen]

Question I wonder if it makes sense to include an example here already. Since we're also describing the FQN queries in detail below, adding an example here would make this more self-contained.

[look]

Good suggestion. Let me consider possible examples.

[look]

I couldn't come up with a good way to add an example that didn't seem extremely long. It seems like we'd want to explain the tagged symbols extracted for a few different types of programming languages. Let me know if you have suggestions.

[hendrikvanantwerpen]

Yeah, I can see that. The fqn section is already quite detailed for being in the top-level README. Perhaps keep it like this, and if we start getting questions because people don't figure out what to do, we can revisit.

[hendrikvanantwerpen]

Question Should we be explicit about our expectation of where these queries (and the FQN queries below) live? Do we require them to be part of the grammar repo, or do we accept custom queries?

[look]

As far as I know, we've always adapted tags queries from the grammar repositories. We've also written our own tags queries, but that is limited because we aren't experts on most languages.

I suppose we could accept a third-party query, but I think it will be much easier to manage if the source of the tags is the same as the parser. Lacking a maintained tags query is a sign that the parser isn't mature enough for us IMHO.

[hendrikvanantwerpen]

I agree that having the queries in the grammar repo is definitely the ideal state. But I thought we currently have our own query files for some languages in symbol-extraction, mostly because the fqn stuff was internal to us. But perhaps we upstreamed those at some point?

[look]

Yes, we have to modify the queries which makes managing them more difficult. I'm saying we should only accept upstream tags queries from the parser repository to avoid making managing tags queries even harder. 😄

[hendrikvanantwerpen]

Upstreaming the FQN logic to tree-sitter would be a nice way of making those things more relevant for the community, increasing the willingness to accept such queries upstream. The idea was floated a couple of times in the past, but nobody got around to it.

[look]

Thanks for the reviews!

[look]

I copied this language from the first paragraph of Navigating code on GitHub, but I like "symbol" as a word to use here, it makes more sense to me than entity or "programming language construct" as I used below.

[look]

As far as I know, we've always adapted tags queries from the grammar repositories. We've also written our own tags queries, but that is limited because we aren't experts on most languages.

I suppose we could accept a third-party query, but I think it will be much easier to manage if the source of the tags is the same as the parser. Lacking a maintained tags query is a sign that the parser isn't mature enough for us IMHO.

[look]

Good suggestion. Let me consider possible examples.

[look]

One high-level question I have is whether you envision this repo just as a means to document the process and interact with the community, or whether we also put artifacts here? More conretely, we could move tree-sitter-parsers here, plus custom query files if we accept those.

I discussed with @dcreager how this repo could evolve. We decided initially that it should be just this documentation, but perhaps in the future it could include the tagging queries or the canonical parser versions we use.