Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Elm: Add support for Elm-Lang via optlib #1260
Thank you for making this pull request. This is the first pull request about optlib made by other than I!
As appveyor ci tells us, building and testing on win32 are done well.
Just a question. You made the optlib "public domain". Is this o.k. As I wrote in bitterjug/vim-tagbar-ctags-elm#1, I can add
Next one is important.
First of all could you read http://docs.ctags.io/en/latest/news.html#reference-tags ?
Elm parser captures
Elm parser captures
I don't know
added a commit
this pull request
Jan 3, 2017
I see. About ctags main/ and parsers/ I would like to keep GPL. However, the other area, especially about optlib, the other license is o.k. Anyway I will add
I see. Thank you for understanding.
Your last point about capturing b in import a as b makes sense only if reference tags are available. So I propose to remove the import matching options (and, maybe, add them back in in as optlib in vim-tagbar-ctags-elm).
Tagging Y in
@bitterjug, thank you for merging and pushing the changes.
I'm not sure following my comment is important. People may think too detailed thing.
I will write about the kind name of the last commit. My choice was
To explain the idea behind
In definition tag, ctags captures a newly introduced name specifying a language object.
Here "main" is a newly introduced name and a function(specified with name "main") is the language object. Of course, the language object is a function.
Let's think about kind; what is "main"? Based on the fundamental rule, the main has "function" kind.
I think in ctags we should say "main" is a function. All things ctags deals with are names. So
"c" is a newly introduced name. "a" may defined somewhere. Just referring here. "b" is defined in "a". Just referring here, too. "b" may specify a language object. "c" specifies a language object specified by "b". However, the detail of language object is unknown from ctags because it is not defined in the input file. Therefore my choice was "unknown".
For the reason I introduced "unknown" kind in the python like:
I'm bad in English. So I wonder "unknown" is good name or not. However, the idea is taken from
Your choice is "rename". "c" is a name renamed from a.b. The naming is not about language object specified with "c". If elm is rather macro language like Cpreprocessor or m4, it is meaningful.
For above consideration. I think unknown is a better name.
However, in following two criteria, "rename" is still acceptable.
If it is really good for vim user. Many editors can read tags file. However, I guess vim is the primary client tool of ctags. I'm monitoring who puts "star" on ctags. It seems that most of them are vim users.
If you, a parser maintainer, are happy with the kind name. ctags was mostly dying project. (https://sourceforge.net/p/ctags/mailman/ctags-devel/?viewmonth=201310&viewday=15)
I see where you're coming from. And I appreciate your appetite for careful design!
I will try and rephrase your point to check I understand properly: you chose the name "Unknown" because it is not possible to know what type of language object has been imported and renamed.
When I saw this, I thought you meant that you didn't know what it was. But you're saying that Ctags doesn't know what it is. However, I still feel uncomfortable with "Unknown", it puts the emphasis on the wrong thing. I've had a look at the Elm parser and read some docs and notes on imports. The other word I have seen used to describe the name that follows
But it turns out that in Elm the thing that comes before
import Module.Submodule exposing (foo, bar, Bas)
You can also say:
import Module as Alias exposing (foo)
Which means both
So, we do know something about the language object being referenced by a local name: it is a module. What kind of tags should this create? Something like:
And how close to this can we get with a regexp parser?
Yes, you understand my intent properly!
As you may know foo, bar and Bas cannot be captured well with regex parser.
As far as hearing about elm from you, kind for
I designed roles and kinds for
In Elm, X of
Updated kind/role design is:
Let's think more for future.
This is similar to "from A import B" in python. With the statement, can you use X?
A bit complicated even for me:-P. However, if we carefully design the kinds/roles, someone can write a very interesting and powerful client tool for code navigation. Helping the one to write such tool is my dream:-) The bad dream is that the github account of the one is @masatake:-(
So, are you saying that the kind for
import X as Y
which introduces a module definition at the top of a file (each file defines a module in Elm)? Perhaps the only difference between them would be that ideally
Returning to the present. What shall I do with the Elm optlib regex for
Thank you for explanation. I understand Z and Y are very different in the aspect.
Your naming, "Renamed imported module" tells the intent well. However, it is a bit long.
(In this case namespace is not a role. It is kind.)
As far as I can remember ctags rejects following patter definitions:
The descriptions for m are different in the two patterns. u-ctags don't allow it. You may want to define sub-kind alike things by giving different descriptions. In u-ctags, there is no such concept. In such case, you should define a new kind. e-ctags allows this technique. And this technique looks popular. I found this technique was used in found But I think it is bad idea. See #1228 .
For long time I have wondered why we have to write the same
In planed `--regex2-=' I would like to improve this area.
Thank you for updating.
As I wrote before, I'm looking for people who wants to improve the optlib and regex facility.
If you have interest, I will make an invitation. Cold you receive it and merge this pull request by yourself?
If not, I will merge this pull request. If you improve elm parser, make a pull request in the future. I or someone in this organization may merge it.
Yes please. I'd be delighted to maintain the elm optlib.
referenced this pull request
Oct 12, 2017
I am testing my prototype for the feature recoding reference tag from a parser defined with
What I did is adding new option
With this new feature puppetParser can capture included manifest as a reference tag.
referenced this pull request
Mar 16, 2018
Can you explain how ctags identifies the file that a language feature is defined in? If there are more than one thing with the same name in different files, does it somehow automatically scope it to the modulie file it was defined in, or would we have to use scoping rules on the module definition?
Yes. I think calling it will be "imported".
I'm not sure I understand what you wrote correctly. Could you fix my example or ahow an alternative example if I misunderstand?
The answer may be no. Let's think an imaginary language X that uses .x as a file extension.
A is defined in both a0.x and a1.x.
My understanding of your questions about ctags parsing b.x is "can ctags know in which
ctags cannot do it.
Even if we take either approach, ctags cannot make perfect results.
Apologies; I was not clear enough. Your example deals with the situation where >1 file define modules with the same name. That won't actually happen in a well set up elm project. I wanted to ask a slightly different question that arises from me starting to think more about ctags working on a whole project not just on a single file.
In the context of a project I should be able to navigate to the place where a language element is defined, no matter which file it was in. So I'm thinking about the situation where file
module One exposing (..) foo : Int foo = 1
Now I import
module Two exposing (..) import One bar : Int bar = One.foo foo : Int foo = 2
Using my current implementation, ctags captures:
Which is to say it does not distinguish
Should the module regexps have
We do have
But it gets cleared on import, etc.
Wow! working with tagbar gave me completely the wrong mindset. Okay, this really needs rethinking.
Mind you, one day I expect this stuff will be done by a custom parser based on the language grammar, like happens for Haskell.
From the view of ctags ideal output may be:
One.foo, One.bar, and Two.bar can be derived from the above output.
foo's scope is broken. bar must be poped at the end of bar's definition. Howerver, no way to detect it in line oriented parser. See #1577.
Anyway, now a role like "imported" can be defined in an optlib parser. I will push updated version of optlib2c translator in soon.
If we can assume function definitions are never nested, an imaginary regex "popIfTopIs,function" may allow to us what you want.
Think following input:
With following imaginary option,
ctags may work as following:
So I'm not sure
I merged following change unintentionally without your ack.
The change for elm.ctags in the later one should go to the former one. I took a mistake.
If elm parser in the latest HEAD doesn't work as you expected, please, let me know. (Please, one a new issue.)
I'm just catching up on this thread.
Here's a pathological example:
foo = (\ bar bas -> bar + bas) (let n1 = 1 in n1) -- bar (let n1 = 1 are not named n2 = 2 n3 = ( let n2 = 6 -- shadows n2 in n1 + n2 -- 1 + 6 in n2 + n3 -- 2 + 7 ) -- bas
Does it even make sense to try and tag the two
I read your comment. Temporary I would like to ignore nested functions and anonymous functions.
I would like to focus on the following input you showed:
When thinking about scope field, what we want to get for the input is (with --fields=+K option):
Am I correct?
Currently what we can get is
No scope information is attached to "bar" and "foo'.
Hi! Thanks for this.
In the case of Elm functions that means only the ones defined at the top level of the module (no indentation). And we could get the correct scope for those using
I missed a point! You're saying the only scope we need to bother with is module scope. In which case, yes you're right. I'll try and find time to give it a go this weekend.…
On 31 March 2018 at 16:47, Masatake YAMATO ***@***.***> wrote: If we have to think about a module and functions under the module only. I think popIfTopIs is not needed. (popIfTopIs is not implemented yet.) What we need is only scope=set (when finding a module) and scope=ref (when finding a function). Am I wrong? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1260 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAeQZp49sncbU0pkqnOvkm6oEqmIIq8mks5tj6UCgaJpZM4LZKrV> .
I have one more question.
Do you want full querified tag output like:
I have code for generating automatic full querified tag. I wonder I should add the code the repository or not.
Now that ctags supports elm, I get the following errors when trying to use my development version of elm.ctags:
Is there a way to override built in language settings or should I be building my changes with make each time?
I'm sorry @masatake, I don't understand your question. I guess you mean "fully qualified" output. And from the example it looks like we're getting two matches for each function, one with just the function name and the other with
I guess the question is which do I need in order to be able to navigate from a tag use to its definition if multiple modules are allowed to define top level functions with the same name?
using the source from above:
module Two exposing (..) import One bar : Int bar = One.foo foo : Int foo = 2
Produces the following:
Trying to navigate with that I see that it doesn't distinguish One.foo from Two.foo. So maybe I do want fully qualified tags. What do other languages do?
If you can rebuild a ctags binary instantly, the cycle "edit optlib/elm.ctags and run make" is the best.
If you don't want to do so in some reasons, disable the built-in elm parser, and use something temporary name for your new code. Let me show an example.
Here we will use elmX as the temporary name.
You understand what I would like to say well. Thank you.
Some of the crafted parsers, written manually in C, emits fully qualified tags by itself.
Some of the crafted parsers use AutomaticFQTag feature I developed to emits FQ output.
No optlib parser emits FQ tags because there is no way to do so.
However, the feature has a weakness; you cannot tune its behavior.
Another example is about kinds of language objects.
This pattern causes that the AutomaticFQTag feature emits
If you can accept or you don't care the weakness, I will work on
Thanks again @masatake. I think I understand AutoFQTag now.
Before I answer this I want to sketch out some context. When I first wrote ctags options for Elm I wanted to use tagbar which shows tags in the current file, ordered by scope. I was used to using it with Python to get an overview of a file's contents. And Elm files are sometimes large. These days I use Elm at work and, increasingly, we're splitting our work into smaller files with fewer types and functions in them. As a result, I'm becoming more interested in being able to navigate to the file where a thing is defined. Tagbar gives an intra-file. But tags also offer useful inter-file navigation.
Inter-file tags in Elm
In Elm you can import a module (and optionally alias it) and then refer to
It is common for Elm modules to define things with the same name. Many modules will contain a type called
Intra-file tags in Elm
Each file contains exactly one module; and that may contain ports, types and functions. Function definitions can contain other function definitions inside anonymous
There is another kind of language artefact that I sometimes thing it would be nice to see in Tagbar. Frequently functions contain large case statements that branch of the different cases of a (tagged) union type. I experimented with capturing those as tags but the result was never satisfactory. Although it is common to have one such case statement in a function, sometimes there may be many and even nested ones. And since the case statements themselves do not have names they do not really define useful scopes.
So perhaps it's not really very useful to try and capture these with tags. I took a quick look at what happens in Haskell which has a very similar language structure. There are several solutions including one built in to GHC itself. Lushtags targets Tagbar specifically. According to its tagbar config it does not use functions as scopes. I can't tell without installing it if nested function definitions get tagged at all.
I think Automatic FQ tagging would be a useful addition to an Elm optlib. (And one day Elm might gain native tagging similar to those available for Haskell)
Thank you. Impressive. Categorizing Inter-file tags and intra-file tags are vital in designing a parser I have recognized them implicitly, but I should recognize them and write them to ctags-optlib.7.rst.in. I will borrow your sentences in the comment when updating ctags-optlib.7.rst.in. I would like you to review the man page when I updated.
Capturing Inter-fie tags has higher priority. While reading your comment, I found
I would like to see an example of "case statements".
I wodner what kind of tags people want.
I see. I will add autoFQtag flag to
Oh, sorry we should focus on Inter-file tags.