Syntax highlighter is incomplete and needs clearer error messages #6
Comments
|
Thank you for your bug report, but the plugin is pre-1.0.0. (I just released https://github.com/KronicDeth/intellij-elixir/releases/tag/v0.1.0) The supported features of Elixir is very limited as you can see from the CHANGELOG.md. In your specific example, the plugin is working as the comments are being highlighted as comments, which is why they are grey and not marked as bad characters (red): The general order of feature support follows order in Elixir's own lexer, which you can find here. So far, I had implemented support for:
If you'd like to see all the supported syntax in the Intellij itself, go to Preferences > Editor > Colors & Fonts > Elixir, where you can select the highlighting colors for each type of token that intellij-elixir currently supports. The example text in the Color Settings panel shows everything the plugin can parse in little snippets. The next feature I'm working on is atom support because @h4cc requested it and he's been very helpful in testing. Following the order of elixir_tokenizer.erl, the next feature would have been Char Tokens, but I'll do that once atom support is done now instead. The bulk of your file would be covered by Identifiers, which is actually the last thing that's really processed by elixir_tokenizer.erl because a valid identifier by definition is anything that doesn't have special handling otherwise. This is only a weekend project for me, but if you'd like to help we could maybe get through all the other token types faster, otherwise I'd advise waiting a few months for me to get to identifier parsing before you should except your example file to parse all the way. I'll leave this bug open to use as an example to keep testing against unless you want to close it. |
|
As for the error message "ElixirTokenType... or ...[..] expected, but got 'd' ", I won't get to cleaning those up until I implement the Annotator I believe and all this porting the elixir_tokenizer is at the level of the lexer and parser. The annotator is built on top of parser as you can see from the custom language plugin tutorial that JetBrains published. |
|
Thanks for the very elaborate response! And thanks for starting this project! After your explanation my main point of critique would be, that the plugin is already in the IntelliJ repository. I think everyone who finds a language plugin there will expect AT LEAST full Syntax Support. So I guess quite some People might start bothering you now :) Maybe you could put some kind of disclaimer and your roadmap at the start of the readme, that would definitely help People understand what's going on here. I'd definitely like to help, but I hardly find time to play with Elixir itself, so I fear I won't find time for this project, too. My personal Workaround BTW is using the IntelliJ TextMate bundle Support plugin and using https://github.com/elixir-lang/elixir-tmbundle for Syntax highlighting. It works very well, only "go to symbol" etc is not working. Maybe you also want to have a look at |
|
It's not a matter of seeing the other IDEs syntax highlighters: each IDE uses a different language for specifying it's syntax highlighting. Emacs uses elisp, vim uses it's own syntax highlighting language, etc. Looking at the other implementations isn't that much more useful than just looking at the elixir_tokenizer source in elixir itself. The hard part is converting it to the JFlex format that IntelliJ's lexer uses since JFlex's support for extended regular expressions is limited. JFlex by itself doesn't have any support for the nested lexing that is required to handle interpolation, so I had to add a stack myself to convert JFlex's output lexer to a https://en.wikipedia.org/wiki/Pushdown_automaton. I've actually talked to the maintainer of elixir-mode.el in person and he said he didn't implement recursive interpolation support, so there are holes in elixir-mode too. On top of JFlex, there is the Parsing Expression Grammar (PEG) used by Grammar Kit, which is sorta like a BNF used by the other IDEs, but not exactly. I can't use a direct translation of the elixir.yecc file because it's BNF would produce an inefficient PEG tree. (I tried doing that and it hung IntelliJ) Additionally, adapting to the Grammar Kit format allows for better error recovery and for me to match invalid elixir syntax that is automatically correctable. So, I'm looking for contributors with experience with JFlex or any of the lex or flex implementations or anyone that has done a lot of work with PEG. I have used PEGs before with the treetop gem in Ruby, but that's it. Probably the most useful source to look at would be the lexer and parser and Rubymine or the Ruby plugin for IntelliJ since those also have to implement recursive interpolation, but those are both closed source and not in the community edition sources. So, as far as I've been able to tell, syntax highlighter for each IDE is a completely manual process unless all us developers can get together to determine a superset of highlighting and invalid code parsing that could be implemented in elixir itself and then we just parse that output into our own IDEs formats as suggested by the elixir-mode maintainer when we met in person. |
|
@cnsgcu The "project" action isn't setup yet. Right now, there's just a file association. If you make a .ex or .exs file, or make a file and tell the IDE that it should be treated as Elixir, then the highlighter will function. You'll know the association is working if the file icon is the elixir "drop" icon: |
|
Thanks @KronicDeth. The IDE does not recognize .ex nor .exs file. I could not find Elixir option under Preferences > Editor > Colors & Fonts either. My IDE's version is 13.1.4 |
|
Did you restart the IDE after installing the plugin @cnsgcu ? |
|
Yes I did. Just try to uninstall and reinstall the plugin and it's still not working. |
|
Are you installing using the IDE's plugin browser, using the zip here on github or downloading the zip from plugins.jetbrainscom? |
|
I used the IDE's plugin browser. |
|
I just downloaded Intellij 13.1 Community ( for Windows) from JetBrain's site, installed it, then installed the Elixir plugin and the Editor settings showed up. I do the development on OSX, so I know it works there too. Are you on Linux? Is there anything in the IDE's Event log? Can you post a screenshot showing the plugins screen like this I'm at a loss for how to debug this beyond looking at the Plugins screen or checking the Event log for errors. The JetBrain's tutorial doesn't really cover installation issues and I don't even control the format of the plugin zip: it's all setup by the Plugin Project template. |
|
Here is my screen shot. I am on Mac OS X Mavericks. Error Log: 2014-09-19 21:39:03,550 [ 28904] ERROR - nsions.impl.ExtensionPointImpl - org/elixir_lang/ElixirFileTypeFactory : Unsupported major.minor version 51.0 [Plugin: org.elixir_lang] com.intellij.diagnostic.PluginException: org/elixir_lang/ElixirFileTypeFactory : Unsupported major.minor version 51.0 [Plugin: org.elixir_lang] |
|
I think the error is due to me building with JDK 7 and you running with JDK 6, based on this http://stackoverflow.com/a/11432195/470451. I'll look into how to get JDK 7 to produce JDK 6 compatible .class files OR how to downgrade to JDK 6 for the builds. |
|
Oracle's current documentation says that JDK 6 doesn't get security patches anymore. You may want to upgrade your jdk to be safe unless you're specifically targeting older JDKs, but I'll still try to fix the build for JDK 6 since it's the default on Mavericks. |
|
Awesome, just changed the Info.plist to use JDK 7 and it's working now. Thank you for your help. |
|
@cnsgcu, I've rebuilt with JDK 6 (and confirmed it works when using the default Mavericks JDK 6 instead of JDK 7). You can get the new build now from github or wait for Jetbrains to approve it in a few days and get it from their site or directly in the app. |
|
I think I'm affected too: |
|
Same error for me too, with IntelliJ 13.1.4 using default Apple JDK 1.6, mac OX 10.9.5, elixir plugin v 0.1.3. I do have Oracle JDK 1.7 installed but use it for other stuff, not starting IntelliJ-- that is haven't changed it per recommendation at https://intellij-support.jetbrains.com/entries/23455956-Selecting-the-JDK-version-the-IDE-will-run-under |
|
@develop7 if you mean you're affected by @Hades32's original bug of I'm not done with the syntax highlighter yet, then yes, I agree. You see that because identifiers aren't implemented in the highlighter yet and I haven't prettified the errors yet, so you're seeing all the token types that the lexer understands at the beginning of the a file. I'd say you're not seeing a bug, just missing features. Open other issues for the things you'd like to syntax highlighting for first and I'll prioritize them. The types of tokens is based on https://github.com/elixir-lang/elixir/blob/4b505286ec61b32cb791b79268ae6c58340c71f0/lib/elixir/src/elixir_tokenizer.erl and I'm currently working on features #15 and #5. |
|
@rsachdeva you need to be clearer about which error you're getting. @Hades32 and @cnsgcu reported different errors on this same bug. |
|
FYI: While we're waiting for @limhoff-r7 / @KronicDeth to complete his great work I started a simple sytax highlighting file for Notepad++: |
|
@limhoff-r7 I was reporting the same error as @develop7 reported. Seems missing features from your reply :) That means if I clone an elixir project and open in IntelliJ I don't get minimal syntax highlighting yet. |
|
@Hades32 is right, writing syntax highlighters for other IDEs is much simpler than in JetBrains as most of them let you use full regular expressions and I also only work on this a few hours a week, so use his Notepad++ or the TextMate-bundle plugin he mentioned before to use TextMate's highlighter in Jetbrains IDEs. If you don't want to have to keep checking each release, I plan to go to v1.0.0 once the syntax highlighter is complete, so you can just wait until then to try the plugin again, but I do appreciate people using the plugin in its current pre-release state as they may catch crashes I don't find myself. Right now, you shouldn't get plugin crashes; just bad characters (with the red squiggle underline). If you get crashes, do open a new bug with the crash/event log. |
|
@KronicDeth Appreciate the work you are doing on this as a side project. |
|
So um @KronicDeth & all: what IDEA version should I use to see syntax highlighting with this plugin? |
|
I'm developing on 13.1.5 right now and I was using 13.1.4 before the update. I've also used the plugin in Rubymine 6 and the 7 EAP. If you haven't read this whole thread: the syntax highlighting is still incomplete. There is only highlighting for what is in the Color Settings panel:
The most obvious thing that users have brought up is that identifiers, which covers function/macro names and variables names aren't highlighted, so |
|
The plugin.xml that the Plugin DevKit plugin generated when I started the project, set the |
|
@KronicDeth I've seen remark about incomplete syntax, I just didn't believe my eyes and thought you meant something else by it :) Anyway, thank you for hard work implementing this in Java |
|
Tengo instalado Intellij 13.1.5 y el Plugin de Elixir 0.1.3 ; les comento que cuando quiero crear un nuevo Proyecto, el editor no me da la Opción , de crear un pro. de Elixir. No aparece por ningún lado... lo he instalado y desinstalado, he puesto distintas versiones, y nada.. Solo aparece para configurarle los colores en el editor. |
|
@Claudiojlf , disculpen mi español , estoy usando Google Translate tanto leer su comentario original y componer esta respuesta . Actualmente no existe una función para crear proyectos de Elixir en el menú Nuevo proyecto . He abierto una nueva emisión de estas solicitudes , #22 , para cubrir esto. Por favor, suscribirse a ese tema si quieres actualizaciones sobre cuando esta característica está disponible . En el futuro , por favor, abra nuevos errores y no agregar a esta cuestión. Es más fácil para mí para fusionar errores duplicados de lo que es desmenuzar el tema en otros temas . Cuando se crea un problema usted será automáticamente suscrito a las actualizaciones en lugar de tener que ir manualmente suscribirse a un tema , como #22 , cuando yo creo. |
Try to prevent duplicates of Issue #6 by putting explanation that parser is incomplete at top of README and change notes in plugin.xml.
|
@KronicDeth are you going to release this plugin? %) |
|
@Elfet some version of the plugin has been released in the JetBrains plugin repository since 2014-04-08. You can install it using the instructions in the README. This issue is still open because the parser won't be complete until v1.0.0. |
|
off topic: do you guys accept donations for the project? it would be nice if we are able to accelerate this work. |
|
+1 on funding, really would like to have first class support in ruby mine(or derivative) just like alchemist dies for emacs |
|
+1 for RubyMine! I use rubymine for both ruby and golang |
|
@smeevil, @nakhli the plugin does work in Rubymine. You can make The only feature that doesn't is "File > New > Project From Existing Source..." because that uses an Extension Point (projectStructureDetector) that JetBrains only supplies for IntelliJ and not for the language-specific IDEs like Rubymine. Extension Points that only work in IntelliJ are in https://github.com/JetBrains/intellij-community/blob/master/resources/src/idea/RichPlatformPlugin.xml. There is probably a similar extension point in non-Intellij IDEs, since Rubymine does support projects, but it's not like IntelliJ projects where you can select different langauges is my understanding since the Rubymine projects involve selecting bootstrap or a different JS framework to use in a Ruby project like Angular or Ember. @nakhli are you saying that golang plugin allows you to create Go-only projects? If it's open-source I can look at what it's doing. |
|
yeah , I am using it in ruby mine :) I was referring to the support and development of the plugin in general. So if you like feel free to put a donation button somewhere ! |
|
@KronicDeth I am using this go plugin: https://github.com/go-lang-plugin-org It allows creating go-only projects. and it works seamlessly on rubymine. |
|
Just threw up a $100 donation to you, good work so far 👍 I was wondering if you have some sort of timeline for reaching 1.0? |
|
Added an other $100 on top of that as well, keep it up :D
|
|
@olivermt 1.0 is a milestone, so you can just monitor that to see what's blocking. Right now, the only thing I know is the bug in Kernel.ex. I'll also probably make a feature to ensure all the tokens and rules have human-readable names for clearer error messages since that's part of this issue in @develop7's screenshot |
|
#172 was fixed by #173. Better human-readable error messages by way of token names landed in #178. Expect the v1.0.0 release sometime this week. Depends on how many changes I need to make to the README. (I keep changes in the CHANGELOG.md and pluginx.ml, but out of the README on master so that users don't open bugs asking where a feature in the README is in the released version that lags master.) |
|
@KronicDeth thanks! |
Even a simple file like this is completely red and no sensible Syntax highlighting.
The error is something like "ElixirTokenType... or ...[..] expected, but got 'd' "
The text was updated successfully, but these errors were encountered: