Julia support

[getzze]

I am starting to use Julia (http://julialang.org/) and I realized there was no support for it in Geany.
After some research I found a lexer for Julia in the pygments project (https://bitbucket.org/birkenfeld/pygments-main/src/) in the file ./pygments/lexers/julia.py
Based on the matlab template, the filetypes.Julia.conf file looks like:

# For complete documentation of this file, please see Geany's main documentation
[styling=Matlab/Octave]

[keywords]
# all items must be in one line
# keywords from pygment lexer (http://pygments.org/)
primary=abstract baremodule begin bitstype break catch ccall const continue do else elseif end export for function global if immutable import importall in let local macro module quote return try type typealias using while
#secondary=exit whos edit load is isa isequal typeof tuple ntuple uid hash finalizer convert promote subtype typemin typemax realmin realmax sizeof eps promote_type method_exists applicable invoke dlopen dlsym system error throw assert new Inf Nan pi im
#identifiers=Bool Int Int8 Int16 Int32 Int64 Uint Uint8 Uint16 Uint32 Uint64 Float32 Float64 Complex64 Complex128 Any Nothing None

[settings]
# use builtin lexer for Matlab
lexer_filetype=Matlab/Octave
tag_parser=Matlab/Octave

# default extension used when saving files
extension=jl

# single comments, like # in this file
comment_single=#
# multiline comments
#comment_open=
#comment_close=

[build_settings]
# %f will be replaced by the complete filename
# %e will be replaced by the filename without extension
# (use only one of it at one time)
compiler=
run_cmd=julia "%f"

The matlab template does not support secondary keywords and identifiers (which are the types for Julia), so I commented it out. But with another template (C for instance) they may work.

[b4n]

After some research I found a lexer for Julia in the pygments project (https://bitbucket.org/birkenfeld/pygments-main/src/) in the file ./pygments/lexers/julia.py

Geany uses Scintilla, so we'd need a Scintilla lexer for it.

The matlab template does not support secondary keywords and identifiers (which are the types for Julia), so I commented it out. But with another template (C for instance) they may work.

The supported features depend on the Scintilla lexer used by the filetype, so for custom filetypes on the features of the lexer of the filetype they inherit (with lexer_filetype=...).

[elextr]

After some research I found a lexer for Julia in the pygments project (https://bitbucket.org/birkenfeld/pygments-main/src/) in the file ./pygments/lexers/julia.py

Geany uses Scintilla, so we'd need a Scintilla lexer for it.

Since Julia's syntax is still in flux I would hold off on making a specific lexer for now, (unless you enjoy modifying C++ lexers to keep up with Julia :). The Matlab one seems to do ok for keywords, strings and numbers and doesn't seem to get confused by stuff it doesn't understand, though I only tested it on one file. Since Matlab doesn't have types (IIUC) the Matlab lexer doesn't support highlighting them.

The name of the Matlab tagparser is just Matlab, not Matlab/Octave. It works for functions defined with the function keyword but not for one liners like f(b) = 2 * b and of course it doesn't recognise any type definitions. In the long term a Julia specific parser would be needed.

You need to add a Julia entry in filetype_extensions.conf and add it to the programming languages section near the bottom.

You could put this on the wiki but at the moment its somewhat limited for distribution with Geany IMO.

[peter1000]

Julia support

Just want to express also my interest in geany - julia support

[peter1000]

How to add limited Julia highlight: Linux

SEE: geany custom-filetypes

• Julia support #434 thread by @getzze
• https://groups.google.com/forum/#!topic/julia-users/tisM_9sKPCc
• https://github.com/JuliaLang/IJulia.jl/blob/master/deps/julia.js
• https://github.com/JuliaLang/julia/blob/71c0aa3e5660258af5c042058d5d8d3b82d93efb/contrib/julia.hrc
• keywords from pygment lexer (http://pygments.org/)

Add a Julia filetypes.Julia.conf

create a file: filetypes.Julia.conf in /usr/share/geany/filetypes.Julia.conf in to the users geany config folder

To get the User config dir:
Geany > Help > Debug Messages: User config dir: e.g. /home/username/.config/geany

Copy the text into it / adjust if needed

# filetypes.Julia.conf
#
# For complete documentation of this file, please see Geany's main documentation
# based on 
# https://groups.google.com/forum/#!topic/julia-users/tisM_9sKPCc
# https://github.com/geany/geany/issues/434 
# https://github.com/JuliaLang/IJulia.jl/blob/master/deps/julia.js
# https://github.com/JuliaLang/julia/blob/71c0aa3e5660258af5c042058d5d8d3b82d93efb/contrib/julia.hrc
# keywords from pygment lexer (http://pygments.org/)
#
# Very limited support for julia based on the Matlab filetypes.matlab: 20150317 peter1000
[styling=Matlab/Octave]

[keywords]
# all items must be in one line
# keywords from pygment lexer (http://pygments.org/)
primary=abstract baremodule begin bitstype break catch ccall const continue do else elseif end export finally for function global if immutable import importall in let local macro module quote return try type typealias using while
#secondary=exit whos edit load is isa isequal typeof tuple ntuple uid hash finalizer convert promote subtype typemin typemax realmin realmax sizeof eps promote_type method_exists applicable invoke dlopen dlsym system error throw assert new Inf Nan pi im
#identifiers=Bool Int Int8 Int16 Int32 Int64 Uint Uint8 Uint16 Uint32 Uint64 Float32 Float64 Complex64 Complex128 Any Nothing None

[settings]
# use builtin lexer for Matlab
lexer_filetype=Matlab/Octave
tag_parser=Matlab

# default extension used when saving files
extension=jl

# single comments, like # in this file
comment_single=#
# multiline comments
#comment_open=#=
#comment_close==#

[indentation]
width=4
# 0 is spaces, 1 is tabs, 2 is tab & spaces
type=0


[build_settings]
# %f will be replaced by the complete filename
# %e will be replaced by the filename without extension
# (use only one of it at one time)
compiler=
run_cmd=julia "%f"

Add Julia to: filetype_extensions.conf (/home/username/.config/geany/filetype_extensions.conf)

Geany > Tools > Configuration Files > filetype_extensions.conf

add a line:

Julia=*.jl;

add Julia to the [Groups]

Programming=Julia;Clojure;CUDA;Cython;Genie;Go;Rust;Scala;

[codebrainz]

This would probably be useful to put on the wiki:
http://wiki.geany.org/filetype?do=showtag&tag=filetype

[peter1000]

@codebrainz

I did not register for the wiki but feel free to copy it there.

[oscardssmith]

bump. What does this need to be finished?

[elextr]

Somebody to do it I guess.

[waynelapierre]

Any updates?

[elextr]

Lets be clear here, Geany is a unpaid volunteer project, there is no project sponsor, no paid devs, or committers, or reviewers, or testers, no hoards of postgrads, or doctoral academics (hi Jeff 😁), nothing happens unless somebody does it .

Its up to the Juliaists to get together and make the pieces and make a pull request, get it third party tested and reviewed, and, after the current situation calms down committers may have more time available, but at the moment several have not been heard from for some time.

[end rant, sorry]

[abianco88]

New to Geany hacking (I fully understand that Geany is an unpaid volunteer project), but I'm also interested in Julia support. Based on a comment by @elextr in #2279, to add Julia support to Geany, there are 3 steps to follow (create a custom filetype file, add a Scintilla lexer and add a uctags parser), but some initial functionality may be already achieved by creating a custom filetype file that relies on existing Scintilla lexer and uctags parser. If that's the case, besides being over 5 years old, what are the issues with the custom filetype proposed above by @peter1000? Generally, what's needed to make a good Julia filetype file?

Moreover, I started looking at the Geany manual, the Hacking Geany guide and the Scintilla lexer reference and I was wondering whether one should first create a lexer and a parser and then work at the filetype file. Finally, is there some example to look at to create a working and exhaustive lexer and parser for Julia? The Scintilla lexer documentation is a bit intimidating...

[waynelapierre]

VSCode is too slow and heavy. Geany is my favorite editor. If Geany can provide the basic auto-completing and block code running functionality for Julia, I will uninstall VSCode.

[abianco88]

That was exactly my line of thought: VSCode is nice and sleek, but it's too heavy even with just the Julia extension loaded. If we could get Geany to work with Julia, we would (finally) get a lightweight editor. Would it be possible to raise enough interest to put together someone willing to support this?

[waynelapierre]

To be honest, I am not an expert on such things. I think it should not be very difficult to add basic Julia support to Geany.

[abianco88]

If some expert from Geany were to provide some guidance and help, we could give it a try.

Not sure if it's a good idea, but would it be possible to start with what has been done for Atom and/or VS Code? One other difficulty I haven't mentioned earlier is to collect all keywords from Julia base and possibly common packages.

[waynelapierre]

I still find it shocking that Geany supports so many programming languages but Julia has not been considered for so many years. I am not complaining. It is just a shocking fact to me.

[getzze]

If that's the case, besides being over 5 years old, what are the issues with the custom filetype proposed above by @peter1000? Generally, what's needed to make a good Julia filetype file?

I'm using it everyday and probably I got used to it but I see 3 big issues that require writing a Scintilla lexer specially for Julia:

• using splatting "..." of varargs makes the end of the line being recognized as comment and lose coloration.
• the "end" keyword sometimes doesn't match the beginning of the block (for, if, function...), I don't know if it's because you can use it for indexing also, but the code collapsing function is not working at all because of that.
• complex strings with interpolation is not correctly parsed, making all the line losing color and even sometimes the next lines.

Also minor problems are:

• symbols are not recognized as special type (like strings) so it is not easy to find them in the code.
• type annotations with ::, <: are not emphasized
• macros are not colored either

I tried to work on the Scintilla lexer but the documentation is too obscure.

[abianco88]

Therefore, besides updating the proposed custom filetype to Julia 1.5 and possibly improving it, there is still a real need to write a Scintilla lexer and a uctags parser (I assume this would address the issues with type annotations and symbols). I totally agree that the documentation is too obscure, at least to someone who isn't used to work on those things like myself, that's exactly why it would be great if we could get some help/guidance from Geany/Scintilla contributors to make it happen.

I have no idea how many Julia users are out there, but I just can't believe they all use either Atom/Juno or VS Code! OK, I admit I know at least one that uses Brackets, but still it would be much better if Geany were among the options...

[codebrainz]

that's exactly why it would be great if we could get some help/guidance from Geany/Scintilla contributors to make it happen.

Both projects' respective mailing lists have a fair few helpful and knowledgeable people:

https://geany.org/support/mailing-lists/
https://groups.google.com/g/scintilla-interest?pli=1

Geany also has an IRC channel (#geany on FreeNode) that often has one or two developers in it, depending on the timezone.

Also it's probably useful to look at various other such endeavors to see where to edit Geany's code:

07002d2
94aa892
b1f93c2

For Scintilla, there's also a useful tutorial (using a language as obscure as Julia 😉) that explains how to implement a lexer. It used to be linked from Scintilla's main documentation page but it 404's here. I found it hosted elsewhere:

https://www.coursehero.com/file/45607974/Scintilla-varaq-Tutorialpdf/

[elextr]

@abianco88 are you volunteering to start writing the Lexer? Feel free to post a link to your github repository where its happening so others can assist you.

Now that Julia is (somewhat) settled syntax the Lexer should be possible, but it will do nothing with types, the lexers are called lexers because they only recognise syntactic elements, they have no knowledge of semantics such as types and type annotations, thats the purpose of the uctags parsers. This keeps lexers light and fast so they can be run as you type while parsers are heavier and do more work and generally run less often. Highlighting of typenames in languages that support it is provided by the uctags typename list results being transferred to the lexer by Geany as if they were keywords.

I'm not sure how good a static Julia parser would be, certainly it could recognise function definitions (although one liners might still be a challenge) but for a dynamic, heavily inferred language like Julia its not going to do much else. Also there is the issue that Unicode is not well supported by uctags and there might be issues there.

[abianco88]

For Scintilla, there's also a useful tutorial (using a language as obscure as Julia 😉) that explains how to implement a lexer. It used to be linked from Scintilla's main documentation page but it 404's here. I found it hosted elsewhere:

https://www.coursehero.com/file/45607974/Scintilla-varaq-Tutorialpdf/

Is there another host for the tutorial @codebrainz suggested? I can only see the first 3 pages without registering.

@elextr I am considering giving a try at writing a Julia lexer, but I can't foresee my time availability at the moment. I have no prior knowledge of lexers/parsers, so I would like to start looking at the reference material at the very least.

[waynelapierre]

timothywalsh seems to have started working on a Julia lexer for Textadept, which is also based on Scintilla. You may ask him for the Scintilla part.

Textadept is a one-person project and lacks adoption while Geany is more mainstream.

https://discourse.julialang.org/t/is-anyone-using-textadept-as-their-editor-of-choice-for-julia/41255

[HybridDog]

Is there another host for the tutorial @codebrainz suggested?

The file has been archived.
To be safe, I've reuploaded it here: Scintilla-var'aq-Tutorial.pdf

[abianco88]

@HybridDog thank you for sharing!

[elextr]

Probably the best tutorials are the existing Scintilla lexers, perhaps the Python one might be a place to start.

[getzze]

I made a pull-request with a lexer I just made using Matlab, Python and other existing lexers.
So far:

• using splatting ... and % make the end of the line being recognized as comment
• parse begin and end as numbers inside indexing brackets, so code folding works.
• complex strings with interpolation are not correctly parsed.
• macros are not colored
• parse unicode operators and variables
• symbols are not recognized as special type (like strings).
• type annotations with ::, <: are not emphasized

[abianco88]

@getzze that's amazing! As for the last 2 bullets of your list, is a parser needed to address them?

[getzze]

I think it can be done in the lexer, by parsing the : character.

If I understood correctly, the ctags parser is used for completion, it would be great to implement it also.

[elextr]

@getzze the ctags parser is used for the symbol pane and passing typenames to the lexer, as well as completions.

But yes the Julia syntax is not context sensitive (unlike say C) and you can recognise and highlight :: and other type operators in the lexer (along with all the myriad of operators Julia allows as your point 5 says). Also of course you can also highlight all the keywords like "primitive type abstract struct" etc and do folding on them.

Whilst you could highlight identifiers that are recognised as typenames in a file, I wouldn't recommend it since the lexer has no way to know about typenames from outside the file (from modules etc) so only local typenames would be highlighted and that would not be very helpful (IMHO, Julia editor without Array highlighted would be sacrilege 😁).

What most of the lexers do is accept lists of identifiers to highlight differently to the default identifiers, a feature that was meant to allow for additional keywords, but which Geany (ab)uses to identify types that the ctags parsers recognise, and in some languages to highlight differently the "built-in" types.

You could use a static list of builtins and standard library in filetypes.Julia until your parser is working and you can make a tags file for standard library stuff to be loaded when the first Julia file is loaded (as is done for Python and C and others).

[waynelapierre]

Any updates?

[codebrainz]

@waynelapierre #2584

[waynelapierre]

Is there any simple way of sending code from Geany to Julia REPL?

[elextr]

You should be able to run Julia repl in the terminal window and copy and paste code into it.

[codebrainz]

@waynelapierre there is a keybinding (unbound by default) Format->Send Selection to Terminal that might be useful.