Syntax highlighting service

[melonaerial]

Hello.
I've installed yavide and I see that my class variables don't highlighted with special color like on your demo for Syntax hightlighting service. I've attached my test project on which I'm testing your IDE. Service is setted to enable in <yavide_install_dir>/core/.globals.vimrc file. So here is the screenshot.

testproject.zip

[melonaerial]

I want to see something like this!

So as you cna see on this screenshot all variables of class highlighted with special color. That's look good. So how can I don it with you service ? For that look in vim I've used color_coded plugin. But you plugin working asynchroniously that sounds great, because color_coded doesn't work in such mode.

[JBakamovic]

Hi @melonaerial,

I think that you may not be using yaflandia colorscheme and that is why you don't see the effect of syntax highlighting. The reason you have to use that particular colorscheme is because standard colorschemes do not support displaying C/C++ symbols at that granularity (i.e. enums, enum values, local variables, functions, classes etc.). You have two choices here:

1. Switch to the yaflandia colorscheme to see the effect of syntax highlighting instantly.
2. Use a colorscheme of your preference and tweak it a little bit to make it compatible with Yavide syntax highlighting service. For example, following file, cpp_syntax_highlight.vim, defines additional symbols which syntax service will understand (and generate) and those simply need to be mapped to particular colors in a colorscheme you want to use. See how yaflandia colorscheme does it. It's trivial ... you just have to pick the colors for the given symbols: yaflandia.

However, under the hood Yavide syntax highlighter utilizes ctags to generate those symbols. ctags actually isn't anymore the best solution for that purpose (even though it mostly does a decent job) but particularly it does not cope well with modern C++ code. Hence, my wish is to replace ctags with clang based approach which is, for example, used by color_coded plugin but I haven't had much time recently to update the implementation.

Developing the async processing framework was what I was mostly focused on. This was my prio number one as without this framework implementing and integrating anything into the environment simply does not make any sense. Blocking the main UI thread while running the indexer/syntax highlighter/build/formatter/etc. was simply not an option.

[melonaerial]

Hello @JBakamovic ,
Thanks for such detailed answer. Yes, blocking UI thread is not an option and that's where cool that you have made such big work on async service. That's cool. But for colorsheme, I'm already using yaflandia from your repo, so it is default configuration of yavide and yaflandia is in default config. Maybe ctags doesn't work or something on my computer. Have can I see some debug information about running of Syntax hightlighting service ?

[JBakamovic]

Hi @melonaerial,

I've tried out your test project on my machine and syntax highlighting seems to be running fine. Did you forget to import the project? You need to create a yavide project in order to get all the services running. Handling projects is easy as it can be and amongst other things it is described in usage document. Following is a small excerpt from that documentation:

1. <Ctrl-s>n Create new project
2. <Ctrl-s>i Import project with already existing code base
3. <Ctrl-s>o Open project
4. <Ctrl-s>c Close project
5. <Ctrl-s>s Save project
6. <Ctrl-s>d Delete project

So, you can try to create a new yavide project by importing already existing code base (see 2.).

[JBakamovic]

Hi @melonaerial,

With commit 9d44c4b I have released an update of syntax highlighting service. It is now using a libclang based back-end. You should now be able to get more satisfying results. See docs for more details.

You can give it a try if you feel like and let me know how it went :)

P.S. Don't forget to update the colorscheme as well by running cd <yavide_install_dir>/colors/yaflandia && git pull. It's not automated yet :/

[iperez319]

Im having the same problem both in Windows 10 and Lubuntu vm version.

[JBakamovic]

Did you try out the steps above, i.e. import a project? You will not get any services running until you create a project context.

[iperez319]

I created a test project

[iperez319]

http://gph.is/2rXq7g8

[JBakamovic]

Hm, in the video I can see that the file you're editing is test.cpp but I cannot see that file in the file-explorer on the left-hand side. Is this file part of that directory? If not, then it is not going to work like that.

Otherwise, you can attach the log which you can find under /tmp/YAVIDE_server.log so I can have a look what is happening.

[iperez319]

The file exists in the directory but does not appear in nerdtree.

YAVIDE_server.log - there seems to be an error

[INFO] [yavide_server.py:124]         yavide_server_run(): Starting a Yavide server ...
[INFO] [yavide_service.py:17]                  __init__(): Yavide instance: YAVIDE
[INFO] [yavide_service.py:18]                  __init__(): Actions: {0: <bound method SourceCodeModel.__startup_request of <services.source_code_model_service.SourceCodeModel instance at 0x7fb58ffe3758>>, 1: <bound method SourceCodeModel.__shutdown_request of <services.source_code_model_service.SourceCodeModel instance at 0x7fb58ffe3758>>, 2: <bound method SourceCodeModel.__request of <services.source_code_model_service.SourceCodeModel instance at 0x7fb58ffe3758>>}
[CRITICAL] [yavide_server.py:99]          handle_exception(): Uncaught exception
Traceback (most recent call last):
  File "/opt/yavide/core/server/yavide_server.py", line 128, in yavide_server_run
    YavideServer(msg_queue, yavide_instance).listen()
  File "/opt/yavide/core/server/yavide_server.py", line 16, in __init__
    0x0 : SourceCodeModel(self.msg_queue, self.yavide_instance),
  File "/opt/yavide/core/services/source_code_model_service.py", line 14, in __init__
    self.parser = ClangParser()
  File "/opt/yavide/core/services/parser/clang_parser.py", line 71, in __init__
    self.index = clang.cindex.Index.create()
  File "/usr/local/lib/python2.7/dist-packages/clang/cindex.py", line 2218, in create
    return Index(conf.lib.clang_createIndex(excludeDecls, 0))
  File "/usr/local/lib/python2.7/dist-packages/clang/cindex.py", line 141, in __get__
    value = self.wrapped(instance)
  File "/usr/local/lib/python2.7/dist-packages/clang/cindex.py", line 3575, in lib
    lib = self.get_cindex_library()
  File "/usr/local/lib/python2.7/dist-packages/clang/cindex.py", line 3606, in get_cindex_library
    raise LibclangError(msg)
LibclangError: libclang.so: cannot open shared object file: No such file or directory. To provide a path to libclang use Config.set_library_path() or Config.set_library_file().

[iperez319]

I'll start writing the instructions for Windows once I get it working in the vm.

[JBakamovic]

You're missing a libclang.so dependency. It should have been installed with the install.sh script though. I am not sure why is that not the case but nevertheless you can install this dependency very easily by yourself. Not sure which distro are you running, but instructions to install it would be distro-specific and are usually straightforward.

Let me know if that fixed the issue for you.

[iperez319]

Have not found the solution. What distro are you running it in?

[iperez319]

Nevermind, I got it to work. I had to edit the cindex.py file in the clang python directory.

[JBakamovic]

I am running Fedora almost exclusively and I haven't experienced such issues. So, if I understood you well, you had libclang.so installed on your system but cindex.py was not able to set the right path to it? Which distro are you running? Can you please share what modification you did so others having the same issue can benefit from it?

[iperez319]

OS: Lubuntu and Ubuntu in Windows 10

First I tested this code in the python shell with the different paths until I did not get the error:

import clang
import clang.cindex
clang.cindex.set_library_file( {DIRECTORY} )
index = clang.cindex.Index.create()

In my case the one that worked was: /usr/lib/x86_64-linux-gnu/libclang-4.0.so.1
Then I hardcoded the file path in the cindex.py file located in /usr/local/lib/python2.7/dist-packages/clang/cindex.py in this line.

[iperez319]

Although it is not autocompleting the algorithms standard library

[JBakamovic]

Hm, auto-completion is handled via clang_complete. I would expect that it should be working out-of-the-box for STL but if not, you can edit .clang_complete file which is located in the root of your project directory. This file should include paths to system include directories as well as include directories specific to your project: i.e. -I flag with corresponding paths that you would normally pass to your compiler.

With respect to libclang.so path problem, I believe that the culprit is that libclang library is not part of your system dynamic linker cache configuration and that is the reason why cdll.LoadLibrary('libclang.so') is failing for your system. You can easily check that by running the following command: ldconfig -p | grep libclang.so. I believe the output will be empty in your case. In my case it looks like:

libclang.so.3.9 (libc,x86-64) => /lib64/libclang.so.3.9
libclang.so (libc,x86-64) => /lib64/libclang.so

If my assumptions are right then you will be able to fix the problem by tweaking the LD_LIBRARY_PATH so it includes the directory where libclang library resides. It's a cleaner solution as it does not require you to mess up with the cindex.py.