On limitations of Geany for C++

[elextr]

Despite the heroic efforts of the Uctags crew and @techee and others there are some limitations that Geany can't overcome when using assistance features in C++.

Since its come up a few times in various issue/PR discussions I thought it might be useful to summarise the fundamental limitations imposed by the underlying design of the Geany/Uctags/Scintilla combination in relation to C++. Or to put it another way, how C++ has outgrown that design which was "adequate" for plain C.

I concentrate on C++ because I use and know it and because it has a Uctags parser that is more capable than most and because it manages to include nearly all the problems that the Geany design has for other languages 😄.

Hopefully an overall understanding of the limitations can guide a coordinated approach to heuristics that attempt to improve usability rather than individual heuristics being applied independently as happens now.

These are in no special order.

1. Scintilla semantic styling (keywords, types, and the rest) for names is global, but C++ can re-use names for objects with different semantics in different scopes (foo can be a type in one scope but a variable or function in another), in different contexts (eg final and override are only keywords in specific contexts, they are variables elsewhere), and even in the same scope if a name is a struct/class/union type, a variable of the same name is allowed and hides the typename.

   Idioms like "first letter caps"ing typenames help, but the STL, Boost++ and other major libraries have all lower case typenames, so if there tags are loaded local variables of the same name are shown by Scintilla as types.

2. Can't follow includes, neither Geany nor Uctags can see includes because the dependencies are specified in the build system and neither can understand those. The result is that the information provided to users is compromised, see below.

3. Types of declared entities depends on declarations in the include files, which are not read, so the variable type is just an opaque name, no members, member functions or constructors information is available either to Geany or the user.

4. Types of expressions are not deduced so inferred types cannot be deduced, eg auto a = foo(bah); This is for two reasons:

   a. Uctags doesn't have any capability to do it, even the type of 1 + 1.0
   b. even if it did have the capability the lack of accurate type information and template definitions from the include files means it cannot.

5. The lack of type information means that assistance features (autocomplete, calltips) are empty.

6. To avoid the total lack of accurate assistance features Geany attempts to use name based lookup, but that spams the lists with many irrelevant names. There is some attempt to filter or to rank by relevance, but its not particularly good simply because it does not have the information to work off.

7. Tags do not provide lexical scope information for variables, so name hiding cannot be deduced, so again name based lookup will show irrelevant options.

8. When C++ modules start being used in the wild they will suffer from all the include file problems and a few extras.

How do a few other editor/IDEs do it?¹

• Vscode - via plugin API, C/C++ is custom plugin IIUC, but one plugin talks Language Server Protocol to a Language Server Process (LSP) for whatever the language is.
• Visual studio - C/C++ builtin custom MS intellisense, has LSP interface for other languages
• Eclipse - C/C++ custom Java code plugin, plugin talking to LSP for other languages eg Rust, C# and more
• Qt creator - LSP for some languages
• Sublime text - LSPs
• Xcode - LSPs for Swift, C/C++ (which is built on clangd)
• Code Blocks - C/C++ plugin talking to clangd via LSP
• Notepad++ - various alpha stage LSP plugins, but IIUC notepad does not use Lexilla.

I am somewhat surprised at how fast LSP use has infiltrated the big guys like MS, Eclipse, and Apple although C/C++ are still supported by legacy code that existed before LSPs in some cases.

So am I suggesting Geany should use LSPs, probably not, the previous attempt to use clangd showed that hard coded ways of doing things is heavily entrenched in Geany, its would take nearly a rewrite. Geany 2.0/GTK4 "somebody", anybody?

Footnotes

1. this list is to the best of my and googles knowledge in 1/2 an hour, it may contain erorrs and is certainly incomplet ↩

[ntrel]

Scintilla semantic styling (keywords, types, and the rest) for names is global

Then Geany should not tell Scintilla to highlight local type names.

final and override are only keywords in specific contexts

Seems like they aren't keywords in the Scintilla sense.

if there tags are loaded local variables of the same name are shown by Scintilla as types.

Maybe too difficult for Scintilla to fix?

Can't follow includes

C++ 20 has modules (only 45 years late!). Not sure how much work it is for C++ projects to switch to modules? Although compiler support might not be there yet. The preprocessor is the source of so many headaches with parsing. I expect supporting modules would be much more straightforward than supporting includes.

---

A LSP does sound like the way to go for all things related to parsing, though I'm not sure that syntax highlighting needs to be too sophisticated. The latter means we can stick with Scintilla.

One thing that would be good is to support an external ctags program - that way e.g. I could use the official dtags program written in D without making Geany depend on a D compiler. That would greatly improve parsing from either geany_c.c's support or uctags.

[elextr]

Then Geany should not tell Scintilla to highlight local type names.

In general the majority of C++ types are in header files so they are not local to my mind. But i'm not sure what you mean by local type names? And simply removing semantic information from some names isn't really a helpful solution if that makes them look different when semantically they are not.

Seems like they aren't keywords in the Scintilla sense.

Yes, the Scintilla sense does not match what C++ does, thats the point.

Maybe too difficult for Scintilla to fix?

It is with the current (ab)use of global word lists as typenames yes.

Although compiler support might not be there yet.

IIUC MSVC does most of the modules spec, gcc and clang do the majority (but different majorities) so not all the fancy details are available yet, but the basics are there. Thats why not much is in the wild with modules yet.

The preprocessor is the source of so many headaches with parsing. I expect supporting modules would be much more straightforward than supporting includes.

Certainly things like #defines can't cause changes to modules, so thats a blessing, but there is still the issue of getting the declarations that are exported in the module interface into where the module is imported. And there is still the issue of the module lookup path that is specified on the compiler command line (for clang at least) so the build system interaction is still needed to find them.

And of course the dear olde STL is still there, so includes won't be going away anytime soon.

A LSP does sound like the way to go for all things related to parsing

It does seem to be the flavour of the day for IDE/Editors yes, and that means that if there is a LSP for one IDE it can be used by others, thats the idea. Google finds at least one for D.

One thing that would be good is to support an external ctags program

That would be slower than the current system of course, but then if its a separate process, not inline with the UI, it can be more thorough. The LSP is of course an external program too, and the method the editor/IDEs I have seen that use LSPs, handle the delay by having a syntax colouring that runs fast inline with typing, and the semantic colouring is provided later from the parallel process of the language server. It would require some adjustment of the syntax lexers to not overwrite "semantic" styles, or an adjustment of Scintilla to store two sets of styling and use the semantic styling first and fallback to the syntax styles if its not present.

A LSP does sound like the way to go for all things related to parsing though I'm not sure that syntax highlighting needs to be too sophisticated. The latter means we can stick with Scintilla.

I suspect that integrating the styling as outlined above would be not too hard, what would be difficult would be to find all uses of tagmanager within Geany and redirect the queries to the LSP given that tagmanager is synchronous, but the LSP is asynchronous.