Too many element types registered

[hurricup]

I've got this in report to my Perl plugin. This looks like your plugin, but outdated or altered. Could you please clarify?

IdeaLoggingEvent[message=null, throwable=java.lang.Throwable: Too many element types registered. Out of (short) range. Most of element types (8497) were registered for 'Language: Markdown': FILE, NONE, WHITESPACE, EOL, BLANK_LINE, ABBREVIATION, ABBREVIATION_OPEN, ABBREVIATION_CLOSE, ABBREVIATION_SHORT_TEXT, ABBREVIATION_EXPANDED_TEXT, ABBREVIATED, ABBREVIATED_TEXT, ANCHOR_LINK, AUTO_LINK, AUTO_LINK_REF, BLOCK_QUOTE, BLOCK_QUOTE_MARKER, ASIDE_BLOCK, ASIDE_BLOCK_MARKER, BOLD, BOLD_TEXT, ...

[vsch]

@hurricup, thank you for the heads up.

The plugin creates "overlay" attributes which combine multiple element attributes. I went through the code and reduced the number from 8k+ to 3600. This is available in the EAP release now and will be in effect with the next release.

It seems the IDE has a limit on the number of highlighter attribute keys to short which I am guessing the positive range so there are only 32767 allowed combinations across all plugins.

[hurricup]

Yes, there is a limit.

I'd suggest you to change this logic to some custom analog of element types.

Your architecture affecting users in a bad way and forces them to make a choice between your plugin and others.

[vsch]

@hurricup, this logic is used to allow combinations of highlighter attributes to be applied during syntax highlighting. The only way I found was to create combination element types and map them to highlighter attributes. For example: bold, italic, code, strike-through. The numbers really go up because there are many elements for highlighting combinations in tables, headings and other inline elements.

The parsing does not need it. It was made more complex to handle synthetic element types which are synthetic combinations of parser defined ones.

The change to 3600 frees up almost 5000 for other languages. I'll probably add an option to turn off synthetic element types for those that hit the limit or don't care for highlighter combinations.

[hurricup]

Your could use an annotator to highlight some of elements (synthetics for example) instead of highlighter. There is a visual drawback though.

Also see https://github.com/Camelcade/Perl5-IDEA/blob/master/src/com/perl5/lang/perl/idea/highlighter/PerlSyntaxHighlighter.java#L115

[vsch]

@hurricup, I do have an external annotator option but it relies on the lexer to tokenize for the right combinations. The annotator gives better typing response than the lexer but does not syntax highlight files other than editor. For example diff view previous version files.

I also have more flexibility when combining attributes since multiple background colors are combined simulating transparency which is not otherwise available.

What is really needed is the ability to add attributes dynamically to the lookup table or to update the table with new attribute keys. This would allow adding only those attributes which have been encountered in files. However, at this time it is not part of the API, unless I can figure out how to force the IDE to reload the attributes programatically instead of on initialization.

[dancioca]

Just a thought.. (maybe i am missing something in this long tail). Do all the element types need to be registered? There is an option to create Element Types with "registered=false" option. You only need to register the elements you refer in token sets, which become your highlighter configuration subjects. If you still want to lookup the elements by id, you can keep a separate book.

My plugin even though it registers 51 element types, got "blamed" a lot for having registered too many elements. Only recent api changes have made visible which language consumes the most from the big pot.
(https://database-navigator.atlassian.net/browse/DBNE-510)

[vsch]

@dancioca, the issue is really the token types used for syntax highlighting.

Markdown has many (that is an understatement) syntax highlights which can be combined to create feedback in the text window through syntax highlighting.

For example: bold, italic, strikethrough, inline code, superscript, subscript. Additionally all these can be combined with table column/row attributes, definitions, headings. The perfect set would be a combination of all of these.

The only solution I came up with was to generate synthetic combined types whose attributes reflect a merge of parent attributes, including a transparency based merge of background attributes. These are recognized and generated by the parsing/lexing and create visual feedback via syntax highlighting.

There are so many that I only include the most basic which still adds up to 3600. All possible ones would be over 25,000 and I have not even attempted to enable all combinations.

It would be preferable to only register those which are encountered across all markdown files in the project but that is not part of the API.

[dancioca]

Still, i don't see situation where the user would want to configure styles for 3k+ elements. They can be categorized in classes and create element types per class. I had same problem with my plugin (2000+ keywords or reserved names), which then i classified as keywords, functions, identifiers aso... and driven the highlighting by that.

Parsing again is another story. You need to know e.g. "which keyword, which function.." to do proper syntax parsing. I ended up creating two lexers, one for highlighting (registered elements) and one for parsing (unregistered).

And, you are still left with annotation highlighting layer, which has more context around it (as opposite to the initial syntax highlighter)

[vsch]

@dancioca, no one is manually configuring these. The whole purpose is to eliminate the need to create combinations of: bold, italic, strike through, code, etc.

Previously, there were 7 attributes for just bold, italic and strike through. Instead now there are only individual attributes and the combinations are created automatically for all common combinations.

Markdown is not a "syntax" based language but a text formatting language, so you need to think of its syntax highlighting as text formatting directives instead. These can be combined.

• bold
• italic
• strike through
• bold-italic
• strike through bold
• strike through italic
• strike through bold italic

Then you have inline code which can also be added:

• bold code
• italic code
• strike through code
• bold-italic code
• strike through bold code
• strike through italic code
• strike through bold italic code

Now consider other elements like headings, tables, definitions, links, etc.

These are all part of generated attribute combinations for syntax highlighting:

Where editor colors config shows four individual attributes for the above, internally 15 are used to represent nested combinations. Doing its best to combine attributes, including computing background based on overlay with transparency.

[vsch]

EAP update with a fix for the issue is available.

Installing EAP Plugin Version

Added option to disable full highlighter attribute generation. Requires restart to take effect.

Turning off the Full highlighter attributes gives the minimal of 300 attributes. The label right of Attrs: shows synthetic/total attributes stats.