[RFC] How to highlight Cpp2

[JohelEGP]

Title: [RFC] How to highlight Cpp2.

Description:

After #429,
I've written another Cpp2 parser at compiler-explorer/compiler-explorer#5061
to highlight code in the editor used by Compiler Explorer.
I've adopted the existing highlighting as-is.
I also took the opportunity to highlight type-only contexts.

I'd like to know
your opinions on my choices
and what you think can be improved.
Please, be aware that
there is science behind using color to represent information.
Color overload can also be detrimental.
So I'd prefer baby-step improvements
to a scheme that isn't demonstrated to benefit the Cpp2 programmer.

For reference, here is the one of two highlighters in the wiki that has a screenshot:
https://github.com/elazarcoh/cpp2-syntax/blob/master/resources/syntax-highligh-example.png.

Preview:

In terms of what Cpp1 and Cpp2 have in common,
the only difference in highlighting is that
Cpp2 highlights declarations and type-only contexts.

---

The default sample code.

---

• Highlighted interpolation.
• Type-only contexts.
• int receives no special treatment.
• _ is a bit hard to see.
  As an anonymous declaration, deduced type, and placeholder, it is highlighted like a keyword.
  As a non-anonymous declaration (a parameter) and use of an identifier (none of the others),
  it is highlighted like any other declaration and use.

---

Benefit of contextual type highlighting.
Know at a glance that something might be wrong.

---

A type nested in a template.
The instantiated template can only be a type.

---

Metafunctions.
See what Herb said at commit d8c1a50:

Metafunction quote:

Re syntax: I considered several options to request a meta function m be applied to the type being defined, including variations of is(m) and as(m) and type(m) and $m. I'm going with @m for now, and not because of Python envy... there are two main reasons:

• I think "generation of new code is happening here" is such a fundamental and important new concept that it should be very visible, and actually warrants taking a precious new symbol. The idea of "generation" is likely to be more widely used, so being able to have a symbol reserved for that meaning everywhere is useful. The list of unused symbols is quite short (Cpp2 already took $ for capture), and the @ swirl maybe even visually connotes generation (like the swirl in a stirred pot -- we're stirring/cooking something up here -- or maybe it's just me).

• I want the syntax to not close the door on applying meta functions to declarations other than types. So putting the decoration up front right after : is important, because putting it at the end of the type would likely much harder to read for variables and especially functions.

The default highlighting makes it all white.
Consistent with the provisions above (meta, science),
suggestions welcome.

---

A type declaration.
TBD Bold font for overloaded operator declaration consistent with other declarations.

Inspect.

jump-statement.

Iteration statements.

Contracts.

Uses-it-all declaration.

The example from https://github.com/hsutter/cppfront/wiki/Design-note%3A-Postfix-operators.

Anonymous return type.

Contextual highlighting of an identifier named after a parameter direction.

Special identifiers.

Chained comparisons.

Consistency with Cpp1.

Monaco's C++ isn't ready for C++23's assume attribute.

Q&A:

Comment: Highlighting types in type-only contexts is color overload/unscientific.
Reason: I used Monarch's built-in type (CSS class for a token), so I hope it isn't.

Comment: Highlighting types is inconsistent with Cpp1.
Reason:
Monaco's C++ and all language definitions at CE
don't parse the language, but rather
use rules to decide how to highlight the code.
This means, as seen in the preview,
that there's less chance for contextually correct highlighting.
E.g., in Cpp1, final is always highlighted as a keyword.

Cpp2 has a greater set of contextual keywords,
and is simpler to parse.
So I took the chance to actually write a parser for it
and use contextual knowledge for correct highlighting.

What I want to do next:

• Feedback the learnings of writing the parser into Clang Format's.
• Look into adding support for GitHub to highlight ```Cpp2 code blocks.

[msadeqhe]

Thanks.

• Look into adding support for GitHub to highlight ```Cpp2 code blocks.

It would be really nice. Currently I use ```haskell to syntax highlight some of my examples in a cluttered way.

[JohelEGP]

I chose to not highlight discouraged spellings of fundamental types.

Here's a glimpse as to why:

cppfront/include/cpp2util.h

Lines 251 to 269 in ebe5ebc

	// Encouraged by default: Fixed-precision names
	using i8 = std::int8_t ;
	using i16 = std::int16_t ;
	using i32 = std::int32_t ;
	using i64 = std::int64_t ;
	using u8 = std::uint8_t ;
	using u16 = std::uint16_t ;
	using u32 = std::uint32_t ;
	using u64 = std::uint64_t ;
	// Discouraged: Variable precision names
	// short
	using ushort = unsigned short;
	// int
	using ulong = unsigned long;
	// long
	using longlong = long long;
	using ulonglong = unsigned long long;
	using longdouble = long double;

I made an exception for longdouble, which has no alternative.

This does means that main,
which has a return type of int,
won't have it highlighted like a keyword.

[JohelEGP]

Look into adding support for GitHub to highlight ```Cpp2 code blocks.

I found the criteria at github-linguist/linguist's CONTRIBUTING.md (followed from here),
which also generates the "language breakdown graph".

Adding a language

We try only to add languages once they have some usage on GitHub.
In most cases we prefer that each new file extension be in use in at least 200 unique :user/:repo repositories before supporting them in Linguist
(but see [#5756][] for a temporary change in the criteria).

[JohelEGP]

For purposes of linguistic (to highlight Cpp2 in GitHub code blocks),
we need the strict

each new file extension be in use in at least 200 unique :user/:repo repositories

or the relaxed

• at least 2000 files per extension indexed in the last year [...]
• with a reasonable distribution across unique :user/:repo combinations [...]

It seems there are 536 unique .cpp2 sources hosted on GitHub
(https://github.com/search?q=%28path%3A*.cpp2+AND+%28content%3A%2F%3A+%3F%5B%3D%28%5D%2F+OR+content%3A%2F%5Ba-zA-Z0-9_%5D%5C%24%2F+OR+content%3A%2F%5B-%2B*%26%7E.%29%5D%5C%24%2F%29%29+NOT+is%3Afork&type=code).
Including .h2 sources adds too much noise.