revised documentation

ChangeLog

@@ -1,6 +1,6 @@
 CHANGELOG Highlight
 
-xx.xx.2017
+20.10.2017
 
 highlight 3.40
 
@@ -10,7 +10,8 @@ highlight 3.40
 -added CMake definition (https://github.com/andre-simon/highlight/issues/20)
 -added email definition (https://github.com/andre-simon/highlight/issues/21)
 -linked scm suffix to lisp definition (https://github.com/andre-simon/highlight/issues/22)
--W32 CLI: support HIGHLIGHT_DATADIR and --data-dir options (https://github.com/andre-simon/highlight/issues/24) 
+-W32 CLI: support HIGHLIGHT_DATADIR and --data-dir options (https://github.com/andre-simon/highlight/issues/24)
+-revised documentation
 
 ---
 highlight 3.39

README

@@ -1,5 +1,5 @@
 -------------------------------------------------------------------------------
----  HIGHLIGHT MANUAL - Version 3.37 ----------------------------- May 2017 ---
+---  HIGHLIGHT MANUAL -------  Version 3.40   --------------- November 2017 ---
 -------------------------------------------------------------------------------
 
 OSI Certified Open Source Software
@@ -593,7 +593,49 @@ HL_IDENTIFIER_END
 HL_UNKNOWN
 HL_REJECT
 
-Hook functions: see README_PLUGINS.
+The function OnStateChange:
+
+This function is a hook which is called if an internal state changes (e.g. from
+HL_STANDARD to HL_KEYWORD if a keyword is found). It can be used to alter
+the new state or to manipulate syntax elements like keyword lists.
+
+OnStateChange(oldState, newState, token, kwGroupID)
+
+  Hook Event: Highlighting parser state change
+  Parameters: oldState:  old state
+              newState:  intended new state
+              token:     the current token which triggered the new state
+              kwGroupID: if newState is HL_KEYWORD, the parameter
+                         contains the keyword group ID
+  Returns:    Correct state to continue OR HL_REJECT
+
+Return HL_REJECT if the recognized token and state should be discarded; the 
+first character of token will be outputted and highlighted as "oldState".
+
+Examples:
+
+function OnStateChange(oldState, newState, token)
+   if token=="]]" and oldState==HL_STRING and newState==HL_BLOCK_COMMENT_END then
+      return HL_STRING_END
+   end
+   return newState
+end
+
+This function resolves a Lua parsing issue with the "]]" close delimiter which
+ends both comments and strings.
+
+
+function OnStateChange(oldState, newState, token)
+   if token=="'" and oldState==HL_NUMBER and newState==HL_STRING then
+      return HL_NUMBER
+   end   
+   return newState
+end
+
+This hook enables correct parsing of C++14 number separator syntax.
+
+See README_PLUGINS for other available functions.
+
 
 Example:
 --------

README_DE

@@ -1,5 +1,5 @@
 -------------------------------------------------------------------------------
----  HIGHLIGHT MANUAL - Version 3.37 ----------------------------- Mai 2017 ---
+---  HIGHLIGHT HANDBUCH -----  Version 3.40   --------------- November 2017 ---
 -------------------------------------------------------------------------------
 
 OSI Certified Open Source Software
@@ -359,8 +359,6 @@ Text-Ausgabe:
 
 Wird als Sprachdefinition txt angegeben, findet keine Syntaxhervorhebung statt.
 
-Beispiel:
-
 highlight -S txt --out-format=latex README > readme.tex
 
 
@@ -452,7 +450,7 @@ Portable GUI (Windows build)
 Starten Sie highlight-gui.exe mit der --portable Option, damit die
 Konfiguration in Textdateien und nicht in der Registry gespeichert wird.
 
-2.7 ENVIRONMENT VARIABLES
+2.7 UMGEBUNGSVARIABLEN
 -------------------------------------------------------------------------------
 
 Die Kommandozeilenversion beruecksichtigt folgende Variablen:
@@ -579,7 +577,6 @@ Globale Variablen:
 Die folgenden Variablen sind in einer Sprachbeschreibung verfuegbar:
 
 HL_LANG_DIR: Verzeichnis der Sprachdefinitionen (Parameter der Lua dofile-Funktion)
-
 Identifiers: Default regex fuer Bezeichner
 Digits:      Default regex fuer Zahlenliterale
 
@@ -612,7 +609,52 @@ HL_IDENTIFIER_END
 HL_UNKNOWN
 HL_REJECT
 
-Hook-Funktionen: siehe README_PLUGINS.
+Die Funktion OnStateChange:
+
+Dieser Hook wird bei Zustandsuebergaengen des Parsers aufgerufen (z.B. beim 
+Wechsel von HL_STANDARD zu HL_KEYWORD wenn ein Schluesselwort erkannt wurde).
+Mit dieser Funktion kann der neue Zustand angepasst werden, oder es koennen
+Syntax-Elemente wie Keyword-Listen erweitert werden.
+
+OnStateChange(oldState, newState, token, kwGroupID)
+
+  Hook Event: Zustandswechsel des Parsers
+  Parameters: oldState:  bisheriger Zustand
+              newState:  geplanter neuer Zustand
+              token:     das Token welches den Wechsel ausgeloest hat
+              kwGroupID: Wenn newState = HL_KEYWORD, enthaelt dieser Parameter
+                         die Gruppen-ID 
+  Returns:    den korrekten Zustand zum fortfahren ODER HL_REJECT
+
+HL_REJECT wird dann zurueckgegeben, wenn das Token und der erkannte Zustand
+verworfen werden sollen; das erste Zeichzen von Token wird dann ausgegeben und 
+als "oldState" hervorgehoben.
+
+
+Beispiele:
+
+function OnStateChange(oldState, newState, token)
+   if token=="]]" and oldState==HL_STRING and newState==HL_BLOCK_COMMENT_END then
+      return HL_STRING_END
+   end
+   return newState
+end
+
+Diese Funktion loest ein Parsing-Problem in Lua, weil der Begrenzer "]]" Kommentare
+und Strings abschliessen kann.
+
+
+function OnStateChange(oldState, newState, token)
+   if token=="'" and oldState==HL_NUMBER and newState==HL_STRING then
+      return HL_NUMBER
+   end   
+   return newState
+end
+
+Dieser Hook ermoeglicht das korrekte Parsen von C++14 Zahl-Literalen.
+
+Weitere Funktionen sind in README_PLUGINS beschrieben.
+
 
 Beispiel:
 ---------
@@ -815,7 +857,7 @@ damit diese im GUI-Dateiauswahldialog als Filter angezeigt werden.
 3.8 PFADE DER KONFIG-DATEIEN
 -------------------------------------------------------------------------------
 
-Die Konfigurationsskripte in den folgenden Verzeichnissen gesucht:
+Konfigurationsskripte werden in folgenden Verzeichnissen gesucht:
 
 1. ~/.highlight/
 2. Wert der Umgebungsvariablen HIGHLIGHT_DATADIR

README_PLUGINS

@@ -1,5 +1,5 @@
 -------------------------------------------------------------------------------
----  HIGHLIGHT PLUGIN MANUAL - Version 3.36   ------------------ March 2017 ---
+---  HIGHLIGHT PLUGIN MANUAL - Version 3.40   --------------- November 2017 ---
 -------------------------------------------------------------------------------
 
 CONTENT
@@ -170,24 +170,10 @@ AddKeyword(keyword, kwGroupID)
 3.2 FUNCTION ONSTATECHANGE
 -------------------------------------------------------------------------------
 
-This function is a hook which is called if an internal state changes (e.g. from
-HL_STANDARD to HL_KEYWORD if a keyword is found). It can be used to alter
-the new state or to manipulate syntax elements.
+As this function is more commonly used in language definitions, refer to README
+for its detailed description.
 
-OnStateChange(oldState, newState, token, kwGroupID)
-
-  Hook Event: Highlighting parser state change
-  Parameters: oldState:  old state
-              newState:  intended new state
-              token:     the current token which triggered the new state
-              kwGroupID: if newState is HL_KEYWORD, the parameter
-                         contains the keyword group ID
-  Returns:    Correct state to continue OR HL_REJECT
-
-Return HL_REJECT if the recognized token and state should be discarded; the 
-first character of token will be outputted and highlighted as oldState.
-
-Examples:
+Example for its usage in a plugin context:
 
 function OnStateChange(oldState, newState, token, kwgroup)
    if newState==HL_KEYWORD and kwgroup==5 then
@@ -201,16 +187,6 @@ keyword belongs to keyword group 5. If keyword group 5 is defined by a regex,
 this token will be recognized later as keyword even if the regular expression 
 does no longer match.
 
-function OnStateChange(oldState, newState, token)
-   if token=="]]" and oldState==HL_STRING and newState==HL_BLOCK_COMMENT_END then
-      return HL_STRING_END
-   end
-   return newState
-end
-
-This function resolves a Lua parsing issue with the "]]" close delimiter which
-ends both comments and strings.
-
 
 3.3 DECORATE FUNCTIONS
 -------------------------------------------------------------------------------

README_REGEX

@@ -392,4 +392,9 @@ Unicode characters in a keyword.
 Andre Simon
 andre.simon1@gmx.de
 http://www.andre-simon.de/
-http://wiki.andre-simon.de/
+
+sf.net project with SVN repository, download mirror, bug tracker, help forum:
+http://sourceforge.net/projects/syntaxhighlight/
+
+Github project with Git repository, bug tracker:
+https://github.com/andre-simon/highlight

src/core/syntaxreader.cpp

@@ -312,7 +312,7 @@ LoadResult SyntaxReader::load ( const string& langDefPath, const string& pluginR
             string escRegex;
             if (ls["Strings"]["Escape"].value()==Diluculum::Nil){
                 escRegex=REGEX_ESCSEQ;
-                ls["Escape"] = escRegex; //for --verbose output
+                ls["Strings[Escape]"] = escRegex; //for --verbose output
             } else {
                 escRegex=ls["Strings"]["Escape"].value().asString();
             }