Added remaining documentation.

source/dev/internals/intellij/index.rst

@@ -17,5 +17,13 @@ a lot of code they live in the same project.
    vfs
    configurations
    languages
+   lexing-and-parsing
    projects
    troubleshooting
+
+See also
+========
+- Bjorn Tipling — `How to make an IntelliJ IDEA plugin in less than 30 minutes <http://bjorn.tipling.com/how-to-make-an-intellij-idea-plugin-in-30-minutes>`_
+- JetBrains — `Custom Language Support Tutorial <http://www.jetbrains.org/intellij/sdk/docs/tutorials/custom_language_support_tutorial.html>`_
+- JetBrains — `Writing Tests For Plugins <http://www.jetbrains.org/intellij/sdk/docs/tutorials/writing_tests_for_plugins.html>`_
+- Terence Parr — `IntelliJ Plugin Development Notes <https://github.com/antlr/jetbrains/blob/master/doc/plugin-dev-notes.md>`_

source/dev/internals/intellij/languages.md

@@ -23,9 +23,26 @@ Currently it only looks in the plugin's resources, but future implementations
 may search Maven Central or [artifacts.metaborg.org][2].
 
 
+## Languages in IntelliJ
+IntelliJ expects a unique implementation of the abstract `Language` class for
+each language. It considers two `Language` to be the same if they are instances
+of the same class. Therefore, to represent multiple Spoofax languages, we need
+to create our own `Language` class implementations dynamically.
+
+The built-in `java.lang.reflect.Proxy` class is not sufficient. First of all
+I'm not sure whether different proxies are different classes, but most
+importantly, the `Proxy` class only supports implementing interfaces. We need
+to implement the abstract class `Language`, so this won't work.
+
+There are several third-party libraries that allow you to create classes at
+runtime. A very promising one is _Javassist_, which I've used before when
+trying to build a profiler for Stratego. [Here is some information][5].
+
+
 
 
 [1]: https://github.com/metaborg/spoofax-intellij/blob/develop/org.metaborg.intellij/src/main/java/org/metaborg/intellij/discovery/ILanguageSource.java
 [2]: http://artifacts.metaborg.org/
 [3]: https://github.com/metaborg/spoofax-intellij/blob/develop/org.metaborg.intellij/src/main/java/org/metaborg/intellij/languages/ILanguageManager.java
 [4]: https://github.com/metaborg/spoofax-intellij/blob/develop/org.metaborg.intellij/src/main/java/org/metaborg/intellij/idea/languages/IIdeaLanguageManager.java
+[5]: http://stackoverflow.com/a/3292208/146622

source/dev/internals/intellij/lexing-and-parsing.md

@@ -0,0 +1,19 @@
+# Lexing and Parsing
+IntelliJ relies on a lexer and parser.
+
+## Lexer
+The lexer shall extend the `com.intellij.lexer.Lexer` class (usually through
+the `com.intellij.lexer.LexerBase` class). IntelliJ will call its `start`
+method, providing it with a character buffer and range that need to be lexed,
+and the lexer state at the start of the region. Then the lexer will lex forward
+on each call to `advance`, providing information about the current token's type
+and offset, and the lexer state at that point.
+
+Note that the lexer may not know which file is being lexed, and it needs to be
+able to start at any arbitrary point in the input. Also, it needs to return
+_every_ token, including layout, even invalid ones.
+
+## Parser
+The parser is fed the lexer tokens, and turns them into AST nodes. A single AST
+node can consist of multiple tokens. Again, the parser must be able to parse
+from within a file.

source/dev/internals/intellij/logging.md

@@ -25,13 +25,44 @@ current stack trace, and return it. You can then throw the exception.
 The stack trace is cleaned up such that the `LoggerUtils` class doesn't appear
 in the trace.
 
+## IntelliJ versus Spoofax
+Both Spoofax Core (and related Metaborg libraries) and IntelliJ IDEA use _slf4j_
+for logging. The Spoofax for IntelliJ plugin also uses _slf4j_ for logging, so
+it depends on `org.slf4j:slf4j-api:1.7.10`.
 
+Normally _slf4j_ looks for a `StaticLoggerBinder` implementation and bind to
+that for logging. IntelliJ IDEA provides a `StaticLoggerBinder` by default
+(from `org.slf4j:slf4j-log4j12:1.7.10`), so we'd like to bind to that. This is
+_slf4j_'s default behavior, so we don't have to add any dependencies for this.
+However, if we try that, we get an exception:
 
+> java.lang.LinkageError: loader constraint violation: when resolving method "org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()Lorg/slf4j/ILoggerFactory;" the class loader (instance of com/intellij/ide/plugins/cl/PluginClassLoader) of the current class, org/slf4j/LoggerFactory, and the class loader (instance of com/intellij/util/lang/UrlClassLoader) for the method's defining class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type org/slf4j/ILoggerFactory used in the signature
+
+Apparently there are two class loaders used to resolve the `StaticLoggerBinder`,
+and they interfere.
+
+We can use our own logger binder (depend on `org.slf4j:slf4j-simple:1.7.10`),
+but this has some downsides: we have to configure the logger ourselves, and
+can't choose the `StaticLoggerBinder` that _slf4j_ should bind to. This causes
+it to warn us that there are now two `StaticLoggerBinder` classes and it doesn't
+know which one to bind to. It will pick one at random:
+
+    SLF4J: Class path contains multiple SLF4J bindings.
+    SLF4J: Found binding in [jar:file:/home/daniel/.IdeaIC14/system/plugins-sandbox/plugins/spoofax-intellij/lib/slf4j-simple-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
+    SLF4J: Found binding in [jar:file:/home/daniel/apps/1507-IntelliJ/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
+    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
+    SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
+
+We don't have a solution for this yet.
+
+```eval_rst
+:download:`Full stack trace of the error <logging_stacktrace.txt>`
+```
 
 
 
 [1]: https://github.com/metaborg/mb-exec/blob/master/org.metaborg.util/src/main/java/org/metaborg/util/log/ILogger.java
-[2]: bindings.md#injecting-the-logger
+[2]: bindings.md
 [3]: https://github.com/JetBrains/intellij-community/blob/3240cd7a32d7aa5e44872527c58eee3f0f3786ce/platform/util/src/com/intellij/openapi/diagnostic/Logger.java
 [4]: https://github.com/metaborg/spoofax-intellij/blob/develop/org.metaborg.intellij/src/main/java/org/slf4j/impl/IntelliJLoggerAdapter.java
 [5]: https://github.com/metaborg/spoofax-intellij/blob/develop/org.metaborg.intellij/src/main/java/org/metaborg/intellij/logging/PrintStreamLogger.java