This is a antlr-based javascript parser for the Yarn syntax, intended for use in the Jacquard editor.
Install jacquard-yarnparser
via your favourite javascript package tool.
API Docs are here
(the implemented JS preprocessor is here)
Because of the difficulties of parsing indented languages in ANTLR we pre-process the text file before passing it into the ANTLR parser. We add invisible shortcut start and shortcut end characters. These characters are 0x1D
and 0x1E
Newlines can be either \n
or \r\n
to be aware of that (you can convert into \n
if you wish)
It's probably a good idea to use a stack as the way of storing the whitspace encountered and if this was part of an option (so you can decide if you need to emit a "close option" and to figure out if this is a deeper level of nesting or not)
- Save our whitespace encountered as the empty string and not to emit a shortcut end character
- After a newline, check the whitespace between the newline and any other character
- If:
- we are in a comment and the line doesn't contain
*/
- then
- Skip to 3 below - If:
- we are in a comment and the line contains
*/
- then
- take everything after the
*/
and and push it back as a new line. - the current line should only be the remainder of the comment.
- Skip to 3 below - If:
- we aren't in a comment and the next characters are
//
- then
- Skip to 3 below - If:
- we aren't in a comment and the next characters are
/*
- then
- we are in a comment
- Skip to 3 below - If:
- we need to emit a close option - and
- the whitespace found is the same as the current whitespace encountered or shorter - then
- Insert a shortcut end character (
0x1E
) before the newline - pop the current whitespace encountered & option off the stack
- start reading from the newline - If:
- the whitespace found is the same or longer than the current whitespace encountered - and
- the next characters are
->
- then
- Insert a single space and a shortcut start character (
0x1D
) immediatley following the->
- Mark off that we are now in an option and the amount of whitespace we have onto the stack.
- Output the start of the line (whitespace and
->
), a space (0x1D
) and the rest of that line and return to 2 above
- we are in a comment and the line doesn't contain
- Output the line and return to 2 above
Jacquard - Yarn Parser requires the following dependencies be pre-installed:
- NodeJS (https://nodejs.org).
- Yarn package manager (https://yarnpkg.com).
First, you need to make sure that you have nodeJs and Yarn installed (see the dependencies section above and install the relevant version for your platform).
Currenlty we're using node v8.9.4
and yarn v1.5.1
.
If you want to modify the ANTLR definitions, you should setup ANTLR for your environment.
Install the Java SE Development Kit (either version 7 or 8) - check java.sun.com for downloads
Ensure that javac
and java
executables are in your path:
- Open a command prompt
- try the command
java -version
(you should be greeted with a version number) - try the command
javac -version
(again, a version should be output)
If either javac or java aren't present check your install (these should work by default on the command line on mac os x)
Next - we need to get ANTLR itself and put it in your classpath. Check out the instructions here
Unfortunatley aliases aren't accessible from node scripts (if you know how to fix this - let us know!), so you'll need to create shell scripts somewhere on your path
Some suggested scripts are here
Install the Java SE Development Kit (either version 7 or 8) - check java.sun.com for downloads
Ensure that javac.exe
and java.exe
executables are in your path:
- Open a command prompt
- try the command
java -version
(you should be greeted with a version number) - try the command
javac -version
(again, a version should be output)
If either javac or java aren't present check the following:
- check your environment variables and ensure your System Variable JAVA_HOME is set to the root of the java SDK (usually something like
C:\Program Files\Java\jdk1.8.0_131
) - check your system path and user path for
%JAVA_HOME%\bin
, if it's not present - add it to the end of either your user or system path. - if you had to add either of the above, sign out and sign back in.
Next - we need to get ANTLR itself and put it in your classpath. Check out the instructions here
To preprocess all the files in the yarn_files
directory yarn preprocessTestFiles
. To preprocess a specific file, or another file, take a look at yarn parse --help
To see the AST for a given file run yarn antlr-gui preprocessed_yarn_files\<filename>
To update the javascript antlr files run yarn antlr-compile
. You'll need to do this if you modify the .g4
files in the src/antlr/
directory
Using your command line, navigate to the directory you've cloned this repo into.
- Execute a
yarn install
. This will download all the necessary libraries you need. - To run the parser on the command line execute
yarn parse
. This will list help.
If you wish to rebuild the documentation, you'll need to have documentation.js installed, which you can do via npm install -g documentation
- then you can run yarn serveDoc
to serve the docs locally or yarn buildDoc
to build the docs directory