Skip to content
AntlrVSIX is an extension for Visual Studio 2019 to support Antlr v4 grammars. It is implemented using LSP. This extension focuses on a complete IDE experience, supporting refactoring grammars in order to make them cleaner, more readable, and more efficient.
C# Java ANTLR C TypeScript Shell
Branch: master
Clone or download

Latest commit

Fetching latest commit…
Cannot retrieve the latest commit at this time.

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
Algorithms
AntlrFAProject
Client
LanguageServer
Logger
LspHelpers
Options
Server
Symtab
VSCode.LspAntlr
Workspaces
corpus-for-codebuff
doc
test
.gitignore
.travis.yml
AntlrVSIX.sln
Gemfile
README.md
_config.yml
appveyor.yml
clean.sh
license.txt

README.md

AntlrVSIX

AntlrVSIX is an extension for Visual Studio 2019 to support editing, analysis, refactoring, and conversion of Antlr v4 grammars. It is implemented using Microsoft's Language Server Protocol (LSP) client and server APIs, Antlr, Antlr4BuildTasks, and a number of other tools. Most of the extension is implemented in C#. There is a client for VS Code, written in Typescript, but I am deferring further development of the client until the server is more or less complete. Supported are colorized tagging, hover, go to def, find all refs, replace, command completion, reformat, and go to visitor/listener. But, there is so much more to Antlrvsix than these basic LSP commands.

Analysis

(This is not implemented yet.)

Antlrvsix will make recommendations on what you should change for readability and performance. For performance, link to a special version of the Antlr library to perform analysis.

Refactoring

Antlrvsix provides a number of transformations that can help to make grammars cleaner (reformatting), more readable (reducing the length of the RHS of a rule), and more efficient (reducing the number of non-terminals) for Antlr.

Some of these refactorings are very specific for Antlr due to the way the parser works, e.g., converting a prioritized chain of productions recognizing an arithmetic expression to a recursive alternate form. The refactorings implemented are:

  • Replace string literals in parser with lexer symbols.
  • Remove useless parser rules.
  • Move start rule to top.
  • Reorder parser rules alphabetically.
  • Reorder parser rules DFS from start rule.
  • Reorder parser rules BFS from start rule.
  • Split combined grammars.
  • Combine splitted grammars.
  • Replace direct left recursion with right recursion.
  • Replace indirect left recursion with right recursion.
  • Replace parser rule symbols that conflict with Antlr keywords.
  • Add lexer rules for string literals in parser.
  • Sort lexer modes alphabetically.
  • Replace direct left/right recursion with Kleene operator.
  • Unfold.
  • Fold.
  • Remove useless parentheses.

Conversion

  • Bison import.

The source code for the extension is open source, free of charge, and free of ads. For the latest developments on the extension, check out my blog.

Installation of Prerequisites

There are no prerequsites for Antlrvsix other than you use Visual Studio 2019. I recommend that you use version 16.6 since it has the latest bug fixes.

If you are building and running an Antlr application--as opposed to simply opening an Antlr grammar file to view--then you will want to set up the build environment. There is support for Antlr C# programs using Antlr4BuildTasks. (Note, you can also develop Antlr programs for other languages that VS2019 supports.) If you plan to build Antlrvsix, then you must set up the build environment as described in the instructions of Antlr4BuildTasks.

Installation

You can install the extension in one of three ways:

  • Download the .vsix from a web browser, then executing the .vsix file from a Windows Explorer.
  • Download and install the extension within Visual Studio 2019 via "Extensions | Manage Extensions", search for Antlrvsix, and "install".
  • Build a copy of the .vsix from the sources and install. NB: I sometimes check in code into the repository that does not compile. This is because I am making huge changes the the code. Grab the source for a released version to be safe.

Documentation

For information on how to use AntlrVSIX, see the User Guide.

Building Antlrvsix:

Build status

  • From a clean "git" root directory, open a "Developer command prompt", and execute:
    • msbuild /t:restore
    • msbuild

The extension is at ./Client/bin/Debug/AntlrVSIX.vsix after building successfully.

  • If you want to make modifications for yourself, you should reset your Experimental Hive for Visual Studio. To do that, Microsoft recommends using CreateExpInstance.exe. Unfortunately, I've found CreateExpInstance doesn't always work because it copies from previous hives stored under the AppData directory. It is often easier to just recursively delete all directories ...\AppData\Local\Microsoft\VisualStudio\16.0_*. I've included a Bash script "clean.sh" in the Antlrvsix source to clean out the build files so one can build from scratch.

Planned for v8.0 (July 2020):

  • Add in Piggy, a transformational system for trees using a specialized tree rewrite language, into Antlrvsix.
  • Start rewriting existing transforms and analyses into Piggy.
  • Add left factoring and inverse.
  • Add in empty string alternative hoist transforms.

Planned for v7.4:

Release notes for v7.3 (31 May 2020):

Release notes for v7.2 (6-May-2020):

Release notes for v7.0 (4-May-2020):

  • Added transforms for the elimination of left recursion.
  • Added transform to convert left or right recursion to Kleene form.
  • Expand tagging of all parts of grammar file.
  • Changed colorizing text to use Visual Studio character classes, with a mapping from Antlr classes to VS classes in the options box.

Release notes for v6.0.1 (13-Apr-2020):

  • Fix colorization after editing.

Release notes for v6.0 (11-Apr-2020):

  • Added selection of colors for grammar symbols in options. (I am now using Dark theme in Visual Studio 2019.)
  • Changed options to use Newtonsoft.Json (breaking change--remove ~/.antlrvsixrc).
  • Added refactoring to sort mode sections in lexer grammars.
  • Added code to abbreviate action blocks in hover tool tips.
  • Added code to allow multiple definitions of a symbol. This can happen for lexer symbols defined in a tokens section and as a lexer rule.
  • Bug fixes.

Release notes for v5.8 (29-Mar-2020):

  • Added in rename on Antlrvsix tool menu because Microsoft broke LSP rename!

Release notes for v5.7 (29-Mar-2020):

  • Disabled completion since this is crashing the server. An option to enable was added so it can be turned on when fixed.

Release notes for v5.6 (28-Mar-2020):

  • Import of Bison/Yacc grammars. This will go well beyond anything that was implemented before (e.g., Bison to ANTLR translator by Parr and Cramer 2006 for Antlr3 conversions), but will stop far short of what I would like. It will rewrite terminals and non-terminals to suitable Antlr symbols, folding of string literals immediately as lexer rules, and declare terminals as tokens in the lexer grammar. Additional fixes may be available if I get more transformations working. But, there is so much I can't do at this point until more infrastructure is set up.
  • Bug fixes.

Release notes for v5.5 (19-Mar-2020):

  • Added in more refactorings: a) Sort rules alphabetically, DFS or BFS traversals. b) Separate/combine grammars.
  • Bug fixes.

Release notes for v5.4 (11-Mar-2020):

  • Added in a few refactoring transformations (remove useless parser productions, convert parser string literals to lexer token symbols, move start rule to top of grammar). Note: Lexer rules are prioritized, so transformations on these types may not be totally correct at the moment.
  • Fixed goto visitor/listener.
  • Added stability fixes.

Release notes for v5.3 (5-Mar-2020):

  • Fixed "Find references" of grammar symbols when opening only lexer grammar file.
  • Fixed Format Document.
  • Fixed Go To Listener/Visitor. Caveat: only works for C# and you must save solution before using.
  • Templates updated, Antlr4BuildTasks 2.2.

Release notes for v5.2 (16-Feb-2020):

  • Re-added About Box, Options Box, and next/previous grammar symbol.
  • Options are now contained in ~/.antlrvsixrc, a JSON file.

Release notes for v5.1 (7-Feb-2020):

  • Re-added colorized tagging of grammar. When I switch to the LSP implementation, this functionality was lost because it's not directly supported by the LSP Client/Server API that Microsoft provides. Microsoft says to implement this functionality using TextMate files, but that is not how it should be done--it duplicates the purpose of the LSP server, and it's hard to get right. Instead, it's implemented with a Visual Studio ITagger<> and a custom message to the LSP server.

Release notes for v5.0 (25-Jan-2020):

  • Restructuring the code as a Language Server Protocol client/server implementation with extensions for VS 2019 (IDE) and VS Code.

  • Templates for C# and C++. Note, Antlr 4.8 currently does not have a C++ pre-built binary release for Windows. You will need to build the runtime and update the generated .vcxproj file with path information. The template will generate a project that is expecting Debug Static and x64 target.

  • These are the LSP features currently implemented. Note, Midrosoft.VisualStudio.LanguageServer.Protocol version 16.4.30 does not implement LSP version 3.14, rather something around version 3.6. What is missing is color tagging. I'm not sure how to deal with this other than pitch the inferior API, then use OmniSharp's API, or as usual write everything myself.

  • Computing the completion symbols is not complete. It only gives token types for completion, not the actual symbols that could be inserted.

Message Support
General
initialize yes (Dec 10, '19)
initiialized yes (Dec 10, '19)
shutdown yes (Dec 10, '19)
exit yes (Dec 10, '19)
$/cancelRequest no
---- ----
Window
window/showMessage no
window/showMessageRequest no
window/logMessage no
---- ----
Telemetry
telemetry/event no
---- ----
Client
client/registerCapability no
client/unregisterCapability no
---- ----
Workspace
workspace/workspaceFolders no
workspace/didChangeWorkspaceFolders no
workspace/didChangeConfiguration no
workspace/configuration no
workspace/didChangeWatchedFiles no
workspace/symbol no
workspace/executeCommand no
workspace/applyEdit no
---- ----
Text Synchronization
textDocument/didOpen yes (Dec 10, '19)
textDocument/didChange yes (Dec 15, '19)
textDocument/willSave yes (Dec 18, '19)
textDocument/willSaveWaitUntil yes (Dec 18, '19)
textDocument/didSave yes (Dec 18, '19)
textDocument/didClose yes (Dec 18, '19)
---- ----
Diagnostics
textDocument/publishDiagnostics yes (Dec 10, '19)
---- ----
Language Features
textDocument/completion no
completionItem/resolve no
textDocument/hover yes (Dec 10, '19)
textDocument/signatureHelp no
textDocument/declaration unavailable in API
textDocument/definition yes (Dec 11, '19)
textDocument/typeDefinition yes (same as "definition")
textDocument/implementation yes (same as "definition")
textDocument/references yes (Dec 11, '19)
textDocument/documentHighlight yes (Dec 11, '19)
textDocument/documentSymbol yes (Dec 10, '19)
textDocument/codeAction no
textDocument/codeLens no
codeLens/resolve no
textDocument/documentLink no
documentLink/resolve no
textDocument/documentColor unavailable in API
textDocument/colorPresentation unavailable in API
textDocument/formatting yes (Dec 17, '19)
textDocument/rangeFormatting no
textDocument/onTypeFormatting no
textDocument/rename yes (Dec 18, '19)
textDocument/prepareRename unavailable in API
textDocument/foldingRange no

Release notes for v4.0.5 (12-Nov-2019):

  • Fixing #26.

Release notes for v4.0.4 (12-Nov-2019):

  • Fixing solution loading crash.
  • Removing parse tree print code from Python--it was used for debugging and should not have been there.
  • Stubbing out Python and Rust targets because they could interfere with (currently) much better extensions. Will add them back once they are up to snuff.

A shout out to all those folks using this extension and have opted-in to the automatic reporting!!

Release notes for v4.0.3 (11-Nov-2019):

  • Restructuring the code for options.
  • Adding in reporting of caught exceptions to server.
  • Some changes for the stability for Java soruce.

Release notes for v4.0.2 (1-Nov-2019):

  • Updated symbol table with new classes to represent files, directories, and search paths. This will help support for better modeling of Java's ClassPath, Antlr's imports, etc., so that scopes can be cleared out quickly and easily when the sources have changed.
  • Fixing issues in stability with Antlr and Java files.
  • Fixing https://github.com/kaby76/AntlrVSIX/issues/23
  • Fixing tool tips and highlighting for Java.
  • Updated symbol table to allow ambigous code editing.
  • Changing About and Options boxes to non-modal.

Release notes for v4.0.1 (28-Oct-2019):

  • Fixing stability issues with Antlr and Java files.

Release notes for v4.0 (26-Oct-2019):

  • Major changes to architecture, focusing on separation of GUI from a backend that works like Language Server Protocol (LSP).
  • In the GUI, tagging was improved in quality and speed.
  • In the backend, a symbol table was added to represent all symbols in a project and solution.
  • For Antlr grammars, imports and tokenVocab are now used to determine the scope of the symbol table.
  • For Antlr grammars, Intellisense pop-ups now give the rule definition and location of the file defining the symbol.
  • No specific improvements were made for Python, Rust, and Java support, in favor of focusing on the improved overall design and implementation of the extension.

Release notes for v3.0 (28-Aug-2019):

  • Supports Java, Python, Rust, Antlr in various stages. Description of languages abstracted into a "Grammar Description".

Release notes for v2.0.6 (13-Aug-2019):

  • The extension will support VS 2019 and VS 2017.

  • A menu for the extension will be added to a submenu under Extensions. The functionality provided will duplicate that in context menus.

  • The source code build files will be updated and migrated to the most recent version of .csproj format that is compatible with VS extensions. Unfortunately, updating to the latest (version 16) is not possible.

  • With my Antlr4BuildTasks NuGet package, .g4 files can be automatically compiled to .cs input within VS 2019 without having to manually run the Antlr4 Java tool on the command line. Building of the extension itself will be upgraded to use the Antlr4BuildTasks package.

  • Listener and Visitor classes are generated for a grammar with a right-click menu operation. For Listeners, there are two methods associated with a nonterminal. Depress the control key to select the Exit method, otherwise it will select the Enter method.

  • An options menu is provided to turn on incremental parsing. By default, incremental

  • parsing is off because it is very slow.

Release notes for v1.2.4 (11-Mar-2017):

  • The extension is now both VS 2017 and 2015 compatible.

  • The results windows of Antlr Find All References is now "Antlr Find Results".

Release notes for v1.2.3 (7-Mar-2017):

  • Color selection through VS Options/Environment/Fonts and Colors. Look for "Antlr ..." named items.

  • Bug fixes with Context Menu entries for AntlrVSIX. AntlrVSIX commands are now only visible when cursor positioned at an Antlr symbol in the grammar. This fixes the segv's when selecting AntlrVSIX commands in non-Antlr files.

Release notes for v1.2.2 (10-Feb-2017):

Release notes for v1.2 (31-Jan-2017):

Release notes for v1.1.1 (18-Jan-2017):

Release notes for v1.1 (17-Jan-2017):

Release notes for v1.0 (17-Jan-2017):

Any questions, email me at ken.domino gmail.com

You can’t perform that action at this time.