Navigation Menu

Skip to content

Commit

Permalink
Added page titles for next/previous navigation
Browse files Browse the repository at this point in the history
  • Loading branch information
citizenmatt committed Jul 10, 2017
1 parent a136208 commit 1121aeb
Show file tree
Hide file tree
Showing 127 changed files with 127 additions and 253 deletions.
3 changes: 1 addition & 2 deletions Architecture/Overview.md
@@ -1,8 +1,7 @@
---
title: Architectural Overview
---

# Architectural Overview

In any given ReSharper plugin, you are likely to be interacting with many different subsystems of the ReSharper product. In this part of the guide, we are going to take a look at a bird's eye view of ReSharper.

At the top level, we can isolate the following aspects of ReSharper:
Expand Down
3 changes: 1 addition & 2 deletions Architecture/PSI.md
@@ -1,8 +1,7 @@
---
title: PSI
---

# PSI

The PSI is the Program Structure Index. It builds on the Platform and Project Model layers to create a complete syntactic and semantic view of a codebase. It provides a language agnostic framework for generating abstract syntax trees and semantic models. It also provides an implementation for approximately 20 languages, including C#, VB, XML, HTML, CSS, JavaScript and even regular expressions.

The building blocks of the PSI are:
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/BuildingPsiTree.md
@@ -1,8 +1,7 @@
---
title: Building the PSI Tree
---

# Building the PSI Tree

One of the main responsibilities of a custom language implementation is to build a PSI tree which represents the syntactic structure of the file, from class declarations, to keywords, to whitespace and comments. This tree is required for just about all other functionality in ReSharper - refactoring, inspections, formatting, etc. all require manipulating or walking a PSI tree.

> **NOTE** The PSI tree is also commonly referred to as an abstract syntax tree (AST), but it is more correct to call it a concrete syntax tree, or parse tree. An [abstract syntax tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree) represents the syntactic structure of a file without necessarily being tied to the physical representation of the syntax in a file. For example, parentheses used in an expression to denote grouping aren't needed in an AST as the grouping is implied by the tree structure itself. A [concrete syntax tree or parse tree](https://en.wikipedia.org/wiki/Parse_tree) represents the full syntactic structure of the file, including all the syntax within the file.
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/InjectedPsi.md
@@ -1,8 +1,7 @@
---
title: Injected PSI
---

# Injected PSI

ReSharper 7 introduces a new lightweight mechanism for ‘slicing’ a single physical code file into multiple PSI files corresponding to the languages that are used within that file. This mechanism is called _Injected PSI_. An example of an injected PSI file is a JSON file in a property of an HTML control in a JavaScript Metro app. Since the injected PSI mechanism is still being introduced, other pertinent areas (e.g., JS and CSS in HTML) currently do not use this mechanism.

## PSI Types
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Overview.md
@@ -1,8 +1,7 @@
---
title: Custom Languages
---

# Custom Languages

In this part of the guide we'll look at developing ReSharper support for a new language. We shall take a look at the following:

* Table of contents
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Lexing.md
@@ -1,8 +1,7 @@
---
title: Lexing
---

# Lexing

[Lexical analysis](https://en.wikipedia.org/wiki/Lexical_analysis) (or "lexing") is the process of converting a text buffer into a sequence of tokens which uniquely identify the building blocks of a custom language. For example, a lexer can convert the C# expression `age = 42;` into the following stream of tokens:

```
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Lexing/CachingLexers.md
@@ -1,8 +1,7 @@
---
title: Caching Lexers
---

# Caching Lexers

The root node in the PSI parse tree is `IFile`, but the implementation of this interface should also implement `IFileImpl`. This exposes properties and methods that are important for the implementation of `IFile`, one of which is `TokenBuffer`.

The `TokenBuffer` property is an optional cache of the tokens in a file. If not-`null`, it contains the start and end offset, lexer state and type of all of the tokens in the file. The constructor will take in an `ILexer`, and it will immediately scan the whole file and store the processed tokens.
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Lexing/CsLex.md
@@ -1,8 +1,7 @@
---
title: Using CsLex
---

# Using CsLex

The SDK ships with a tool called **CsLex**, which is a C# utility for building lexers. It takes in a specially formatted file and generates a C# class that is then compiled into the project. The generated lexer is very efficient, implemented as a set of lookup decision tables that implement the regular expression rules that describe how to match a token.

The version of CsLex in the SDK is based on the [version by Brad Merrill](http://www.cybercom.net/~zbrad/DotNet/Lex/Lex.htm), with [some modifications](#modifications-to-the-original-cslex), most notably to add support for Unicode text.
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Lexing/FilteringLexers.md
@@ -1,8 +1,7 @@
---
title: Filtering Lexers
---

# Filtering Lexers

Parsers are designed to recognise specific sequences of tokens, and if that sequence contains lots of whitespace or comments, or other syntactically and semantically insignificant tokens, then it makes recognising patterns much harder.

A filtering lexer will filter out whitespace, comments and other insignificant tokens, allowing the parser a "clean" sequence of tokens that make it easier to pattern match against.
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Lexing/ImplementingLexers.md
@@ -1,8 +1,7 @@
---
title: Implementing a lexer
---

# Implementing a lexer

ReSharper requires a custom language to create a lexer that implements (at least) the `ILexer` interface:

```csharp
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Lexing/LexerFactories.md
@@ -1,8 +1,7 @@
---
title: Lexer Factories
---

# Lexer Factories

ReSharper requires a custom language to implement `ILexerFactory`, which is exposed via `IProjectFileLanguageService.GetMixedLexerFactory` and the `LanguageService.GetPrimaryLexerFactory` abstract method.

The lexer factory is assigned to the `IFile` root node of the PSI tree, via the `IFileImpl` interface, which exposes implementation details rather than PSI tree information. This is made available so that various parts of ReSharper can easily create a fresh lexer on a new buffer.
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Lexing/LexerUtil.md
@@ -1,8 +1,7 @@
---
title: Lexer Utility Methods
---

# Lexer Utility Methods

The `LexerUtil` class provides a number of utility and extension functions for lexing and parsing. Of particular interest are the `LookaheadToken` methods, `GetTokenLength` and `GetCurrTokenText`:

```csharp
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Lexing/Tao.md
@@ -1,8 +1,7 @@
---
title: The Tao of Writing a Lexer
---

# The Tao of Writing a Lexer

Writing a lexer for a ReSharper custom language is as easy and as hard as writing any other kind of lexer, but the requirements of writing a lexer for IDE support are slightly different to the requirements for building a compiler or other batch processor. The other pages in this section give technical details on how to implement a lexer. This page is intended to provide insight into what needs to be implemented, and describe what is needed for IDE support.

## Tokens and node types
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Lexing/Testing.md
@@ -1,8 +1,7 @@
---
title: Testing Lexers
---

# Testing Lexers

The `LexerTestBase` class can be used to create tests for lexers. It derives from `BaseTestWithSingleProject`, so it creates an in-memory ReSharper instance with a solution loaded that contains a single project. The project will consist of the file or files named in the test methods.

The input file, that gets added to the in-memory project, should be an example file in the custom language. The `LexerTestBase` class will create an instance of the language's `ILexer`, with the default implementation using the file extension to get an instance of the custom language's `IProjectFileLanguageService`, and then calling `GetMixedLexerFactory`.
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Lexing/TextBuffers.md
@@ -1,8 +1,7 @@
---
title: Text Buffers
---

# Text Buffers

Lexers process a block of text to produce a sequence of tokens. ReSharper provides the `IBuffer` interface to abstract accessing this text block:

```csharp
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Lexing/UtilityClasses.md
@@ -1,8 +1,7 @@
---
title: Utility Classes
---

# Utility Classes

ReSharper includes a couple of classes that are useful when writing a lexer, namely `NodeTypeSet` and `LexerDictionary<T>`. The `NodeTypeSet` class the lexer can use to test if a `NodeType` instance is part of a particular set of nodes, such as keywords, symbols and so on. The `LexerDictionary` class is a dictionary of strings to `NodeType`, optimised for use in a lexer.

## NodeTypeSet
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/NodeTypes.md
@@ -1,8 +1,7 @@
---
title: Node Types
---

# Node Types

Each node in a given PSI tree implements `ITreeNode`, which has a property called `NodeType` that identifies the type of the node. The value is a singleton instance that derives from the abstract base class `NodeType`. Leaf elements in the tree have a node type that further derives from the `TokenNodeType` abstract base class, while interior tree elements derive from `CompositeNodeType`.

The `NodeType` class is the abstract base class for all node types:
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/NodeTypes/CompositeNodeTypes.md
@@ -1,8 +1,7 @@
---
title: Composite Node Types
---

# Composite Node Types

The interior nodes of the tree have a node type that typically derives from `CompositeNodeType`. ReSharper doesn't have any strict requirements for the node type of an interior tree node, unless the custom language implements tree node caching, in which case, the node type must implement the `ICompositeNodeType` interface:

```csharp
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/NodeTypes/CreatingNodeTypes.md
@@ -1,8 +1,7 @@
---
title: Creating Node Types
---

# Creating Node Types

A custom language requires many `NodeType` classes and instances to be created. After all, every distinct type of node in a PSI tree needs a distinct `NodeType` instance. While it is possible to create these classes and instances by hand, it is more manageable to use tooling to generate the code.

The SDK provides the `TokenGenerator` tool to convert an XML file listing tokens and keywords into `TokenNodeType` classes and static singleton instances. The `PsiGen` tool is used to create a parser from a `.psi` file, but it also creates `ITreeNode` classes and `CompositeNodeType` classes and instances.
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/NodeTypes/TokenNodeTypes.md
@@ -1,8 +1,7 @@
---
title: Token node types
---

# Token node types

Each leaf element in a PSI tree has a node type that is a singleton instance of a class that derives from the `TokenNodeType` base class, and implements the `ITokenNodeType` interface. This singleton instance is the token produced by the lexer.

```csharp
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Parsing.md
@@ -1,8 +1,7 @@
---
title: Parsing
---

# Parsing

Parsing or ["syntactic analysis"](https://en.wikipedia.org/wiki/Parsing) is the process of taking an input stream of tokens and producing a [parse tree](https://en.wikipedia.org/wiki/Parse_tree). This parse tree is used by ReSharper to build a semantic model, with rich information about types, and also used to navigate, analyse and manipulate the code. For example, the unit testing functionality is implemented by walking the parse tree of a file and looking for certain constructs, such as class and method decorations with specific attributes. Similarly, refactoring is the act of rewriting the parse tree, which has the effect of rewriting the underlying text file.

A parse tree built by a ReSharper language is sometimes known as a **PSI tree**, named for the Program Structure Index subsystem that is responsible for building and maintaining such trees. It is also commonly referred to as an abstract syntax tree, but it is more correct to call it a concrete parse tree. A concrete parse tree is an accurate model of the syntax in a file, while an abstract syntax tree can be simplified. See [this page for a good description of the differences](http://eli.thegreenplace.net/2009/02/16/abstract-vs-concrete-syntax-trees).
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Parsing/Parsing/Testing.md
@@ -1,8 +1,7 @@
---
title: Testing Parsers
---

# Testing Parsers

The `ParserTestBase<TLanguage>` can be used to create tests for a parser of a specific language. It derives from `BaseTestWithTextControl`, which means it will create an in-memory ReSharper instance, with a solution and a project loaded. The test will load a specific text file into an in-memory text control, parse it, retrieve any `IFile` instances that match `TLanguage`, and compare the results against a `.gold` file.

The `.gold` file will be the `IFile` and children, written as an indented text file. Each line will be another node in the tree, with the text being the result of calling `ToString()` on the node instance.
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Registration.md
@@ -1,8 +1,7 @@
---
title: Registering a Custom Language
---

# Registering a Custom Language

Before a custom language can be recognised and parsed by ReSharper, it must first be registered. There are several classes that need to be registered:

* [Project Model file type](Registration/ProjectFileType.md)
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Registration/PerLanguageComponents.md
@@ -1,8 +1,7 @@
---
title: Per-Language Components
---

# Per-Language Components

ReSharper is designed to work with multiple languages and file types. It provides features that are cross language, but that require language specific implementations. For example, formatting is a feature that works across all languages, but each language must provide its own formatting rules.

If these per-language implementations were implemented as shell or solution components, then any consumers would need to retrieve all implementations and filter them appropriately, either by a method on the implemented interface, or some other metadata, such as attributes.
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Registration/ProjectFileLanguageService.md
@@ -1,8 +1,7 @@
---
title: Project File Language Service
---

# Project File Language Service

If a custom language defines a `ProjectFileType`, it must also associate that file type with a PSI language type, which ReSharper can use to build a PSI tree of the file's contents. The `IProjectFileLanguageService` interface provides this functionality, and can be considered as a bridge between the Project Model and the PSI.

The project file language service is implemented as a [per-file type component](PerLanguageComponents.md#file-type-specific-components). Given a `ProjectFileType`, it is possible to retrieve that file type's project file language service, which exposes the custom language's `PsiLanguageType`. This can be used as the key to retrieve all [per-language components](PerLanguageComponents.md#psi-language-specific-components), and get access to the custom language's lexer and parser, to build a PSI tree.
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Registration/ProjectFileType.md
@@ -1,8 +1,7 @@
---
title: Registering a Project File Type
---

# Registering a Project File Type

If a custom language is to be the primary language of a certain file, it should register a file type with the Project Model.

> **NOTE** Not all custom languages require a file type. For example, regular expressions are handled as an injected PSI language. The string literal that describes the regular expression in, say, a C# file, becomes a holder for a new PSI tree that describes the content of the regular expression. It doesn't make sense for a file to be a "regular expression file", so there is no project file type for regular expressions. Instead, the host language, such as C#, VB or JavaScript explicitly creates the injected PSI language.
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Registration/PsiLanguageDefinition.md
@@ -1,8 +1,7 @@
---
title: Registering a PSI Language
---

# Registering a PSI Language

Once a custom language's [file type has been registered with the Project Model](ProjectFileType.md), the language definition needs to be registered with the PSI. If it's not registered with the PSI, ReSharper will not be able to build a PSI tree, and in fact will not consider it registered with the Project Model, either.

The PSI maintains a hierarchy of supported languages, in a similar manner to how the Project Model maintains a hierarchy of supported file types. The base class is `PsiLanguageType`, and there are again, two directly derived classes - `KnownLanguage` and `UnknownLanguage`. The first is the base class for all known languages, and can be used to mean any and all known PSI languages. The second, `UnknownLanguage`, is a sealed class, and represents a language that the PSI does not support.
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Registration/PsiLanguageService.md
@@ -1,8 +1,7 @@
---
title: PSI Language Service
---

# PSI Language Service

Each custom language must register a PSI language service. This is a [per-language component](PerLanguageComponents.md#psi-language-specific-components) that derives from the `LanguageService` abstract class. It provides the entry points for lexing, parsing and building the PSI tree of a custom language.

As it is the main entry point for functionality for a custom language, it is a rather large base class. Some of the type members have implementations, while others are virtual or abstract. The class looks like this (with implementations elided):
Expand Down
3 changes: 1 addition & 2 deletions CustomLanguages/Registration/Testing.md
@@ -1,8 +1,7 @@
---
title: Testing
---

# Testing

Testing language registration is fairly quick and easy. It's mostly checking that the Component Model has loaded the types correctly. This means the tests need to derive from `BaseTest`, so that the environment and Component Model has been initialised, but a solution hasn't been created.

Testing the `IProjectFileLanguageService` and PSI `LanguageService` implementations is not so straightforward. It's better to test them indirectly, by testing other functionality such as lexing, parsing, and so on.
Expand Down
3 changes: 1 addition & 2 deletions Extensions/CompiledExtensions.md
@@ -1,10 +1,9 @@
---
title: Compiled Extensions (Plugins)
redirect_from:
- /Intro/CompiledExtensions.html
---

# Compiled Extensions (Plugins)

ReSharper supports two types of extensions - compiled and [declarative](DeclarativeExtensions.md). A declarative extension is an extension that contains External Annotations, or features that can be stored in settings files, such as Live Templates or Structural Search and Replace patterns.

A compiled extension, known more simply as a plugin, is a .net assembly that is loaded by the ReSharper Platform, and has full access to the API. ReSharper doesn't have a "plugin API", preferring to make the whole API public and allow plugins to make full use of the platform. This gives the plugin author great power - plugins have full access to the same API as ReSharper's native functionality.
Expand Down
3 changes: 1 addition & 2 deletions Extensions/DeclarativeExtensions.md
@@ -1,10 +1,9 @@
---
title: Declarative Extensions
redirect_from:
- /Intro/DeclarativeExtensions.html
---

# Declarative Extensions

The ReSharper Platform supports two types of extensions - [compiled extensions (aka plugins)](CompiledExtensions.md), and declarative extensions. Plugins are code based, .dll assemblies that are loaded into the process and have full access to the API.

Declarative extensions are either settings files or External Annotation files. They do not contain code, but can be very versatile.
Expand Down
3 changes: 1 addition & 2 deletions Extensions/Deployment/LocalInstallation.md
@@ -1,8 +1,7 @@
---
title: Local Installation
---

# Local Installation

In order to manually test an extension, whether it's a [plugin](/Extensions/CompiledExtensions.md) or a [declarative extension](/Extensions/DeclarativeExtensions.md), it must first be installed into the ReSharper Platform. Once installed, existing files can generally be updated by copying a new version without re-runnning the installatione. This is a similar pattern to Visual Studio's VSIX development model.

> **NOTE** Previous versions of ReSharper allowed plugins to be run via a command line parameter (`/ReSharper.plugin`). This is no longer supported, as the ReSharper Platform now requires all extensions (including Products such as dotTrace and dotPeek) to be statically registered. Visual Studio integration plays a large role in this decision. Dynamically registering components with Visual Studio introduced issues that are not evident when statically registering (e.g. dynamically added menu items not maintaining assigned keyboard shortcuts). So, while the Component Model allows for dynamic live loading of assemblies and types, the ReSharper Platform requires static registration.
Expand Down
3 changes: 1 addition & 2 deletions Extensions/Deployment/LocalInstallation/CopyOnBuild.md
@@ -1,8 +1,7 @@
---
title: Copy Plugin on Build
---

# Copy Plugin on Build

When developing a plugin, it must [first be installed](/Extensions/Plugins/ProjectSetup/InitialInstallation.md) in order to manually test and debug it. While it is possible to repackage and update through the extension manager on each compile, this is an overhead that is not necessary for day-to-day development. Instead, the files in the installation folder can be overwritten, and ReSharper will use the new version when the host (e.g. Visual Studio) is restarted.

This method works as long as there are no changes that require re-registering the extension with ReSharper or Visual Studio. In this case, the extension should be repackaged and updated through the Extension Manager, using a custom source to install from a local folder.
Expand Down

0 comments on commit 1121aeb

Please sign in to comment.