Skip to content

Commit

Permalink
Improve documentation and polish several details
Browse files Browse the repository at this point in the history 
- Simplify `github:` expansino
- New "Developer guide" section with a comprehensive tutorial for
  writing a syntactic linters and semantic rewrite.
- Upgrade to scalafmt v1.5.1 and use javadoc formatting for docstrings
- Docstrings for SymbolInfo, SType and STree (still needs more work)
  • Loading branch information
@olafurpg
olafurpg committed Sep 7, 2018
1 parent 6584af2 commit 387fd27
Show file tree
Hide file tree
Showing 58 changed files with 2,078 additions and 958 deletions.
2 changes: 2 additions & 0 deletions .scalafmt.conf
View file
@@ -1,4 +1,6 @@
project.git = true
docstrings = Javadoc
optIn.blankLineBeforeDocstring = true
assumeStandardLibraryStripMargin = true
align = none
onTestFailure = "To fix this, run `./scalafmt` from the project base directory"
Expand Down
95 changes: 95 additions & 0 deletions docs/developers/api.md
View file
@@ -0,0 +1,95 @@
---
id: api
title: API Documentation
---

The Scalafix public API documentation is composed of several packages.

## Scalafix v1

Latest Scaladoc:
[v@VERSION@](https://static.javadoc.io/ch.epfl.scala/scalafix-core_2.12/@VERSION@/scalafix/v1/index.html)

The Scalafix v1 API is available through `import scalafix.v1._`. Key data
structures include:

- `Patch`: to describe source code rewrites such as removing or replacing tokens
and trees.
- `Diagnostic`: a linter error message that can be reported at a source file
location.
- `SyntacticRule`: super class for all syntactic rules.
- `SyntacticDocument`: context about a single source file containing syntactic
information such as trees/tokens.
- `SemanticRule`: super class for all semantic rules.
- `SemanticDocument`: context about a single source file containing syntactic
information such as trees/tokens and semantic information such as
symbols/types/synthetics.
- `Symbol`: a unique identifier for a definition. For example, the `String`
class has the symbol `java/lang/String#`. The type alias to `String` in the
Scala standard library has the symbol `scala/Predef.Symbol#`.
- `SymbolInfo`: metadata about a `Symbol`, including accessibility
(private/protected/...), kind (val/def/class/trait/...), properties
(final/abstract/implicit), language (Scala/Java) and type signature.
- `SType`: an ADT that encodes the full Scala type system.
- `STree`: an ADT that encodes synthetic tree nodes such as inferred `.apply`,
inferred type parameters, implicit arguments and implicit conversions.

## Scalafix v0

Latest Scaladoc:
[v@VERSION@](https://static.javadoc.io/ch.epfl.scala/scalafix-core_2.12/@VERSION@/scalafix/v0/index.html)

This is a legacy API that exists to smoothen the migration for existing Scalafix
v0.5 rules. If you are writing a new Scalafix rule, please use the v1 API.

## Scalameta Trees

Latest Scaladoc:
[v@SCALAMETA@](https://static.javadoc.io/org.scalameta/trees_2.12/@SCALAMETA@/scala/meta/index.html)

The type `scala.meta.Tree` is a
["lossless syntax tree"](http://www.oilshell.org/blog/2017/02/11.html) for the
Scala language syntax. A Scalameta Tree preserves more details than the Scala
compiler syntax trees. The detailed syntax tree nodes in Scalameta are great for
fine-grained manipulation source code like is required in Scalafix.

The tree API is available through `import scala.meta._`. Key data structures
include:

- `Tree`: the supertype of all tree nodes. Key methods include:
- `pos: Position`: the source code position of this tree node
- `symbol: Symbol`: extension method made available via
`import scalafix.v1._`, requires an implicit `SemanticDocument`. Returns the
the symbol of this tree node
- `syntax: String`: the pretty-printed tree node
- `structure: String`: the raw structure of this tree node, useful for
figuring out how to pattern match against a particular tree node.
- `Term`: the supertype for all term nodes in "term position". Terms are the
parts of programs that get executed at runtime such as `run(42)` in
`val x: String = run(42)`.
- `Type`: the supertype for all type nodes in "type position". Type are the
parts of programs that only exist at compile time, such as `String` in
`val x: String = ""`.
- `Stat`: the supertype for all tree nodes that can appear in "statement
position". Statement position is the
- `Defn`: the supertype for all definitions

## Scalameta Tokens

Latest Scaladoc:
[v@SCALAMETA@](https://static.javadoc.io/org.scalameta/tokens_2.12/@SCALAMETA@/scala/meta/tokens/Token.html)

The type `scala.meta.Token` is a representation of all lexical tokens in the
Scala language. Each token kind has it's own type, such as `Token.Space`, and
`Token.KwClass` ("keyword `class`").

The token API is available through `import scala.meta._`. Key data structures
include:

- `Token`: the supertype for all token kinds. Sub-types of `Token` include
`Token.Space`, `Token.KwClass` ("keyword `class`"), `Token.LeftParen` and
`Token.Colon`.
- `Tokens`: a sequence of tokens with efficient collection operations such as
slice, take and drop. It's important to keep in mind that a normal source file
can have tens of thousands of tokens so operations like `filter` may perform
badly.
61 changes: 61 additions & 0 deletions docs/developers/before-you-begin.md
View file
@@ -0,0 +1,61 @@
---
id: before-you-begin
title: Before you write code
---

Before you dive right into the implementation details of your rule, it's good to
ask yourself the following questions first.

## Do you want a linter or rewrite?

- Rewrites can automatically fix problems, which is ideal but means rewrites can
only address problems that have obvious solutions.

- Linters highlight problematic code without providing a fix, which means they
demand more interaction from the user but can at the same time target a larger
problem space than rewrites.

## Is your rule syntactic or semantic?

- Syntactic rules are simple to run since they don't require compilation but
syntactic rules are limited since they don't have access to symbols and types.
- Semantic rules are are slower and more complicated to run since they require
compilation but at the same time semantic rules are able to perform more
advanced analysis since they have access to symbols and types.

## What is an acceptable false-positive rate?

- If you are doing a one-off migration script for your company codebase then it
might be OK to manually fix trickier corner cases.
- If your linter catches critical production bugs it's maybe OK that it has a
few false positives. If your linter catches low-importance bugs like enforcing
a code style then false positives are not acceptable.

## What diff does the rewrite produce?

Before implementing a rule, it's good to manually migrate/refactor a few
examples first. Manual refactoring is great to discover corner and estimate how
complicated the rule needs to be.

## Who will use your rule?

The target audience/users of your rule can impact the implementation the rule.
If you are the only end-user of the rule, then you can maybe take shortcuts and
worry less about rare corner cases that may be easier to fix manually. If your
rule is intended to be used by the entire Scala community, then you might want
to be more careful with corner cases.

## What codebase does your rule target?

Is your rule specific to a particular codebase or is the rule intended to be
used for any arbitrary project? It's easier to validate your a rule if it will
only run on a single codebase. You may not even need tests, since the codebase
is the only test. If your rule is intended to be used in any random codebase,
you need to put more effort into handling corner cases. In general, the smaller
the target domain of your rule, the easier it is to implement a rule.

## How often will your rule run?

Are you writing a one-off migration script or will your rule run on every pull
request? A rule that runs on every pull request should ideally have some unit
tests and be documented so that other people can help maintain the rule.
110 changes: 110 additions & 0 deletions docs/developers/setup.md
View file
@@ -0,0 +1,110 @@
---
id: setup
title: Setup
---

Scalafix supports writing custom linter and rewrite rules. Custom rules have
access to a rich Scalafix public API and users can run custom rules with the
same UX as built-in rules, including tab completions in the sbt-scalafix plugin.

## sbt

The quickest way to get started in sbt is to use the `scalafix.g8` project
template:

```
cd reponame # The project you want to implement rules for.
sbt new scalacenter/scalafix.g8 --rule="reponame" --version="v1.0"
cd scalafix
sbt tests/test
```

The `--rule=<reponame>` option should match the name of your GitHub repository.
The template generates a `scalafix/` directory with four different sbt projects

```scala
scalafix
├── rules // rule implementations
├── input // code to be analyzed by rule
├── output // expected output from running input on rules
└── tests // small project where unit tests run
```

The `scalafix/` directory is a self-contained sbt build and can live in the same
directory as your existing library.

To run unit tests, execute `tests/test`. For a productive edit/test/debug
workflow, it's recommended to open an sbt shell and re-run the tests on file
save

```
$ sbt
> ~tests/test
```

### Import into IntelliJ

The project generated should import into IntelliJ like a normal project. Input
and output test files are written in plain `*.scala` files and should have full
IDE support.

### Customizing input and output projects

The input and output projects are regular sbt projects and can have custom
library dependencies. To add library dependencies to the input project, add the
following sbt settings

```diff
lazy val input = project.settings(
+ libraryDependencies += "org" %% "name" % "version"
)
```

The only requirement is that the input project uses a Scala compiler version
that is supported by
[SemanticDB](https://github.com/scalameta/scalameta/blob/master/semanticdb/semanticdb3/semanticdb3.md)
compiler plugin. Supported Scala compiler versions include:

- Scala @SCALA211@
- Scala @SCALA212@

The output project can use any Scala compiler version.

### Running a single test

To run a single test case instead of the full test suite run the following
command: `tests/testOnly *MySuite -- -z MyRegex`. This will run only test cases
with the names matching the regular expression "MyRegex" inside test suites
matching the regular expression `*MySuite`. See the ScalaTest documentation
section:
[Selecting suites and tests](http://www.scalatest.org/user_guide/using_the_runner#selectingSuitesAndTests)
for more details about the `-z` option.

## Library API

To test custom Scalafix rules outside of sbt use the library API of
`scalafix-testkit` directly. Start by adding a dependency on the
[scalafix-testkit](https://search.maven.org/artifact/ch.epfl.scala/scalafix-testkit_@SCALA212@/@VERSION@/jar)
artifact

```
coursier fetch ch.epfl.scala:scalafix-testkit_@SCALA212@:@VERSION@
```

Next, create a test suite that extends the class `SemanticRuleSuite`

```scala
package myproject
class MyTests extends scalafix.testkit.SemanticRuleSuite {
runAllTests()
}
```

This setup assumes there will be a resource file `scalafix-testkit.properties`
with instructions about the full classpath of the input sources and source
directories for the input and output sources. To learn more about what values
should go into `scalafix-testkit.properties`, consult the Scaladocs.

Latest Scaladoc:
[`TestkitProperties`](https://static.javadoc.io/ch.epfl.scala/scalafix-testkit_@SCALA212@/@VERSION@/scalafix/testkit/TestkitProperties.html)
109 changes: 109 additions & 0 deletions docs/developers/sharing-rules.md
View file
@@ -0,0 +1,109 @@
---
id: sharing-rules
title: Sharing rules
---

## Run the rule from source code

Running a rule from source code is the simplest way to run a custom rule.
However, rules that are compiled from source have the following limitations:

- Limited code re-use, rule must be implemented in a single source file
- No dependencies, rules can only use the Scalafix public API
- Slow, rule is re-compiled on every invocation so it's not great for
interactive usage.
- No tab completion in the sbt shell, users need to manually type the path to
the source file to discover the rule

To run a custom rule from source, start by installing the sbt plugin as
instructed in [here](../users/installation.md#sbt). The SemanticDB compiler
plugin must be enabled to run semantic rules like `NamedLiteralArguments`.

Next, open an sbt shell

```
sbt
>
```

You have different options to run the rule from soruce: `file:`, `http:` or
`github:`

### Using `file:`

The easiest way to run a custom rule from source code is to use the
`file:/path/to/NamedLiteralArguments.scala` syntax.

```
scalafix file:/path/to/NamedLiteralArguments.scala
```

### Using `http:`

Another way to run a rule from source is to publish it as a gist and share the
raw URL

```
scalafix https://gist.githubusercontent.com/olafurpg/eeccf32f035d13f0728bc94d8ec0a776/raw/78c81bb7f390eb98178dd26ea03c42bd5a998666/NamedLiteralArguments.scala
```

### Using `github:`

Another way to run custom rules from source is to use the `github:org/repo`
scheme.

```
sbt
> scalafix github:olafurpg/named-literal-arguments
...
```

The expansion rules for `github:org/repo` is the following

| Before | After |
| -------------------------------------- | ------------------------------------------------------------------------ |
| `github:org/repo` | `scalafix/rules/src/main/scala/fix/Repo.scala` |
| `github:org/some-repo` | `scalafix/rules/src/main/scala/fix/SomeRepo.scala` |
| `github:org/repo/RuleName` | `scalafix/rules/src/main/scala/fix/RuleName.scala` |
| `github:org/repo/RuleName?sha=HASH125` | (at commit `HASH125`) `scalafix/rules/src/main/scala/fix/RuleName.scala` |

## Publish your rule to Maven Central

Branch:
[`step-6`](https://github.com/olafurpg/named-literal-arguments/commit/88f18b16c9dd939a3f1c08672b121ac2bc1c590d)

The most robust way to share a custom rule is to publish it as a library to
Maven Central. The diff in `step-6` shows the necessary changes to build.sbt to
populate publishing information. The
[sbt-ci-release](https://github.com/olafurpg/sbt-ci-release) readme documents
the further steps to configure gpg and Sonatype.

Once you have your rule, users can depend on it in the sbt plugin by updating
`scalafixDependencies`

```scala
// build.sbt
scalafixDependencies in ThisBuild +=
"com.geirsson" % "named-literal-arguments_2.12" % "VERSION"
// sbt shell
> scalafix NamedLiteralArguments
```

Users of the Scalafix command-line interface can use the `--tool-classpath` flag

```
scalafix \
--tool-classpath $(coursier fetch com.geirsson:named-literal-arguments_2.12:VERSION) \
-r NamedLiteralArguments \
--classpath MY_PROJECT_CLASSPATH \
my-project/src/main/scala
```

Don't be intimidating by publishing to Maven Central, it gets easier once you've
done it the first time. The benefits of publishing a rule to Maven Central are
many.

- Dependencies, you can use custom library dependency to implement your rule
- Fast to run, no need to re-compile the rule on every Scalafix invocation
- Tab completion in sbt, users can tab-complete your rule when using sbt plugin
j

0 comments on commit 387fd27

Please sign in to comment.