Configuration

Reviewer uses YAML for configuration. You can set up the configuration yourself or pull from pre-built recipes for popular code review tools.

Configuration File Locations

TODO

• Local/Personal
• Project/Repo

A Quick Summary

Reviewer exists to make it seamless for you to use a variety of code quality tools without having to remember individual quirks or worry about typos on long commands.

The configuration serves to support up to six key types of commands for each tool. Not all are required, but filling out more of them provides more super-powers.

• install
• prepare
• review
• format
• serve
• generate

The primary idea is that instead of remembering or typing a lengthy cryptic command for each of these, you can specify your command once, and then all of your tools have the same command-line interface. Moreover, because they have the same interface, Reviewer can easily run commands individually or in automated batches.

All of the rest of the configuration helps support how the commands work and what information Reviewer presents if a tool fails. That all makes it much easier to keep moving seamlessly when Reviewer catches something you need to fix.

Order Matters

The order in which you configure the tools in the YAML file will determine the order in which Reviewer executes the tools. This is a core design principle behind Reviewer's implementation for several reasons.

Since Reviewer's purpose is to help you manage multiple code review tools, you invariably have some tools that are more important, and you have some tools where a failure would mean that it's a waste of time to run some of the other tools. And when Reviewer encounters a tool with a failing score, it stops immediately to focus on the results of that tool without running any additional tools.

For example, if you run bundler-audit, and it fails due to a serious security issue in one of your core dependencies, it's probably not a great use of your time to work on syntax problems. So tools that review your dependencies are great to put at the beginning of your configuration. Similarly, if your test suite fails, fixing syntax issues isn't as important as fixing the test failures.

While it may seem obvious, here's a rough outline for a good approach to ordering your tools:

1. Dependency Audits
2. Unit Test Suite/Coverage
3. System/Integration Test Suite/Coverage
4. Code Complexity Audits
5. Security Static Analysis
6. Accessibility Audits
7. Performance Testing
8. Lingering TODO/NOTE/FIXME/OPTIMIZE Cleanup
9. Linters/Formatters
10. Documentation Quality

With that sequence, you can be fairly confident you won't be fixing problems that are actually downstream effects of upstream problems. Of course, you may not be using tools in all of those categories, but hopefully it provides some big picture food for thought.

Tool Configuration Block Syntax

The YAML serves as a structured way to unify the options for running each tool. It also provides opportunities for adding additional meta data like name, description, and links to help with a variety of house-keeping related to the tools.

Here's a quick example to show the big picture before diving into each configuration option individually.

<tool-key>:
  disabled: false
  name: Tool
  description: A tool that finds issues and fixes code.
  tags: [syntax, security]
  links:
    home: https://example.com
    install: https://example.com/install
    ignore_syntax: https://example.com/ignore
    disable_syntax: https://example.com/disable
  commands:
    install: 'bundle install tool'
    prepare: 'bundle exec tool update'
    review: 'bundle exec tool'
    format: 'bundle exec tool --format'
    serve: 'bundle exec tool server && open http://192.168.0.1:8000/tool'
    generate:
    publish:
    quiet_option: '--quiet'
    max_exit_status: 1
  env:
    report: false
  flags:
    format: json

The only truly required configuration option is the review command entry. So if you want to try out a new tool without investing too much time in the initial configuration, you could use something as simple as the following and then fill in the other values if you decide to use it long-term.

tool-name-key:
  commands:
    review: 'bundle exec tool'
    quiet_option: '--quiet' # Not required, but strongly suggested.

Note: The quiet_option is technically optional (many tools don't provide an option for it), but it is strongly suggested for the best overall experience with Reviewer since it helps improve the signal-to-noise ratio of running multiple tools.

Tool Configuration Options

YAML Key

<tool-key> serves as a shortcut to run a specific tool. So if you have dozens of tools configured, you may not want to wait for all of them to run and may only want the one you're focused on. So if you have an entry named tests, you could run rvw tests, and it will only run that tool.

Disabled - Boolean

If there isn't an entry for disabled, a tool is assumed to be enabled. So disabled: false would do the same as excluding the key entirely. disabled: true means that tool will be skipped when running multiple tools.

When a tool is disabled, you can still run it individually by specifiying it. So if you run rvw <tool-key>, that tool will be run even if it is disabled. This makes it easier to configure a tool that you occasionally want easy access to but otherwise don't want to run regularly.

Name & Description - Strings

These are primarily used for readable output when the tools are run. By default, Reviewer suppresses the output for tools and instead shows only their name, description, and how long the command took to run. If you don't provide a value for the name, Reviewer will use the tool's key as the name when displayed in the output.

These are most helpful when used to serve as reminders or explanations of a tool's purpose and provide flexibility. For example, if you're using Rubocop, you could just as easily use the name 'Rubocop' as 'Ruby Syntax'. It's purely descriptive and doesn't affect behavior.

And you can optionally use the description to expand on what it does. With only a couple of tools, you may not find it all that helpful, but once you have a suite of tools, the description can come in really handy when different team members add new tools.

Rubocop · Review Ruby Syntax
Success (1.586s)

Tags - Array of Strings

The Tags entry enables one of the core features of Reviewer by letting you easily run different subsets or combinations of tools. For example, if you only want to run specific tools in your continuous integration, you could tag those tools with 'ci' and run rvw ci (or rvw -t ci or rvw --tags ci).

There's incredible flexibility with tags. You could use them to identify the types of files that tool scans: ruby, css, html, javascript, js, erb, yaml, etc. Or you could use them based on teams and responsibilities: frontend, backend, ops, etc. Or you could use purpose-based tags: security, syntax, docs, etc. Context-based tags can come in handy as well: ci, pr, release, etc. And sometimes, speed-based tags could come in handy: fast, slow, turtle, rabbit, etc.

Links

Links help make it easier to work with the individual tools. They aren't required, but they invariably come in handy.

There are specific keys to use for different link types:

• home: links to the tool's home page
• install: links to the tool's installation instructions for when the system reports that a command is missing. If you have customized installation instructions for your setup, you can of course link to that instead.
• ignore_syntax: links to the tool's documenation for the proper syntax to ignore a specific piece of code.
• disable_syntax: links to the tool's documentation for fully disabling specific rules so they don't need to be ignored.

For instance, in the case where you run reviewer, but the system can't find the necessary executable, it displays a link to the tool's home page and installation instructions if they're present in the tool's configuration.

Tool · This is a tool without the command installed
Failure · Tool executable for 'missing_tool'

Reviewer ran this command:
bundle exec missing_tool > /dev/null

Try installing the tool:
bundle exec gem install missing_tool

Read the installation guidance:
https://example.com/docs/install

Commands

Commands are the core element of Reviewer, and there are several types:

• prepare: allows for tools that may need an extra step to fetch current data before running. Bundler Audit's bundle-audit update is a good example of where this comes in handy. It will run before the review command if it hasn't been run in the previous 6 hours. Otherwise, Reviewer assumes that it's fresh enough and skips it to enable faster runs.
• install makes it easier for someone to install a missing tool. It's never directly executed by Reviewer. Instead, if a command seems to be missing, it's displayed by Reviewer to save you from having to look up the precise command for installation. In cases where installation is more complex, leave it out and let the install link show the way.
• review is the only required command and will be the main command run by Reviewer. It's best to keep it pared down to the absolute minimum command necessary. If you have environment variables or flags you'd like to use every time, they can be configured separately. (More on that below.)
• format allows for tools like linters that can auto-correct issues. Like the review command, it's best to keep it to the bare minimum command and configure flags and environment variables separately.
• serve supports tools that can serve reports locally and is especially handy when they have live reloading options
• generate lets you specify a version of the command to generate a static report locally. It will frequently be very similar to the review command but with different flags related to generating and formatting a report. For some tools, generate and review may implicitly be the same thing if they always generate a report locally as a side effect. In those cases, you may want to have review explicitly skip generating output to improve performance or avoid wasting disk space.

As you might expect, when using the rvw command, Reviewer will look to the review command entry, and when using the fmt command, Reviewer will look for the format command entry. If a tool doesn't have an entry for the format command, it won't be run at all when formatting.

In addition to the command types, there are two additional command-related options:

• quiet_option is not required but is strongly suggested and informs Reviewer how to reduce/suppress (depending on the tool's behavior) the tool's output. Reviewer also adds > /dev/null to commands to help fully silence tools. If a tool fails, Reviewer will automatically re-run it without the quiet options so that you can see the results.
• max_exit_status defaults to 0 if left out but provides a way to set a higher "failure" threshhold for a tool. While anyting other than 0 is considered a failure for most code review tools, some have a more nuanced defintion of failure. In those cases, you can specify something higher so Reviewer is less draconian about results.

Env & Flags

Environment variables and flags can (and generally should) be configured separately from commands. Environment variables will be pre-pended to all commands run by Reviewer, but flags will only be appended to the review command when it runs. Environment variables are configured separately so they can easily be shared across all commands and reduce repetition.

Reviewer strives to be fairly smart about how it converts these values into the proper command-line format without being "too smart" about it. For example, if your environment variable's value involves spaces and needs quotes, Reviewer will handle it.

Environment Variables

A quick example might do a better job of illustrating how it works. If you had the following in your configuration...

example:
  commands:
    review: 'bundle exec example'
  env:
    example: var
    another: var with spaces

...Reviewer would convert it to...

EXAMPLE=var; ANOTHER='var with spaces'; bundle exec example

If you're curious about the specific implementation, take a look at Reviewer::Tool::Env and the associated tests.

Flags

Similar to environment variables, Reviewer is smart with flags and supports both single-letter (ex. -f value) and multi-letter (ex. --flag value) flags.

However, anytime a tool offers both single-letter and multi-letter options for a flag, you should definitely lean towards the verbose multi-letter options. Since you won't have to type them repeatedly, there's no significant benefit to configuring the single-letter version when a flag supports both options. And the multi-letter versions provide an extra layer of documentation and make it easier to use search engines to investigate a specific flag.

Like with environment variables, if you're curious about the precise implementation, you'll want to look at Reviewer::Tools::Flags and its associated tests.

example:
  commands:
    review: 'bundle exec example'
  flags:
    e: var
    example: variable

...Reviewer would convert it to...

bundle exec example -e var --example variable

If flags are only used for the review command, you might be wondering why flags have their own configuration section instead of adding them directly to the review command, and that would be a very fair question that's worth discussing.

You can absolutely add flags directly to the commands, and everything should work fine. In practice, though, I've found giving them dedicated individual entries helps make them more readable as well as allowing for comments to explain their purpose. Since not all flags are clear or self-explanatory, you can never have too much documentation.

If you ever run into a case where Reviewer incorrectly translates the configuration values to the appropriate options in the command, adding the flags directly to the review command is a good workaround. Of course, if that happens, submitting an issue would be greatly appreciated.