TokenIgnores has no effect

[andrewvaughan]

Check for existing issues

• Completed

Environment

macOS, Homebrew install

vale version 2.30.0

Describe the bug / provide steps to reproduce it

With the following .vale.ini configuration:

##
# Vale prose linter.
#
# Vale is a command-line and open source linting tool that allows developers to define and apply custom rules for
# documentation/editorial style guides. Incorporating Vale in documentation helps to detect errors and provides error-
# fixing suggestions.
#
# MegaLinter manages all execution of this linter.
#
# @link https://vale.sh/
# @link https://vale.sh/docs/topics/config/
# @link https://vale.sh/explorer/
# @link https://megalinter.io/beta/descriptors/spell_vale/
#


##
# The linter configuration root is the base for Vale styles.
#
StylesPath = .config/linters/vale/styles


##
# This project's hardened consistency requirement dictates the most verbose option.
#
# The default level is `warning` for projects that don't need stringent prose requirements.
#
MinAlertLevel = suggestion


##
# This project uses the Google Developer Documentation Style Guide.
#
# @link https://developers.google.com/style/
#
Packages = Google


##
# Two vocab sets exist for ops and the project.
#
# This isn't used for spellcheck, as CSpell manages that. Instead, this informs Vale what words to treat as proper nouns
# and various formatting standards.
#
# These Vocab configurations are available in the `.config/linters/vale/styles/Vocab` folder.
#
# @link https://vale.sh/docs/topics/vocab/
#
Vocab = Ops, Project


## ---------------------------------------------------------------------------------------------------------------------
# Global rules.
#
[*]


##
# The Google style-guide is the default for this project.
#
BasedOnStyles = Vale, Google


##
# Don't lint code examples inside of comments.
#
# @link https://vale.sh/docs/topics/config/#blockignores
#
#BlockIgnores = (?s) *(```.*```)
#BlockIgnores = (?s) *(\x60{3}([^\x60]|\x60[^\x60]|\x60{2}[^\x60])*\x60{3})


##
# Inline ignores.
#
# Vale ignores the following in-line scenarios:
#
#   1. Inline `code examples`
#   2. Comment markers starting with `@`
#
# TODO Additionally, this configuration ignores all URLs due to a problem with Vale being able to decipher URLs in code
# blocks. This is an interim fix.
# @link https://github.com/errata-ai/vale/issues/249
#
# @link https://vale.sh/docs/topics/config/#tokenignores
#
#TokenIgnores = (@.*|`.*`|https?:\/\/[^\n\s]+)          # <-- Easier to read version
#TokenIgnores =  (@[^\n]+|\x60[^\x60\n]+\x60|<?https?:\/{2}[^\n\s]+>?)    # <-- I believe this is required for Go RegEx / Vale
TokenIgnores = (.*)    # <-- Trying to just ignore everything to see if Ignore is working


##
# Users of these projects understand common acronyms.
#
Google.Acronyms = NO


##
# The project disables spelling as the project uses CSpell for spellcheck.
#
Vale.Spelling = NO


## ---------------------------------------------------------------------------------------------------------------------
# File extension associations.
#
# These instructions tell Vale how to parse comments for extensions it doesn't natively support.
#
# @link https://vale.sh/docs/topics/scoping/#code-1
# @link https://vale.sh/docs/topics/config/#format-associations
#
#[formats]

#ecrc = js
#editorconfig = pl
#gitattributes = pl
#gitignore = pl
#ini = pl
#Makefile = pl
#mk = pl
#prettierignore = pl

I can't get TokenIgnores to have any effect on my files. Specifically, I am trying to get Vale to ignore any @ tags in my Javascript comments, for example:

/**
 * This is a JavaDoc comment.
 *
 * @see EnhancedCore.LOG_LEVEL
 * @see {@link https://example.com}
 */

However, even if I set TokenIgnores to (.*) (which I presume would ignore everything?) I get the same results:

...

 .github/workflows/src/EnhancedCore.js
 138:22  error       'e.L' should have one space.  Google.Spacing 
 156:42  error       's.A' should have one space.  Google.Spacing 
 158:10  suggestion  Use parentheses judiciously.  Google.Parens  
 402:42  error       'e.A' should have one space.  Google.Spacing 
 404:43  error       'e.A' should have one space.  Google.Spacing 
 525:37  error       Avoid using '#'.              Vale.Avoid  

...

The errors above are all the result of Vale processing comments after @ characters.

[jdkato]

I've added some information to the ignore pattern docs to hopefully clarify their intended use case. Most importantly, they are a markup-only concept that is supported in formats that are converted to HTML.

Additionally, Vale doesn't offer a means of adding support for file formats through configuration. If a file format doesn't appear under Formats, it isn't supported. Trying to use a combination of [formats] and ignore patterns is rarely going to produce good results.

---

That said, what you're trying to do here is possible.

Using this configuration

[formats]
js = md

[*.md]
BasedOnStyles = Vale, Google

TokenIgnores = (@+[^\n]+)

with this file

/**
 * This is a JavaDoc comment.
 *
 * @see EnhancedCore.LOG_LEVEL
 * @see {@link https://example.com}
 */
function test() {
    return 1;
}

// This is is a comment.

// @foo This is is a comment.

// This `is is` a Markdown comment.

will produce:

❯ vale cases/test.js                                                                                                                         

 cases/test.js
 11:9  error  'is' is repeated!  Vale.Repetition 

✖ 1 error, 0 warnings and 0 suggestions in 1 file.

See Code for information on assigning embedded markup formats.