feat: TechDocs - Add vale linter to check words quality in md files. #2631

Marvin9 · 2020-09-28T08:40:59Z

Hey, I just made a Pull Request!

Closes #2031

Approach to solve #2031 using vale linter.

Added basic vale setup file .vale.ini. source
vale does not provide functionality to ignore folders, we have to provide files manually vale foo.md bar.md to run linter. So automatic way to achieve it is by grep & git.

list all files which are tracked by git. (node_modules and other unnecessary directories are ignored).
find md files.
pass above files to vale

git ls-files | grep -h ".md" | xargs vale

added above command in package.json as lint:docs.

Generated vocab.txt file.

Some keywords, names (eg. api, https) are considered as vale.Spelling error, so we have to add them inside vocab.txt file.
We can list all unique words which are considered as error in vocab.txt using below command.

yarn run lint:docs | grep -o "'[a-z A-Z]*'" | grep -o "[a-z A-Z]*" | sort | uniq > .github/styles/vocab.txt

Now we have to manually validate all words listed in vocab.txt, discard which are invalid and correct them in file where they were located.
Added vale github actions.

we have to add that action before node_modules are added.

Above steps are included in this PR.

Usage:

yarn run lint:docs

Limitation:

update in vocab.txt is required for false positives.

✔️ Checklist

All tests are passing yarn test
Screenshots attached (for UI changes)
Relevant documentation updated
Prettier run on changed files
Tests added for new functionality
Regression tests added for bug fixes

Initialize .vale.ini file Add 'lint:docs' script to package.json, to lint all md files except the ones which are located in node_modules Generate 'vocab.txt' by using command 'yarn run lint:docs' | grep -o ''[a-z A-Z]*'' | grep -o '[a-z A-Z]*' | sort | uniq > .github/styles/vocab.txt Add steps to github workflow 'master' to check docs quality

emmaindal · 2020-09-28T08:44:34Z

WOW!! This is awesome, will definitely look through it more asap 💯

andrewthauer

This is really neat. I looked at vale a little while back but haven't tried it. Good to see it in use.

An additional thought would be to have a lint-staged / git hook check as well. This would help avoid people not noticing issues until CI.

package.json

.github/workflows/master.yml

…atform

…latform

…ent purpose

…est event

Marvin9 · 2020-09-28T13:29:47Z

Updates:

separate CI to check docs quality. All files are checked on push and only updated filese are checked on pull_request
used shx npm package to wrap grep command.
xargs is not supported in shx, to solve this issue on windows platform, I've tried to achieve similar functionality in node script at /scripts/check-docs-quality.js
Added command in pre-commit hook.

Can you please check yarn run lint:docs on windows machine
@emmaindal @andrewthauer

.github/workflows/docs-quality-checker.yml

hooloovooo · 2020-09-28T14:00:17Z

Updates:

* separate CI to check docs quality. All files are checked on `push` and only updated filese are checked on `pull_request`

* used [shx](https://github.com/shelljs/shx) npm package to wrap `grep` command.

* `xargs` is not supported in `shx`, to solve this issue on windows platform, I've tried to achieve similar functionality in node script at `/scripts/check-docs-quality.js`

* Added command in pre-commit hook.

Can you please check yarn run lint:docs on windows machine
@emmaindal @andrewthauer

Hey! This is the error i'm getting trying to run on windows.

PS C:\Users\sqvar\projects\backstage> yarn run lint:docs
yarn run v1.22.1
$ node ./scripts/check-docs-quality
The command line is too long.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

…e(s) on pull request

Marvin9 · 2020-09-28T16:13:46Z

@hooloovooo Can you please check it again?

package.json

scripts/check-docs-quality.js

hooloovooo · 2020-09-29T08:32:24Z

@hooloovooo Can you please check it again?

Now i'm getting this error when running the lint command

hooloovooo · 2020-09-29T08:39:34Z

@hooloovooo Can you please check it again?

If running check-docs-quality.js directly using node I get this error instead

'.' is not recognized as an internal or external command,
operable program or batch file.

Marvin9 · 2020-09-29T12:21:36Z

@andrewthauer I have removed shebang as we are using this script only once. Is it ok?

@hooloovooo This time I have checked in windows machine and is working. Can you please verify?

andrewthauer · 2020-09-29T13:11:13Z

@Marvin9 - I'm a bit indifferent, but my default is to use executables for scripts. I would say that if the file has an executable flag the the shebang should be there if not it's pointless.

hooloovooo · 2020-09-30T07:36:28Z

@Marvin9 Confirmed to work now! :)

Marvin9 · 2020-09-30T07:51:56Z

Documentation updates required to mention installation of vale cli. I will update that.

emmaindal · 2020-10-01T08:18:47Z

@Marvin9 Looks good to me, there is just a markdown files quality check that needs to pass before merge!

…at/docs-linter

emmaindal · 2020-10-01T11:01:19Z

@spotify/silver-lining can we get a review from you since this PR makes some spelling changes to your documentation!

brendasukh

LGTM 👍

emmaindal · 2020-10-02T05:21:16Z

@Marvin9 thank you so much for your contribution 🥳

Marvin9 added 2 commits September 28, 2020 10:30

fix(docs): typos which were reflacted from vale linter's command

3fa4211

Marvin9 requested review from a team as code owners September 28, 2020 08:40

Marvin9 changed the title ~~Feat/docs linter~~ feat: Add vale linter to check words quality in md files. Sep 28, 2020

OrkoHunter changed the title ~~feat: Add vale linter to check words quality in md files.~~ feat: TechDocs - Add vale linter to check words quality in md files. Sep 28, 2020

OrkoHunter added the area:techdocs Related to the TechDocs Project Area label Sep 28, 2020

OrkoHunter added this to Incoming in [DEPRECATED] Hello docs-like-code Sep 28, 2020

andrewthauer reviewed Sep 28, 2020

View reviewed changes

package.json Outdated Show resolved Hide resolved

emmaindal reviewed Sep 28, 2020

View reviewed changes

.github/workflows/master.yml Outdated Show resolved Hide resolved

Marvin9 added 5 commits September 28, 2020 18:09

chore: Separate workflow for quality checking

fa749dc

chore: Added 'shx' dev dependency to support grep command in cross pl…

de2f077

…atform

feat: Add script to operate same quality check process on different p…

97be203

…latform

ignore: remove lint:docs from lint-stages which was added for experim…

3c5e90f

…ent purpose

fix: check-all-files on push event & check-changed-files on pull_requ…

2356fc6

…est event

Marvin9 requested review from andrewthauer and emmaindal September 28, 2020 13:29

emmaindal reviewed Sep 28, 2020

View reviewed changes

.github/workflows/docs-quality-checker.yml Show resolved Hide resolved

.github/workflows/docs-quality-checker.yml Outdated Show resolved Hide resolved

OrkoHunter moved this from Incoming to Awaiting Review/Review In Progress in [DEPRECATED] Hello docs-like-code Sep 28, 2020

Marvin9 added 3 commits September 28, 2020 20:05

chore(CI): triggle workflow only when there is any updates in .md fil…

c6cfec9

…e(s) on pull request

fix: use spawnSync to solve 'The command line is too long.' error

e093faa

fix: github workflow syntax

a212602

Marvin9 requested a review from emmaindal September 28, 2020 16:13

fix: prettier error

15ded28

andrewthauer reviewed Sep 28, 2020

View reviewed changes

package.json Outdated Show resolved Hide resolved

andrewthauer reviewed Sep 28, 2020

View reviewed changes

scripts/check-docs-quality.js Show resolved Hide resolved

Marvin9 added 2 commits September 28, 2020 23:25

chore: add vale command directly to lint-staged

0764fbd

chore: use shebang for easy access

035564c

Marvin9 requested a review from andrewthauer September 29, 2020 06:30

fix: windows script issue & remove shebang

28e80ac

chore: Add shebang flag

995ffc2

benjdlambert approved these changes Sep 30, 2020

View reviewed changes

emmaindal approved these changes Sep 30, 2020

View reviewed changes

Marvin9 added 2 commits October 1, 2020 09:51

chore: better error message related to vale

2df6dfc

chore: mention vale linter in documentation

994ae1e

Marvin9 requested a review from emmaindal October 1, 2020 04:31

Marvin9 added 2 commits October 1, 2020 14:06

Merge branch 'master' of https://github.com/spotify/backstage into fe…

8fb0b4e

…at/docs-linter

fix: spelling errors & add keywords to vocab.txt

967912b

Marvin9 requested a review from a team as a code owner October 1, 2020 08:48

emmaindal approved these changes Oct 1, 2020

View reviewed changes

hooloovooo approved these changes Oct 1, 2020

View reviewed changes

brendasukh approved these changes Oct 1, 2020

View reviewed changes

emmaindal merged commit 5c6a035 into backstage:master Oct 2, 2020

[DEPRECATED] Hello docs-like-code automation moved this from Awaiting Review/Review In Progress to Done Oct 2, 2020

emmaindal mentioned this pull request Oct 7, 2020

Investigate in how we can add vale linter quality checks to all docs components created through backstage #2780

Closed

Marvin9 mentioned this pull request Oct 8, 2020

feature: add ignore directory option to cli errata-ai/vale#243

Closed

OrkoHunter mentioned this pull request Oct 12, 2020

Run vale linter only if it is installed, skip otherwise #2864

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: TechDocs - Add vale linter to check words quality in md files. #2631

feat: TechDocs - Add vale linter to check words quality in md files. #2631

Marvin9 commented Sep 28, 2020 •

edited

emmaindal commented Sep 28, 2020 •

edited

andrewthauer left a comment

Marvin9 commented Sep 28, 2020

hooloovooo commented Sep 28, 2020

Updates:

Marvin9 commented Sep 28, 2020

hooloovooo commented Sep 29, 2020

hooloovooo commented Sep 29, 2020

Marvin9 commented Sep 29, 2020

andrewthauer commented Sep 29, 2020

hooloovooo commented Sep 30, 2020

Marvin9 commented Sep 30, 2020

emmaindal commented Oct 1, 2020

emmaindal commented Oct 1, 2020

brendasukh left a comment

emmaindal commented Oct 2, 2020

feat: TechDocs - Add vale linter to check words quality in md files. #2631

feat: TechDocs - Add vale linter to check words quality in md files. #2631

Conversation

Marvin9 commented Sep 28, 2020 • edited

Hey, I just made a Pull Request!

Usage:

Limitation:

✔️ Checklist

emmaindal commented Sep 28, 2020 • edited

andrewthauer left a comment

Choose a reason for hiding this comment

Marvin9 commented Sep 28, 2020

Updates:

hooloovooo commented Sep 28, 2020

Updates:

Marvin9 commented Sep 28, 2020

hooloovooo commented Sep 29, 2020

hooloovooo commented Sep 29, 2020

Marvin9 commented Sep 29, 2020

andrewthauer commented Sep 29, 2020

hooloovooo commented Sep 30, 2020

Marvin9 commented Sep 30, 2020

emmaindal commented Oct 1, 2020

emmaindal commented Oct 1, 2020

brendasukh left a comment

Choose a reason for hiding this comment

emmaindal commented Oct 2, 2020

Marvin9 commented Sep 28, 2020 •

edited

emmaindal commented Sep 28, 2020 •

edited