flow-coverage-report

[bvaughn]

During last weeks' sync in London, we chatted about Flow coverage stats and how we might track them over time. I noticed an NPM module called flow-coverage-report that seemed worth testing, so I thought I'd put up a PR for discussion.

Pros

• The module generates a file in flow-coverage/flow-coverage.json that contains lots of useful stats that we could use to compare over time, including:

{
  threshold: 90,
  covered_count: 20770,
  uncovered_count: 10001,
  percent: 67,
  flowStatus: {
    flowVersion: "0.57.3",
    errors: [],
    passed: true
  },
  flowAnnotations: {
    passed: false,
    flowFiles: 72,
    flowWeakFiles: 0,
    noFlowFiles: 76,
    totalFiles: 148
  },
  files: {
    "path/to/file.js": {
      expressions: {
        covered_count: 31,
        uncovered_count: 38,
        uncovered_locs: [
          {
            source:
              "~/path/to/file.js",
            type: "SourceFile",
            start: { line: 10, column: 5, offset: 229 },
            end: { line: 10, column: 43, offset: 268 }
          },
          // ...
        ]
      },
      filename: "path/to/file.js",
      annotation: "no flow",
      percent: 44
    }
  },
  globIncludePatterns: ["packages/**/src/**/*.js"],
  globExcludePatterns: [
    "packages/**/__tests__/*.js",
    "packages/**/__mocks__/*.js"
  ],
}

• CLI supports a "threshold" param, below which it will fail with a non-0 status, so it would be easy to fail on CI if coverage dropped below a certain amount. (I'm not sure we'd actually want to do this.)

Cons

• This takes ~165 seconds to run (on my Macbook Pro).
• I don't know how actively developed the tool is.

Example Syntax

yarn flow-coverage-report         \
  -i "packages/**/src/**/*.js"    \
  -x "packages/**/__tests__/*.js" \
  -x "packages/**/__mocks__/*.js" \
  -t html                         \
  -t json                         \
  -t text                         \
  --threshold 90

Output

[gaearon]

I'd be interested in getting it in if I can run it as yarn flow-coverage without all those arguments. I don't care about using it for CI or anything else, but I'd find it helpful for some local checks now and then.

[bvaughn]

Rebased and added a yarn flow-coverage command.

[gaearon]

Thanks

[gaearon]

IMO would be nice to avoid another top level dotfile if we can.
I'd move it to scripts/flow/coverage-config.

[bvaughn]

I'm not sure how to prevent Flow from trying to parse the JSON file if I move it here. Even adding it to the [ignore] block in the .flowconfig isn't sufficient.

[bvaughn]

For now I'm just going to leave this as a dotfile. I don't know how to make Flow ignore it (and not error) otherwise, and it doesn't seem pressing enough to invest more than a few minutes into.