Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run F# backend against coverage tool #3688

Closed
5 tasks
StachuDotNet opened this issue Apr 12, 2022 · 5 comments
Closed
5 tasks

Run F# backend against coverage tool #3688

StachuDotNet opened this issue Apr 12, 2022 · 5 comments

Comments

@StachuDotNet
Copy link
Member

StachuDotNet commented Apr 12, 2022

either as a one-off, or long-term through CI.

Some options to consider:

@StachuDotNet StachuDotNet changed the title Consider running code against coverage tool Run F# backend against coverage tool Apr 19, 2022
@StachuDotNet
Copy link
Member Author

StachuDotNet commented Apr 28, 2022

Currently working through hooking up AltCover.

At first, it seemed that many .net code coverage frameworks seem to assume that you use dotnet test to collect/run your tests.
That said, I've finally gotten a(n empty) code coverage report from AltCover with a dotnet tool install --global altcover.global and altcover -i 'fsharp-backend/src' -o '.altcover'. Currently working my way through the usage docs to figure out why the report is empty. I'm likely just missing some options there.

The results I'm seeing are

Instrumenting files from /home/dark/app/fsharp-backend/src/
Writing files to /home/dark/app/.altcover/

Coverage Report: /home/dark/app/coverage.xml

with coverage.xml being rather dull:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<CoverageSession xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Summary numSequencePoints="0" visitedSequencePoints="0" numBranchPoints="0" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="0" minCyclomaticComplexity="1" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="0" minCrapScore="0" maxCrapScore="0" />
  <Modules />
</CoverageSession>

This also copies the fsharp-backend/src folder to .altcover but seems to do no actual 'instrumentation', and the process feels instant.

Going to spend a bit more time with altcover, otherwise may explore other tools.
It may end up being worth allowing dotnet test - there are some options to pursue that path.

@StachuDotNet
Copy link
Member Author

A very quick attempt with Coverlet yields something:

dark@dark-dev:~/app$ coverlet ./fsharp-backend/Build/out/Tests/Debug/net6.0/linux-x64/Tests.dll --target 'dotnet'
Usage: dotnet [options]
Usage: dotnet [path-to-application]
Options:
  -h|--help         Display help.
  --info            Display .NET information.
  --list-sdks       Display the installed SDKs.
  --list-runtimes   Display the installed runtimes.
path-to-application:
  The path to an application .dll file to execute.

Calculating coverage result...
  Generating report '/home/dark/app/coverage.json'
+------------------------+------+--------+--------+
| Module                 | Line | Branch | Method |
+------------------------+------+--------+--------+
| LibService             | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| BwdServer              | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| BackendOnlyStdLib      | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| TestUtils              | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| HttpMiddleware         | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| ApiServer              | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| LibExecution           | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| FuzzTests              | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| LibBinarySerialization | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| QueueWorker            | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| LibExecutionStdLib     | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| Tablecloth             | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| LibRealExecution       | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| LibBackend             | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+
| Prelude                | 0%   | 0%     | 0%     |
+------------------------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 0%   | 0%     | 0%     |
+---------+------+--------+--------+
| Average | 0%   | 0%     | 0%     |
+---------+------+--------+--------+

@StachuDotNet
Copy link
Member Author

StachuDotNet commented Apr 28, 2022

haf/expecto#219 seemingly-useful thread to read after lunch

Ooh looks promising: haf/expecto#219 (comment)

@pbiggar pbiggar moved this from F# rewrite to Complete F# migration / Remove OCaml / Smooth the edges in Old darklang priorities (moving to https://github.com/orgs/darklang/projects/5) May 5, 2022
@StachuDotNet StachuDotNet moved this from Complete F# migration / Remove OCaml / Smooth the edges to In Progress in Old darklang priorities (moving to https://github.com/orgs/darklang/projects/5) May 8, 2022
@StachuDotNet
Copy link
Member Author

I'm abandoning my old PR at #3756. Mostly, it became less of a priority since we migrated to the F# backend (though it's still important) and I'd like to reconsider if AltCov is ideal for us - after some work with it, reaching for something more "mainstream" may be good. Regardless, there are notes in that PR description worth reviewing when this issue is revisited.

@StachuDotNet
Copy link
Member Author

Folded into #5246

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done in Release 3
Development

Successfully merging a pull request may close this issue.

1 participant