Skip to content

Add fsharp-project-structure skill#248

Open
T-Gro wants to merge 4 commits intodotnet:mainfrom
T-Gro:feature/fsharp-projects-mgmt
Open

Add fsharp-project-structure skill#248
T-Gro wants to merge 4 commits intodotnet:mainfrom
T-Gro:feature/fsharp-projects-mgmt

Conversation

@T-Gro
Copy link
Member

@T-Gro T-Gro commented Mar 5, 2026

Adds skill for F# project file structure: compilation order, .fsi signature files, and common FS0039/FS0010/FS0034 fixes.

Eval results (gpt-4.1, 3 runs, judged by claude-opus-4.6):

Scenario Baseline With Skill Pairwise
Add module to project 2.3/5 5.0/5 MuchBetter
Fix broken file order (FS0039) 2.7/5 4.3/5 MuchBetter
Add .fsi signature file 1.7/5 4.0/5 MuchBetter

Overall: +64.6%, significant, CI [+17.2%, +63.0%], overfitting 0.08 (low).

T-Gro and others added 3 commits March 5, 2026 22:53
… files

Teaches agents F# compilation order rules, signature file placement,
and common FS0039/FS0010/FS0034 fixes. Includes eval with three scenarios
(add module, fix broken order, add signature file) and test fixtures.

Validated +64.6% improvement on gpt-4.1 (significant, CI [+17.2%, +63.0%]).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… files

Teaches agents F# compilation order rules, signature file placement,
and common FS0039/FS0010/FS0034 fixes. Includes eval with three scenarios
(add module, fix broken order, add signature file) and test fixtures.

Validated +64.6% improvement on gpt-4.1 (significant, CI [+17.2%, +63.0%]).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@T-Gro T-Gro requested a review from timheuer as a code owner March 5, 2026 21:58
Copilot AI review requested due to automatic review settings March 5, 2026 21:58
@T-Gro T-Gro requested a review from dbreshears as a code owner March 5, 2026 21:58
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new .NET skill focused on F# .fsproj compilation ordering and signature file (.fsi) placement, along with an eval suite and fixtures to measure improvements.

Changes:

  • Added fsharp-project-structure skill documentation under plugins/dotnet/skills/.
  • Added eval scenarios + F# project fixtures under tests/dotnet/fsharp-project-structure/.
  • Updated .github/CODEOWNERS to assign ownership for the new skill and test area.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/dotnet/fsharp-project-structure/eval.yaml Adds 3 eval scenarios covering adding modules, fixing file order, and adding .fsi signature files.
tests/dotnet/fsharp-project-structure/OrderService/Domain.fs Adds an F# domain model fixture for ordering/signature scenarios.
tests/dotnet/fsharp-project-structure/OrderService/Services.fs Adds an F# services fixture referencing Domain.
tests/dotnet/fsharp-project-structure/OrderService/Program.fs Adds an executable entry point fixture used by scenarios.
tests/dotnet/fsharp-project-structure/OrderService/OrderService.fsproj Adds a “correctly ordered” .fsproj fixture.
tests/dotnet/fsharp-project-structure/BrokenOrder/Domain.fs Adds a domain fixture used in the “broken order” scenario.
tests/dotnet/fsharp-project-structure/BrokenOrder/Services.fs Adds a services fixture that will fail if compiled before Domain.fs.
tests/dotnet/fsharp-project-structure/BrokenOrder/Program.fs Adds an entry point fixture for the broken-order project.
tests/dotnet/fsharp-project-structure/BrokenOrder/BrokenOrder.fsproj Adds an intentionally misordered .fsproj fixture to trigger FS0039.
plugins/dotnet/skills/fsharp-project-structure/SKILL.md Adds the new skill guidance for .fsproj ordering and .fsi rules.
.github/CODEOWNERS Assigns code owners for the new skill and its tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@T-Gro T-Gro requested a review from a team March 6, 2026 10:35
@ViktorHofer
Copy link
Member

/evaluate

@github-actions
Copy link
Contributor

github-actions bot commented Mar 6, 2026

Skill Validation Results

Skill Scenario Baseline With Skill Δ Skills Loaded Overfit Verdict
dotnet-trace-collect High CPU in Kubernetes on Linux (.NET 8) 4.0/5 5.0/5 +1.0 ✅ dotnet-trace-collect; tools: skill, report_intent, view ✅ 0.18
dotnet-trace-collect .NET Framework on Windows without admin privileges 2.3/5 5.0/5 +2.7 ✅ dotnet-trace-collect; tools: skill ✅ 0.18
dotnet-trace-collect .NET 10 on Linux with root access and native call stacks 1.0/5 4.3/5 +3.3 ✅ dotnet-trace-collect; tools: skill ✅ 0.18
dotnet-trace-collect Memory leak on Linux (.NET 8) 3.0/5 3.3/5 +0.3 ✅ dotnet-trace-collect; tools: skill, report_intent, view ✅ 0.18
dotnet-trace-collect Slow requests on Windows with PerfView 4.7/5 5.0/5 +0.3 ✅ dotnet-trace-collect; tools: skill, report_intent, view, glob ✅ 0.18
dotnet-trace-collect Excessive GC on Linux (.NET 8) 4.0/5 4.7/5 +0.7 ✅ dotnet-trace-collect; tools: skill ✅ 0.18
dotnet-trace-collect Hang or deadlock diagnosis on Linux 3.7/5 4.0/5 +0.3 ✅ dotnet-trace-collect; tools: skill, report_intent, view ✅ 0.18
dotnet-trace-collect Windows container high CPU with PerfView 1.7/5 5.0/5 +3.3 ✅ dotnet-trace-collect; tools: skill, report_intent, view, glob ✅ 0.18
dotnet-trace-collect Long-running intermittent issue with PerfView triggers 3.0/5 5.0/5 +2.0 ✅ dotnet-trace-collect; tools: skill, report_intent, view, glob ✅ 0.18
dotnet-trace-collect Linux pre-.NET 10 needing native call stacks 2.0/5 5.0/5 +3.0 ✅ dotnet-trace-collect; tools: skill, report_intent, view, glob ✅ 0.18
dotnet-trace-collect Windows modern .NET with admin high CPU 2.3/5 5.0/5 +2.7 ✅ dotnet-trace-collect; tools: skill, report_intent, view, glob ✅ 0.18
dotnet-trace-collect Memory leak on .NET Framework Windows 3.3/5 5.0/5 +1.7 ✅ dotnet-trace-collect; tools: skill, report_intent, view ✅ 0.18
dotnet-trace-collect Kubernetes with console access prefers console tools 5.0/5 4.7/5 -0.3 ✅ dotnet-trace-collect; tools: skill, report_intent, view, glob ✅ 0.18
dotnet-trace-collect Container installation without .NET SDK 3.3/5 4.0/5 +0.7 ✅ dotnet-trace-collect; tools: skill ✅ 0.18
dotnet-pinvoke Generate LibraryImport declaration from C header (.NET 8+) 3.7/5 5.0/5 +1.3 ✅ dotnet-pinvoke; tools: skill ✅ 0.11
dotnet-pinvoke Generate LibraryImport declaration from C header (.NET Framework) 3.7/5 5.0/5 +1.3 ✅ dotnet-pinvoke; tools: skill ✅ 0.11
fsharp-project-structure Add a module to an F# project 5.0/5 5.0/5 0.0 ✅ fsharp-project-structure; tools: skill ✅ 0.08
fsharp-project-structure Fix broken file order causing FS0039 4.3/5 4.0/5 -0.3 ⚠️ NOT ACTIVATED ✅ 0.08
fsharp-project-structure Add a signature file to define public API 5.0/5 5.0/5 0.0 ✅ fsharp-project-structure; tools: skill, glob, read_bash ✅ 0.08

Model: claude-opus-4.6 | Judge: claude-opus-4.6

Full results

@ManishJayaswal
Copy link
Contributor

@T-Gro - can you update this PR to create a new plugin named dotnet-fsharp and add this skill to it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants