Skip to content

Adjust onboarding skill for clarity and improved adherence.#101

Merged
DanverImbue merged 13 commits intomainfrom
danver/adjust-onboarding-skill
Mar 13, 2026
Merged

Adjust onboarding skill for clarity and improved adherence.#101
DanverImbue merged 13 commits intomainfrom
danver/adjust-onboarding-skill

Conversation

@danverbraganza
Copy link
Collaborator

Why

I was onboarding sculptor and noticed a few things with how Sculptor was being onboarded that I didn't like.

I specifically started to fix the .gitignored .offload-image-cache but then this took on a larger scope.

Initial prompt

Please read the onboarding SKILL and suggest any changes that might be made to improve agent understanding and adherence to our goalFlag any ambiguity or imprecision. Do not make
  any edits, just flag suggested improvements to me, and I will work with you to make them. Each set of changes MUST be made in an atomic revision.

…ce, remove redundant step

Merge Modal authentication into Step 1 prerequisites (with blocking
language) and remove the standalone auth step. Move Offload install
note to the intro. Define --copy-dir mapping once in the invocation
script step and reference the script elsewhere instead of repeating
the raw flag. Renumber steps 7-11 accordingly and fix all cross-refs.
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vet found 0 issues.

@danverbraganza
Copy link
Collaborator Author

First response:

High: Likely to cause agent error or incorrect output

  1. Step 1 conflates detection with prerequisite verification (lines 14-34)
  The step title says "Detect Test Framework and Test Paths" but halfway through it pivots to prerequisite verification (uv, modal CLI, cargo-nextest). These are two distinct concerns. An
   agent could finish "Step 1" after detection and skip the prerequisite gate. Suggest: split into Step 1 (Detect) and Step 2 (Verify Prerequisites), renumbering downstream.

  2. stream_output = true in config templates (lines 92, 121) — not in config reference
  The README's configuration reference has no stream_output field under [offload]. Either this field doesn't exist (agent will produce invalid config) or the README is missing it. Verify
  which is correct and align the skill with reality.

  3. Step 4 Approach A / B selection criteria are vague (line 192)
  "Try Approach A first. If it fails" — but the agent has no concrete way to know it will fail until runtime. The actual distinguishing factor (pytest version, custom Item subclasses)
  should be stated as the decision criterion upfront, not "try and see."

  4. Step 4 sub-step 2 is a single 6-line sentence (lines 195-196)
  This compound instruction packs detection of existing hooks, showing the user, explaining the JUnit requirement, asking permission, replacing, and alternatively switching to type =
  "default" into one bullet. An agent will likely miss branches. Break into a decision tree or numbered sub-steps.

  5. Pattern A vs Pattern B naming is confusing (lines 83, 115)
  "Pattern A — Modal provider" uses type = "modal", and "Pattern B — Default provider with Modal scripts" uses type = "default". Both involve Modal. The distinction is "managed vs manual
  Modal lifecycle" but the names don't convey that. The "When to use" guidance for Pattern B (lines 158-166) is separated from Pattern B's definition by 40+ lines, forcing the agent to
  scan back and forth.

  6. .dockerignore lists .offload-image-cache (line 497) with a contradictory comment
  Line 497 says # excluded from sandbox, but should be checked in to git. Line 300 says .offload-image-cache should be checked in to git. But the .dockerignore template in Step 6's
  troubleshooting section excludes it from the Docker context, which is correct — however an agent may confuse "excluded from .dockerignore" with "excluded from .gitignore" since Step 6
  is about .gitignore. The comment should be more precise about what it's excluded from.

DanverImbue and others added 10 commits March 9, 2026 18:43
Remove Pattern A/B naming and the type="default" provider template
with @modal_sandbox.py commands. The Modal provider (type="modal") is
now the sole provider path. Document sandbox_init_cmd as an optional
optimization for build-time setup. Retain the type="default" framework
section for custom discovery/execution, with updated example showing
Modal provider context. Update Step 5 cross-reference accordingly.

Co-Authored-By: Claude <noreply@anthropic.com>
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vet found 0 issues.

@DanverImbue DanverImbue requested a review from jsk11235 March 13, 2026 01:02

This skill walks through onboarding the current repository to use **Offload** — a parallel test runner that executes tests across Modal cloud sandboxes.

**Install Offload**: `cargo install offload@0.5.0`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove hardcoding of the version

- `modal` CLI — run `modal token new` if not yet authenticated
### Step 2: Verify Prerequisites

Verify the following are installed and authenticated. **Do not continue until all prerequisites are confirmed.**
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LLM suggested we break up Step 1 and step 2

@DanverImbue DanverImbue merged commit 4f8477a into main Mar 13, 2026
4 checks passed
@DanverImbue DanverImbue deleted the danver/adjust-onboarding-skill branch March 13, 2026 01:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants