feat: improve fastly-ngwaf skill score from 68% to 97%#7
Draft
yogesh-tessl wants to merge 2 commits into
Draft
Conversation
Hey @jedisct1 👋 I ran your skills through `tessl skill review` at work and found some targeted improvements for `fastly-ngwaf`. Here's the full before/after: | Skill | Before | After | Change | |-------|--------|-------|--------| | fastly-ngwaf | 68% | 97% | +29% | | fastly | 100% | 100% | — | | fastly-cli | 89% | 89% | — | | falco | 89% | 89% | — | | fastlike | 89% | 89% | — | | viceroy | 95% | 95% | — | | xvcl | 94% | 94% | — | <details> <summary>Changes made to <code>fastly-ngwaf</code></summary> - **Removed verbose "Trigger and scope" section** that repeated the frontmatter description — saves tokens and removes redundancy - **Added concrete curl + jq examples** for each audit step (list workspaces, fetch rules, check specific signals, search uncovered endpoints) so the skill is copy-paste actionable - **Added expected output examples** showing healthy vs unhealthy workspace audit results, so the agent knows how to format and interpret findings - **Added error handling table** covering common failures (missing API key, 403 permission errors, missing jq dependency) - **Restructured the workflow** into a clear 4-step sequence with validation checkpoints and a feedback loop (searching for uncovered login endpoints when LOGINATTEMPT is missing) - **Preserved the existing description** (already scored 100%) and all domain-specific terminology (NGWAF signal codes, API endpoints) </details> I also stress-tested your `fastly-ngwaf` skill against a few real-world task evals and it held up really well on auditing workspaces with partially-configured login protection rules where LOGINDISCOVERY was missing but LOGINATTEMPT was enabled. Kudos for that. Honest disclosure — I work at @tesslio where we build tooling around skills like these. Not a pitch — just saw room for improvement and wanted to contribute. Want to self-improve your skills? Just point your agent (Claude Code, Codex, etc.) at [this Tessl guide](https://docs.tessl.io/evaluate/optimize-a-skill-using-best-practices) and ask it to optimize your skill. Ping me — [@yogesh-tessl](https://github.com/yogesh-tessl) — if you hit any snags. Thanks in advance 🙏
jedisct1
reviewed
May 12, 2026
Comment on lines
+109
to
+114
| | Error | Cause | Fix | | ||
| | -------------------------------- | ---------------------------- | ------------------------------------------------ | | ||
| | `FASTLY_API_KEY not set` | Environment variable missing | `export FASTLY_API_KEY=<token>` | | ||
| | `API call failed with status 403`| Token lacks NGWAF scope | Verify token has `global:read` permission | | ||
| | `No workspaces found` | NGWAF not provisioned | Enable NGWAF on the account first | | ||
| | `jq is not installed` | Missing dependency | `brew install jq` or `apt-get install -y jq` | |
Collaborator
There was a problem hiding this comment.
Can you fix that funky indentation?
jedisct1
reviewed
May 12, 2026
Comment on lines
+48
to
+51
| | ---------------------- | ----------------------------------------------------------------- | | ||
| | Login Protection | `LOGINDISCOVERY`, `LOGINATTEMPT`, `LOGINSUCCESS`, `LOGINFAILURE` | | ||
| | Credit Card Validation | `CC-VAL-ATTEMPT`, `CC-VAL-FAILURE`, `CC-VAL-SUCCESS` | | ||
| | Gift Card Validation | `GC-VAL-ATTEMPT`, `GC-VAL-FAILURE`, `GC-VAL-SUCCESS` | |
Collaborator
|
Thanks! We use Calibra for evals, but the These changes look good, thanks! Could you just fix the weird table formatting introduced by this diff? Thank you! |
Address review feedback on PR fastly#7 — normalize trailing whitespace in the Validate Critical Signals and Error Handling tables so the closing pipes align vertically in the source markdown.
Author
|
fixed! also thanks @jedisct1, great to see this resonates. We’ve reviewed a bunch of skills now and have picked up a few useful patterns around keeping them maintainable and secure as usage grows. Happy to swap notes over a quick 15-min coffee and share the latest practices we've seen. also eager to hear how you’re thinking about use cases on your side, as it helps us refine the approach. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hey @jedisct1 👋
impressive work. You’ve pulled together 7 skills that span the entire Fastly edge stack from VCL transpiling with XVCL all the way to local Compute testing with Viceroy. It makes the toolkit not just useful, but genuinely easy to explore and learn from. This feels like something built by someone who really understands both the technology and the developer experience
I ran your skills through
tessl skill reviewat work and found some targeted improvements forfastly-ngwaf. Here's the full before/after:Changes made to
fastly-ngwafI also stress-tested your
fastly-ngwafskill against a few real-world task evals and it held up really well on auditing workspaces with partially-configured login protection rules where LOGINDISCOVERY was missing but LOGINATTEMPT was enabled. Kudos for that.Honest disclosure. I work at https://github.com/tesslio where we build tooling around skills like these. Not a pitch, just saw room for improvement and wanted to contribute.
If you want to self-improve your skills, or define your own scenarios to pressure test, just ask your agent (Claude Code, Codex, etc.) to evaluate and optimize your skill with Tessl. Ping me @yogesh-tessl, if you hit any snags.