Skip to content

[curriculum-eval] curriculum-scorecard: overall page scores and trend analysis #2528

Description

@github-actions

This scorecard summarises curriculum quality scores for all 87 workshop pages using Bloom's Taxonomy, Cognitive Load Theory, and Mayer's Multimedia Principles. Scores are computed by the shared rubric in .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py.


Part Summary

Part Files Mean Score Std Dev
Part 1 — core path (lessons 00–14) 12 5.88 / 10.0 ±0.42
Part 2 — advanced (lessons 15+) 75 6.12 / 10.0 ±0.42
Overall corpus 87 6.09 / 10.0 ±0.43

Part 1 Scores (lessons 00–14)

File Overall Score
workshop/07d-confirm-model-access.md 5.29 / 10.0
workshop/04-github-actions-intro.md 5.42 / 10.0
workshop/05-agentic-workflows-intro.md 5.43 / 10.0
workshop/08-run-your-workflow.md 5.67 / 10.0
workshop/14b-pr-reviewer-workflow.md 5.67 / 10.0
workshop/05b-agentic-workflows-security.md 5.75 / 10.0
workshop/14-next-steps.md 5.91 / 10.0
workshop/09-agentic-editing.md 6.03 / 10.0
workshop/08b-interpret-your-run.md 6.07 / 10.0
workshop/02a-setup-codespace.md 6.09 / 10.0
workshop/05c-agentic-workflows-practice.md 6.59 / 10.0
workshop/07-your-first-workflow.md 6.59 / 10.0

Part 2 Scores (lessons 15+)

File Overall Score
workshop/side-quest-01-02-environment-reference.md 5.31 / 10.0
workshop/side-quest-11-06-anthropic-key.md 5.39 / 10.0
workshop/side-quest-17-07-repo-poisoning.md 5.47 / 10.0
workshop/side-quest-05-03-two-file-structure.md 5.51 / 10.0
workshop/15-conditional-logic.md 5.53 / 10.0
workshop/side-quest-20-01-memory-patterns.md 5.59 / 10.0
workshop/side-quest-11-01b-workflow-structure.md 5.62 / 10.0
workshop/side-quest-11-08-frontmatter-tools-outputs.md 5.64 / 10.0
workshop/16-connect-data-source.md 5.71 / 10.0
workshop/side-quest-11-09-agent-session-phases.md 5.71 / 10.0
workshop/side-quest-13-04-token-optimization.md 5.71 / 10.0
workshop/side-quest-11-04-annotated-workflow.md 5.72 / 10.0
workshop/17-add-mcp-tools.md 5.75 / 10.0
workshop/side-quest-06-03a-copilot-requests-permission.md 5.75 / 10.0
workshop/side-quest-17-02-security-architecture.md 5.77 / 10.0
workshop/side-quest-enterprise-setup.md 5.77 / 10.0
workshop/side-quest-16-02-secrets-and-permissions.md 5.79 / 10.0
workshop/20-persistent-memory.md 5.81 / 10.0
workshop/side-quest-17-03-prompt-injection.md 5.81 / 10.0
workshop/side-quest-06-03-copilot-token.md 5.83 / 10.0
workshop/26-manage-costs-and-budgets.md 5.85 / 10.0
workshop/side-quest-05-01-actions-power-user.md 5.85 / 10.0
workshop/side-quest-16-03-token-exfiltration.md 5.85 / 10.0
workshop/side-quest-05-02-aw-deep-dive.md 5.87 / 10.0
workshop/side-quest-10-02-jailbreak-brief.md 5.89 / 10.0
workshop/side-quest-11-05-event-triggers.md 5.89 / 10.0
workshop/side-quest-13-01-schedule-expressions.md 5.91 / 10.0
workshop/side-quest-17-01-mcp-concepts.md 5.91 / 10.0
workshop/side-quest-12-01-iterate-agent-output.md 5.93 / 10.0
workshop/side-quest-17-04-permission-escalation.md 5.93 / 10.0
workshop/side-quest-25-01-audit-reference.md 5.93 / 10.0
workshop/side-quest-11-07-openai-key.md 5.95 / 10.0
workshop/side-quest-17-06-output-injection.md 5.97 / 10.0
workshop/21-inline-sub-agents.md 5.99 / 10.0
workshop/side-quest-09-01f-debugging-checklist.md 6.01 / 10.0
workshop/side-quest-16-05-long-lived-credentials.md 6.01 / 10.0
workshop/side-quest-17-05-supply-chain-mcp.md 6.01 / 10.0
workshop/side-quest-08-01-codespaces-actions-write.md 6.03 / 10.0
workshop/side-quest-16-04-deterministic-vs-agentic-data-ops.md 6.03 / 10.0
workshop/side-quest-15-01-expressions-and-contexts.md 6.05 / 10.0
workshop/side-quest-10-01-agent-brief.md 6.09 / 10.0
workshop/18-share-and-reuse.md 6.15 / 10.0
workshop/side-quest-09-01-debug-output.md 6.15 / 10.0
workshop/side-quest-24-01-runner-infrastructure.md 6.19 / 10.0
workshop/side-quest-09-01a-pattern-long-plan-chain.md 6.20 / 10.0
workshop/25-audit-and-observability.md 6.23 / 10.0
workshop/28-orchestrate-workflows.md 6.23 / 10.0
workshop/side-quest-02-01-local-terminal.md 6.25 / 10.0
workshop/side-quest-15-02-chaining-conditions.md 6.25 / 10.0
workshop/side-quest-01-01-terminal-basics.md 6.26 / 10.0
workshop/22-error-handling-and-resilience.md 6.27 / 10.0
workshop/side-quest-09-01b-pattern-empty-results.md 6.35 / 10.0
workshop/side-quest-09-01c-pattern-safe-output-blocked.md 6.35 / 10.0
workshop/19-research-driven-training-node.md 6.37 / 10.0
workshop/27-evaluate-workflow-quality.md 6.37 / 10.0
workshop/side-quest-09-01d-pattern-permission-denied.md 6.37 / 10.0
workshop/side-quest-13-02-pr-summary-pattern.md 6.37 / 10.0
workshop/side-quest-09-01e-pattern-done-no-write.md 6.39 / 10.0
workshop/side-quest-06-01-install-troubleshooting.md 6.43 / 10.0
workshop/side-quest-13-03-pr-checklist-pattern.md 6.47 / 10.0
workshop/side-quest-16-01-github-output.md 6.48 / 10.0
workshop/side-quest-07-01-compile-workflow.md 6.49 / 10.0
workshop/side-quest-26-01-forecast-costs.md 6.51 / 10.0
workshop/23-ab-experiments.md 6.53 / 10.0
workshop/side-quest-06-03c-copilot-github-token-ui-only.md 6.53 / 10.0
workshop/side-quest-11-03-better-prompts.md 6.61 / 10.0
workshop/24-self-hosted-runners.md 6.63 / 10.0
workshop/side-quest-13-01-pr-labeler-pattern.md 6.67 / 10.0
workshop/side-quest-06-04-install-local.md 6.77 / 10.0
workshop/side-quest-01-03-permission-errors.md 6.79 / 10.0
workshop/side-quest-06-03b-copilot-github-token.md 6.79 / 10.0
workshop/side-quest-21-01-sub-agent-syntax.md 6.95 / 10.0
workshop/side-quest-11-01-frontmatter-deep-dive.md 7.05 / 10.0
workshop/side-quest-06-02-cca-codespace.md 7.25 / 10.0
workshop/side-quest-11-02-yaml-frontmatter.md 7.25 / 10.0

Score Trends (history window: 20 commits)

File Baseline Latest Delta Direction
workshop/02a-setup-codespace.md 6.09 6.09 +0.0 stable
workshop/04-github-actions-intro.md 5.42 5.42 +0.0 stable
workshop/05-agentic-workflows-intro.md 5.57 5.43 -0.14 declining
workshop/05b-agentic-workflows-security.md 5.83 5.75 -0.08 declining
workshop/05c-agentic-workflows-practice.md 6.59 6.59 +0.0 stable
workshop/07-your-first-workflow.md 6.59 6.59 +0.0 stable
workshop/07d-confirm-model-access.md 5.29 5.29 +0.0 stable
workshop/08-run-your-workflow.md 5.67 5.67 +0.0 stable
workshop/08b-interpret-your-run.md 6.07 6.07 +0.0 stable
workshop/09-agentic-editing.md 6.03 6.03 +0.0 stable
workshop/14-next-steps.md 5.91 5.91 +0.0 stable
workshop/14b-pr-reviewer-workflow.md 5.67 5.67 +0.0 stable
workshop/15-conditional-logic.md 5.53 5.53 +0.0 stable
workshop/16-connect-data-source.md 5.71 5.71 +0.0 stable
workshop/17-add-mcp-tools.md 5.75 5.75 +0.0 stable
workshop/18-share-and-reuse.md 6.15 6.15 +0.0 stable
workshop/19-research-driven-training-node.md 6.37 6.37 +0.0 stable
workshop/20-persistent-memory.md 5.81 5.81 +0.0 stable
workshop/21-inline-sub-agents.md 5.99 5.99 +0.0 stable
workshop/22-error-handling-and-resilience.md 6.27 6.27 +0.0 stable
workshop/23-ab-experiments.md 6.53 6.53 +0.0 stable
workshop/24-self-hosted-runners.md 6.63 6.63 +0.0 stable
workshop/25-audit-and-observability.md 6.23 6.23 +0.0 stable
workshop/26-manage-costs-and-budgets.md 5.85 5.85 +0.0 stable
workshop/27-evaluate-workflow-quality.md 6.35 6.37 +0.02 stable
workshop/side-quest-01-01-terminal-basics.md 6.26 6.26 +0.0 stable
workshop/side-quest-01-02-environment-reference.md 5.31 5.31 +0.0 stable
workshop/side-quest-01-03-permission-errors.md 6.79 6.79 +0.0 stable
workshop/side-quest-02-01-local-terminal.md 6.25 6.25 +0.0 stable
workshop/side-quest-05-01-actions-power-user.md 5.85 5.85 +0.0 stable
workshop/side-quest-05-02-aw-deep-dive.md 5.89 5.87 -0.02 stable
workshop/side-quest-05-03-two-file-structure.md 5.55 5.51 -0.04 stable
workshop/side-quest-06-01-install-troubleshooting.md 6.43 6.43 +0.0 stable
workshop/side-quest-06-02-cca-codespace.md 7.25 7.25 +0.0 stable
workshop/side-quest-06-03-copilot-token.md 5.87 5.83 -0.04 stable
workshop/side-quest-06-03a-copilot-requests-permission.md 5.79 5.75 -0.04 stable
workshop/side-quest-06-03b-copilot-github-token.md 6.81 6.79 -0.02 stable
workshop/side-quest-06-03c-copilot-github-token-ui-only.md 6.55 6.53 -0.02 stable
workshop/side-quest-06-04-install-local.md 6.83 6.77 -0.06 declining
workshop/side-quest-07-01-compile-workflow.md 6.51 6.49 -0.02 stable
workshop/side-quest-08-01-codespaces-actions-write.md 6.05 6.03 -0.02 stable
workshop/side-quest-09-01-debug-output.md 6.15 6.15 +0.0 stable
workshop/side-quest-09-01a-pattern-long-plan-chain.md 6.20 6.20 +0.0 stable
workshop/side-quest-09-01b-pattern-empty-results.md 6.39 6.35 -0.04 stable
workshop/side-quest-09-01c-pattern-safe-output-blocked.md 6.35 6.35 +0.0 stable
workshop/side-quest-09-01d-pattern-permission-denied.md 6.37 6.37 +0.0 stable
workshop/side-quest-09-01e-pattern-done-no-write.md 6.43 6.39 -0.04 stable
workshop/side-quest-09-01f-debugging-checklist.md 6.01 6.01 +0.0 stable
workshop/side-quest-10-01-agent-brief.md 6.11 6.09 -0.02 stable
workshop/side-quest-10-02-jailbreak-brief.md 5.89 5.89 +0.0 stable
workshop/side-quest-11-01-frontmatter-deep-dive.md 7.09 7.05 -0.04 stable
workshop/side-quest-11-01b-workflow-structure.md 5.65 5.62 -0.03 stable
workshop/side-quest-11-02-yaml-frontmatter.md 7.25 7.25 +0.0 stable
workshop/side-quest-11-03-better-prompts.md 6.61 6.61 +0.0 stable
workshop/side-quest-11-04-annotated-workflow.md 5.72 5.72 +0.0 stable
workshop/side-quest-11-05-event-triggers.md 5.89 5.89 +0.0 stable
workshop/side-quest-11-06-anthropic-key.md 5.51 5.39 -0.12 declining
workshop/side-quest-11-07-openai-key.md 5.95 5.95 +0.0 stable
workshop/side-quest-11-08-frontmatter-tools-outputs.md 5.64 5.64 +0.0 stable
workshop/side-quest-11-09-agent-session-phases.md 5.71 5.71 +0.0 stable
workshop/side-quest-12-01-iterate-agent-output.md 5.93 5.93 +0.0 stable
workshop/side-quest-13-01-pr-labeler-pattern.md 6.67 6.67 +0.0 stable
workshop/side-quest-13-01-schedule-expressions.md 5.91 5.91 +0.0 stable
workshop/side-quest-13-02-pr-summary-pattern.md 6.37 6.37 +0.0 stable
workshop/side-quest-13-03-pr-checklist-pattern.md 6.47 6.47 +0.0 stable
workshop/side-quest-13-04-token-optimization.md 5.71 5.71 +0.0 stable
workshop/side-quest-15-01-expressions-and-contexts.md 6.05 6.05 +0.0 stable
workshop/side-quest-15-02-chaining-conditions.md 6.25 6.25 +0.0 stable
workshop/side-quest-16-01-github-output.md 6.48 6.48 +0.0 stable
workshop/side-quest-16-02-secrets-and-permissions.md 5.79 5.79 +0.0 stable
workshop/side-quest-16-03-token-exfiltration.md 5.85 5.85 +0.0 stable
workshop/side-quest-16-04-deterministic-vs-agentic-data-ops.md 6.03 6.03 +0.0 stable
workshop/side-quest-16-05-long-lived-credentials.md 6.01 6.01 +0.0 stable
workshop/side-quest-17-01-mcp-concepts.md 5.91 5.91 +0.0 stable
workshop/side-quest-17-02-security-architecture.md 5.77 5.77 +0.0 stable
workshop/side-quest-17-03-prompt-injection.md 5.81 5.81 +0.0 stable
workshop/side-quest-17-04-permission-escalation.md 5.93 5.93 +0.0 stable
workshop/side-quest-17-05-supply-chain-mcp.md 6.01 6.01 +0.0 stable
workshop/side-quest-17-06-output-injection.md 5.97 5.97 +0.0 stable
workshop/side-quest-17-07-repo-poisoning.md 5.47 5.47 +0.0 stable
workshop/side-quest-20-01-memory-patterns.md 5.59 5.59 +0.0 stable
workshop/side-quest-21-01-sub-agent-syntax.md 6.95 6.95 +0.0 stable
workshop/side-quest-24-01-runner-infrastructure.md 6.19 6.19 +0.0 stable
workshop/side-quest-25-01-audit-reference.md 5.93 5.93 +0.0 stable
workshop/side-quest-26-01-forecast-costs.md 6.55 6.51 -0.04 stable
workshop/side-quest-enterprise-setup.md 5.77 5.77 +0.0 stable

Overall Trend

  • Baseline mean score: 6.10
  • Latest mean score: 6.09
  • Delta: -0.01
  • Direction: stable

Bloom's Taxonomy Distribution

Level Count % of corpus
remember 5 6%
understand 15 17%
apply 47 54%
analyze 7 8%
evaluate 4 5%
create 9 10%

Assessment: The distribution is acceptably pyramidal. The corpus has 15 understand-level and 7 analyze-level pages, satisfying both minimum thresholds (has_min_understand_steps: true, has_min_analyze_steps: true). However, 54% at the Apply level with only 23% at Analyze/Evaluate/Create suggests the workshop could benefit from more synthesis and critique exercises in later lessons.


Bloom's Reclassification Audit

File Level Justification
02a-setup-codespace.md apply Primary cues: intro:install, title:setup.
04-github-actions-intro.md understand Primary cues: title:intro, intro:intro.
05-agentic-workflows-intro.md understand Primary cues: title:intro, intro:intro.
05b-agentic-workflows-security.md apply Primary cues: intro:run, intro:schedule.
05c-agentic-workflows-practice.md apply Primary cues: intro:run, intro:use.
07-your-first-workflow.md create Primary cues: intro:create, title:write.
07d-confirm-model-access.md apply Primary cues: intro:run, intro:install.
08-run-your-workflow.md apply Primary cues: title:run, intro:run.
08b-interpret-your-run.md apply Primary cues: title:run, intro:run.
09-agentic-editing.md evaluate Primary cues: intro:review, title:test.
14-next-steps.md apply Primary cues: intro:schedule, activity:commands_or_code.
14b-pr-reviewer-workflow.md create Primary cues: intro:create, title:build.
15-conditional-logic.md apply Primary cues: intro:run, intro:use.
16-connect-data-source.md apply Primary cues: intro:run, intro:install.
17-add-mcp-tools.md apply Primary cues: intro:install, activity:commands_or_code.
18-share-and-reuse.md apply Primary cues: intro:run, activity:commands_or_code.
19-research-driven-training-node.md create Primary cues: title:build, intro:build.
20-persistent-memory.md apply Primary cues: intro:run, activity:commands_or_code.
21-inline-sub-agents.md remember Primary cues: title:reference, intro:reference.
22-error-handling-and-resilience.md apply Primary cues: intro:run, activity:commands_or_code.
23-ab-experiments.md evaluate Primary cues: title:test, intro:test.
24-self-hosted-runners.md apply Primary cues: title:run, intro:run.
25-audit-and-observability.md apply Primary cues: intro:run, intro:install.
26-manage-costs-and-budgets.md apply Primary cues: intro:run, activity:commands_or_code.
27-evaluate-workflow-quality.md evaluate Primary cues: title:evaluate, intro:review.
28-orchestrate-workflows.md create Primary cues: intro:build, intro:compose.
side-quest-01-01-terminal-basics.md apply Primary cues: intro:run, intro:use.
side-quest-01-02-environment-reference.md remember Primary cues: title:reference, intro:reference.
side-quest-01-03-permission-errors.md apply Primary cues: intro:run, intro:setup.
side-quest-02-01-local-terminal.md apply Primary cues: intro:run, intro:install.
side-quest-05-01-actions-power-user.md understand Primary cues: intro:intro, intro:introduction.
side-quest-05-02-aw-deep-dive.md understand Primary cues: intro:intro, intro:introduction.
side-quest-05-03-two-file-structure.md understand Primary cues: intro:intro, intro:introduction.
side-quest-06-01-install-troubleshooting.md apply Primary cues: intro:run, title:install.
side-quest-06-02-cca-codespace.md apply Primary cues: intro:run, intro:install.
side-quest-06-03-copilot-token.md apply Primary cues: intro:run, intro:install.
side-quest-06-03a-copilot-requests-permission.md apply Primary cues: intro:run, intro:setup.
side-quest-06-03b-copilot-github-token.md apply Primary cues: intro:run, intro:configure.
side-quest-06-03c-copilot-github-token-ui-only.md apply Primary cues: intro:setup, intro:configure.
side-quest-06-04-install-local.md apply Primary cues: intro:run, title:install.
side-quest-07-01-compile-workflow.md apply Primary cues: intro:run, intro:use.
side-quest-08-01-codespaces-actions-write.md apply Primary cues: title:run, intro:run.
side-quest-09-01-debug-output.md analyze Primary cues: title:debug, intro:diagnose.
side-quest-09-01a-pattern-long-plan-chain.md apply Primary cues: intro:run, intro:setup.
side-quest-09-01b-pattern-empty-results.md apply Primary cues: intro:run, intro:setup.
side-quest-09-01c-pattern-safe-output-blocked.md analyze Primary cues: intro:debug, title:output.
side-quest-09-01d-pattern-permission-denied.md analyze Primary cues: intro:inspect, intro:debug.
side-quest-09-01e-pattern-done-no-write.md create Primary cues: intro:create, title:write.
side-quest-09-01f-debugging-checklist.md analyze Primary cues: intro:inspect, intro:debug.
side-quest-10-01-agent-brief.md apply Primary cues: intro:run, intro:setup.
side-quest-10-02-jailbreak-brief.md understand Primary cues: intro:understand, intro:architecture.
side-quest-11-01-frontmatter-deep-dive.md apply Primary cues: intro:run, intro:configure.
side-quest-11-01b-workflow-structure.md understand Primary cues: intro:intro, intro:introduction.
side-quest-11-02-yaml-frontmatter.md apply Primary cues: intro:use, activity:commands_or_code.
side-quest-11-03-better-prompts.md apply
side-quest-11-04-annotated-workflow.md apply
side-quest-11-05-event-triggers.md apply
side-quest-11-06-anthropic-key.md apply Primary cues: intro:run, title:configure.
side-quest-11-07-openai-key.md apply
side-quest-11-08-frontmatter-tools-outputs.md apply
side-quest-11-09-agent-session-phases.md understand
side-quest-12-01-iterate-agent-output.md apply
side-quest-13-01-pr-labeler-pattern.md apply
side-quest-13-01-schedule-expressions.md apply
side-quest-13-02-pr-summary-pattern.md apply
side-quest-13-03-pr-checklist-pattern.md apply
side-quest-13-04-token-optimization.md apply
side-quest-15-01-expressions-and-contexts.md apply
side-quest-15-02-chaining-conditions.md apply
side-quest-16-01-github-output.md apply
side-quest-16-02-secrets-and-permissions.md apply
side-quest-16-03-token-exfiltration.md apply
side-quest-16-04-deterministic-vs-agentic-data-ops.md analyze
side-quest-16-05-long-lived-credentials.md apply
side-quest-17-01-mcp-concepts.md understand
side-quest-17-02-security-architecture.md apply
side-quest-17-03-prompt-injection.md apply
side-quest-17-04-permission-escalation.md apply
side-quest-17-05-supply-chain-mcp.md apply
side-quest-17-06-output-injection.md apply
side-quest-17-07-repo-poisoning.md apply
side-quest-20-01-memory-patterns.md apply
side-quest-21-01-sub-agent-syntax.md apply
side-quest-24-01-runner-infrastructure.md apply
side-quest-25-01-audit-reference.md apply
side-quest-26-01-forecast-costs.md apply
side-quest-enterprise-setup.md apply

Corpus-Level Observations

  • Checkpoint coverage: All 5 below-threshold files (07d-confirm-model-access.md, side-quest-01-02-environment-reference.md, side-quest-11-06-anthropic-key.md, 04-github-actions-intro.md, 05-agentic-workflows-intro.md) are missing a ## :white_check_mark: Checkpoint section, scoring 0.0 on checkpoint_quality. This is the single most consistent contributor to low scores across the corpus.
  • Active learning gap: Pages scoring below 6.0 uniformly show activity_density < 1.5 — not enough code blocks or checklist items per 100 words for learners to practice while reading.
  • Concept overload: 04-github-actions-intro.md (25 new concepts) and side-quest-11-06-anthropic-key.md (26 new concepts) exceed the 15-concept ideal by a wide margin, placing excessive intrinsic load on learners encountering these pages early in the learning path.
  • Declining pages: Three pages show measurable score decline across the 20-commit window: 05-agentic-workflows-intro.md (−0.14), side-quest-11-06-anthropic-key.md (−0.12), and side-quest-06-04-install-local.md (−0.06). These warrant attention.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔬 Curriculum Quality Evaluator · 78.6 AIC · ⌖ 6.91 AIC · ⊞ 6.2K ·

  • expires on Aug 8, 2026, 1:11 PM UTC

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions