🧪 Test gap analysis — 11 gaps found in standalone generators, git ref validation, and create_pr helpers

[github-actions]

Test Gap Analysis

Test suite snapshot: 817 total tests (unit + integration); 55 integration tests in tests/compiler_tests.rs; 3 init tests; 8 MCP HTTP tests

Previous gaps (all resolved ✅): ExecutionResult::warning(), sanitize_config ##[ shorthand, get_tool_config sanitization, lean runtime integration test, schedule object form with branches.

---

Priority Gaps

Module	Function/Path	Why It Matters	Suggested Test
safeoutputs/mod.rs	validate_git_ref_name — rules for @{, ~, ^, :, ?, *, [, \\, //, .lock suffix, path component starting with .	Security-critical: used by create_branch and create_git_tag to block git refname injection. All 10+ rules are untested in isolation.	Dedicated test per rule (see below)
compile/common.rs	generate_setup_job (standalone, pool-aware 3-arg version)	Generates SetupJob YAML for standalone pipelines; the 1ES version is tested but the common.rs public function is not	Test empty steps → "", non-empty → contains SetupJob, pool name, agent display name
compile/common.rs	generate_teardown_job (standalone, 3-arg)	Same issue as above; teardown job must dependsOn: ProcessSafeOutputs	Test empty → "", non-empty → TeardownJob, dependsOn: ProcessSafeOutputs, pool
compile/common.rs	generate_agentic_depends_on	Controls whether PerformAgenticTask waits for SetupJob; wrong output breaks pipeline ordering	Test empty steps → "", non-empty → "dependsOn: SetupJob"
compile/common.rs	generate_finalize_steps	Inline post-steps in the agentic task; always untested	Test empty → "", non-empty → step content present
compile/common.rs	generate_checkout_steps	Generates - checkout: <alias> entries; wrong output silently omits repository checkouts	Test empty slice → "", multiple aliases → each checkout: line present
compile/common.rs	generate_repositories	Generates repository resource YAML; untested	Test empty → "", multiple repos → each repository:, type:, name:, ref: field correct
tests/compiler_tests.rs	Integration: complete-agent.md compiled output	The fixture has setup:, teardown:, post-steps: but the YAML-validity test doesn't assert SetupJob, TeardownJob, or dependsOn: SetupJob actually appear	Assert SetupJob, TeardownJob, dependsOn: SetupJob in compiled output
agent_stats.rs	sanitize_for_markdown — ##[ → [filtered][ path	The ##[error]bad replacement is not exercised; only ##vso[ is tested	assert_eq!(sanitize_for_markdown("##[error]bad"), "[filtered][error]bad")
safeoutputs/create_branch.rs	Validation: leading -, spaces, >200-char name, source_branch path traversal	Three of five validation rules have no test	branch_name = "-bad", "has space", "a".repeat(201), source_branch = "../evil"
safeoutputs/create_pr.rs	truncate_error_body	Truncates long error text to avoid oversized API payloads; Unicode boundary safety is unverified	Test "hello" with max_len=3 → "hel", multi-byte char boundary → safe truncation

---

Suggested Test Cases

1. validate_git_ref_name — dedicated unit tests (security-critical)

// In src/safeoutputs/mod.rs tests block
#[test]
fn test_validate_git_ref_name_rejects_at_brace() {
    assert!(validate_git_ref_name("branch@{0}", "b").is_err());
}
#[test]
fn test_validate_git_ref_name_rejects_dotlock_suffix() {
    assert!(validate_git_ref_name("my-branch.lock", "b").is_err());
}
#[test]
fn test_validate_git_ref_name_rejects_consecutive_slashes() {
    assert!(validate_git_ref_name("feat//thing", "b").is_err());
}
#[test]
fn test_validate_git_ref_name_rejects_backslash() {
    assert!(validate_git_ref_name("feat\\evil", "b").is_err());
}
#[test]
fn test_validate_git_ref_name_rejects_special_chars() {
    for ch in ['~', '^', ':', '?', '*', '['] {
        let name = format!("feat{ch}bad");
        assert!(validate_git_ref_name(&name, "b").is_err(), "should reject '{ch}'");
    }
}
#[test]
fn test_validate_git_ref_name_rejects_component_starting_with_dot() {
    assert!(validate_git_ref_name("feat/.hidden", "b").is_err());
}
#[test]
fn test_validate_git_ref_name_rejects_trailing_dot() {
    assert!(validate_git_ref_name("my-branch.", "b").is_err());
}
#[test]
fn test_validate_git_ref_name_accepts_valid_refs() {
    assert!(validate_git_ref_name("feature/add-login", "b").is_ok());
    assert!(validate_git_ref_name("v1.2.3", "b").is_ok());
    assert!(validate_git_ref_name("release/2026-04-17", "b").is_ok());
}

2. Standalone generate_setup_job / generate_teardown_job / generate_agentic_depends_on / generate_finalize_steps

// In src/compile/common.rs tests block
#[test]
fn test_generate_setup_job_empty_returns_empty() {
    assert!(generate_setup_job(&[], "My Agent", "MyPool").is_empty());
}
#[test]
fn test_generate_setup_job_with_steps() {
    let step: serde_yaml::Value = serde_yaml::from_str("bash: echo setup").unwrap();
    let out = generate_setup_job(&[step], "My Agent", "MyPool");
    assert!(out.contains("SetupJob"));
    assert!(out.contains("My Agent - Setup"));
    assert!(out.contains("name: MyPool"));
    assert!(out.contains("checkout: self"));
    assert!(out.contains("echo setup"));
}
#[test]
fn test_generate_teardown_job_with_steps() {
    let step: serde_yaml::Value = serde_yaml::from_str("bash: echo td").unwrap();
    let out = generate_teardown_job(&[step], "My Agent", "MyPool");
    assert!(out.contains("TeardownJob"));
    assert!(out.contains("dependsOn: ProcessSafeOutputs"));
    assert!(out.contains("name: MyPool"));
}
#[test]
fn test_generate_agentic_depends_on_empty_steps() {
    assert!(generate_agentic_depends_on(&[]).is_empty());
}
#[test]
fn test_generate_agentic_depends_on_with_steps() {
    let step: serde_yaml::Value = serde_yaml::from_str("bash: x").unwrap();
    assert_eq!(generate_agentic_depends_on(&[step]), "dependsOn: SetupJob");
}
#[test]
fn test_generate_finalize_steps_empty() {
    assert!(generate_finalize_steps(&[]).is_empty());
}
#[test]
fn test_generate_finalize_steps_with_step() {
    let step: serde_yaml::Value = serde_yaml::from_str("bash: echo done").unwrap();
    let out = generate_finalize_steps(&[step]);
    assert!(out.contains("echo done"));
}

3. generate_checkout_steps / generate_repositories

#[test]
fn test_generate_checkout_steps_empty() {
    assert!(generate_checkout_steps(&[]).is_empty());
}
#[test]
fn test_generate_checkout_steps_multiple() {
    let aliases = vec!["repo-a".to_string(), "repo-b".to_string()];
    let out = generate_checkout_steps(&aliases);
    assert!(out.contains("- checkout: repo-a"));
    assert!(out.contains("- checkout: repo-b"));
}
#[test]
fn test_generate_repositories_empty() {
    assert!(generate_repositories(&[]).is_empty());
}
#[test]
fn test_generate_repositories_single() {
    use crate::compile::types::Repository;
    let repos = vec![Repository {
        repository: "my-repo".to_string(),
        repo_type: "git".to_string(),
        name: "org/my-repo".to_string(),
        repo_ref: "refs/heads/main".to_string(),
    }];
    let out = generate_repositories(&repos);
    assert!(out.contains("repository: my-repo"));
    assert!(out.contains("type: git"));
    assert!(out.contains("name: org/my-repo"));
    assert!(out.contains("ref: refs/heads/main"));
}

4. Integration: compiled complete-agent.md asserts setup/teardown jobs

// In tests/compiler_tests.rs
#[test]
fn test_standalone_complete_agent_has_setup_and_teardown_jobs() {
    let compiled = compile_fixture("complete-agent.md");
    assert!(compiled.contains("SetupJob"), "Should generate SetupJob");
    assert!(compiled.contains("TeardownJob"), "Should generate TeardownJob");
    assert!(compiled.contains("dependsOn: SetupJob"),
        "PerformAgenticTask should depend on SetupJob");
    assert!(compiled.contains("echo \"Setup step\"") || compiled.contains("echo 'Setup step'"),
        "Should include setup step content");
}

5. sanitize_for_markdown ##[ path

// In src/agent_stats.rs tests block
#[test]
fn test_sanitize_for_markdown_strips_shorthand_pipeline_command() {
    assert_eq!(
        sanitize_for_markdown("##[error]Something bad"),
        "[filtered][error]Something bad"
    );
}

6. create_branch missing validation cases

#[test]
fn test_validation_rejects_branch_starting_with_dash() {
    // ... branch_name = "-bad" → Err
}
#[test]
fn test_validation_rejects_branch_with_spaces() {
    // ... branch_name = "my branch" → Err
}
#[test]
fn test_validation_rejects_branch_over_200_chars() {
    // ... branch_name = "a".repeat(201) → Err
}
#[test]
fn test_validation_rejects_source_branch_with_traversal() {
    // ... source_branch = "../evil" → Err
}

7. truncate_error_body

// In src/safeoutputs/create_pr.rs tests block
#[test]
fn test_truncate_error_body_shorter_than_max() {
    assert_eq!(truncate_error_body("hello", 100), "hello");
}
#[test]
fn test_truncate_error_body_exact_max() {
    assert_eq!(truncate_error_body("hello", 5), "hello");
}
#[test]
fn test_truncate_error_body_longer_than_max() {
    assert_eq!(truncate_error_body("hello world", 5), "hello");
}
#[test]
fn test_truncate_error_body_multibyte_boundary() {
    // "héllo" — é is 2 bytes; char boundary must not be violated
    let s = "héllo";
    let result = truncate_error_body(s, 3); // 3 chars: h, é, l
    assert_eq!(result, "hél");
}

---

Coverage Summary

Module	Public Fns	Tests	Notes
safeoutputs/mod.rs	validate_git_ref_name	0 direct	Security-critical; indirect coverage partial
compile/common.rs	generate_setup_job, generate_teardown_job, generate_agentic_depends_on, generate_finalize_steps, generate_checkout_steps, generate_repositories	0	All standalone-specific generators untested
agent_stats.rs	sanitize_for_markdown	1 (partial)	##[ branch not covered
safeoutputs/create_branch.rs	validate()	2/6 paths	Leading dash, spaces, >200 chars, source_branch missing
safeoutputs/create_pr.rs	truncate_error_body	0	Unicode-safe truncation unverified

---

This issue was created by the automated test gap finder. Previous run: 2026-04-15. Modules audited this cycle: all. Total tests found: 817.

Generated by Test Gap Finder · ● 4.7M · ◷