Skip to content

v2.0.0 — Raw source injection, deterministic output, POM output directory

Choose a tag to compare

@cport1 cport1 released this 20 Apr 18:57
· 9 commits to main since this release

Breaking changes

This is a major rework of how AutoSpec handles project context. The regex-based POM scanner has been replaced with raw source code injection.

What changed

1. Raw POM source in prompts (replaces regex extraction)

The LLM now sees the actual source code of your page objects and utilities — not a regex-extracted summary. This eliminates hallucination at the root:

  • No more inventing methods like elements.getCaseFlowChart() that don't exist
  • No more wrong class names in imports (Investigations instead of InvestigationsPage)
  • Works with ANY POM pattern — nested objects, factory functions, deeply nested elements — because the LLM reads the actual code

The scanner is now ~100 lines instead of ~300 lines of fragile regex.

2. Temperature 0 — deterministic output

Same PR now produces the same tests on every run. Previously temperature: 0.1/0.2 caused different test plans and code for identical diffs.

3. POM output directory

New pom_output_directory input. When configured, the LLM can create new page object files as separate files instead of inlining classes in test specs.

- uses: autospec-ai/action@v1
  with:
    pom_output_directory: 'e2e/pages'  # new POMs go here as separate files

When not set, the LLM is instructed to use existing POMs or raw page.locator() calls — no more inline class definitions.

4. Better test coverage detection

The coverage section now shows individual test names (e.g., "should login with valid credentials | should show error for invalid password") so the LLM can see exactly what's already tested and avoid duplicating coverage.

Also

  • Added **/*.po.ts to default POM auto-detection patterns
  • project_context_budget now controls how much source code is included (increase for large projects with many POMs)

Recommended configuration

- uses: autospec-ai/action@v2
  with:
    llm_api_key: ${{ secrets.OPENAI_API_KEY }}
    test_directory: 'e2e/tests'
    pom_patterns: '**/*.po.ts,**/pages/**/*.ts'
    pom_output_directory: 'e2e/pages'
    project_context_budget: '12000'