Skip to content

Stale test: LegalTextValidator constructed without required 'tech' arg (hidden by Azure-key skip) #467

@rajeee

Description

@rajeee

Summary

Two tests construct LegalTextValidator without the required tech argument, so they raise TypeError: LegalTextValidator.__init__() missing 1 required positional argument: 'tech' whenever they actually run:

The required tech parameter is defined here: LegalTextValidator.__init__.

# test_validation_content.py:100
legal_text_validator = LegalTextValidator(
    llm_service=oai_llm_service, temperature=0, seed=42, timeout=30
)
# content.py:318 — tech is a required positional arg
def __init__(self, tech, *args, score_threshold=None, doc_is_from_ocr=False, **kwargs):

The constructor is the first statement in each test, so it fails immediately (before any document load or LLM call). The @flaky(max_runs=3) decorator retries each failing case 3x — all three fail fast with the same TypeError, but each retry emits a verbose traceback report, producing a lot of log noise for a deterministic failure.

Why CI never caught it

Both tests are gated by @pytest.mark.skipif(SHOULD_SKIP, reason="requires Azure OpenAI key"), where SHOULD_SKIP is true when AZURE_OPENAI_API_KEY is unset. CI has no Azure key, so the tests are skipped and the broken constructor calls are never exercised. They only fail for local runs that have Azure credentials configured.

When it broke

tech became a required positional arg of LegalTextValidator.__init__ around #308; the tests' constructor calls were never updated. #447 later reformatted the signature but tech was already required.

Suggested fix

Pass a tech value matching the heuristic used in the tests (they use WindHeuristic()):

LegalTextValidator(tech="wind", llm_service=oai_llm_service, temperature=0, seed=42, timeout=30)

Broader note

Credential-skipped tests can hide signature/API drift indefinitely. Worth considering a lightweight mocked-construction smoke test that runs in CI (no key needed) to catch this class of break.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions