Skip to content

Testing

Jump to bottom
Mohammad Hasan Kaman edited this page Oct 21, 2026 · 1 revision

Testing

The test suite is fully offline. It runs with no network, no API key, and no provider account — fake engines stand in for the real ones, and every file it touches goes into a temporary directory.

python tests/test_v3.py
# expected: ALL V3 TESTS PASSED  (302 checks)

302 checks. Zero failures. That is the measured number, not a rounded-up one.

Why the suite can run without a key

Two deliberate design decisions make this possible:

1 — The core is Qt-free. kaman_core.py imports no GUI toolkit. It contains the configuration and DPAPI encryption, the prompts, the chunking, the structure analysis, the engines, the job store, and the exporters. Everything worth testing lives here, and it can be imported headlessly.

2 — The engine is injectable. make_engine(cfg) is the single factory that builds a transport. The tests replace it with a stub:

t3.make_engine = lambda cfg: stub_batch

Everything downstream of the factory — the batch worker, the prompts, the chunking, the job store, the UI state machine — then runs against the stub with no network traffic at all.

The temporary profile

At the top of the suite, before anything else is imported:

TEST_HOME = tempfile.mkdtemp(prefix="kaman_v3_tests_")
os.environ["APPDATA"] = TEST_HOME

APPDATA is redirected before the app modules are imported, so config_dir() resolves to the temporary folder and the real user profile is never touched. No test can read or write the actual config.json, the real job files, or the real API key.

What the sections cover

The suite is organised into lettered sections, each printing a header as it runs.

Section Area Checks
A API key encryption and configuration 11
B Configuration validation 5
C Chunking and rolling context (regression) 4
D Bidirectional prompts and QC 5
E Book structure analysis (template) 4
F translate_page: chunking + context + scanned page 13
3b Raw JSON is never displayed (custom-engine bug) 9
G Automatic review (QC loop) 4
H JobStore (resume and page storage) 7
I Exports (TXT / Markdown / HTML) 6
J OpenAI-compatible engine (simulated local server) 14
K UI: window, file, saved view, review, source toggle 33
L Settings dialog (engine, encryption, theme) 7
M Whole-book dialog + guards 9
N Whole-book page queue via the UI (stub engine) 30
O Cancel and quota in BatchWorker (unit) 6
P Single-page translation via the UI (stub) 5
Q Book structure: three settings switches + template editor + checkboxes 45
R Page-reading accuracy + translation quality + warning cleanup 25
S Percentage bar on the right + header without page count 8
T Pause/cancel buttons for whole-book translation (wiring) 7
U Live page following + free navigation + settings while paused 16
V Copy button + live page number + start from the beginning + editor help 19
W Revealing the local data-storage location in the app 5
X Fixing the silent swallowing in encrypt_secret (code feedback) 4
Total 302

The three largest sections — K (33), N (30) and Q (45) — are the interface ones, and they are the ones that assert geometry: that the source panel lands on the right in RTL, that the percentage bar is right-aligned, that the copy button is blue only when there is something to copy, that the pause and cancel buttons are enabled exactly while a batch is running.

The stub engine

class StubEngine:
    """موتور جعلی: پاسخ JSON فارسی برمی‌گرداند؛ QC را هم شبیه‌سازی می‌کند."""

    provider = "stub"

    def __init__(self, delay=0.0, quota_at=None):
        self.calls = []
        self.qc_calls = 0
        self.delay = delay
        self.quota_at = quota_at

It records every call — the prompt, the system prompt, and whether an image was attached — so a test can assert what the app actually sent, not just that it finished. delay makes the worker slow enough to test pause and cancel, and quota_at raises QuotaExhaustedError on the Nth call to test the quota path.

Because the stub returns a valid Persian JSON payload, the whole downstream pipeline — JSON parsing, chunk assembly, context rolling, the QC loop — is exercised for real. Only the network is missing.

The simulated local server

Section J goes one level deeper for the OpenAI-compatible transport. It stands up a simulated local server in-process and points the engine at it, exercising the actual HTTP request construction, the JSON shape of the request body, and the response handling — still with no real socket traffic off the machine.

The event pump

Qt tests need the event loop to turn, so the suite has one helper:

def pump_until(cond, timeout=40, app=None):
    app = app or QApplication.instance()
    end = time.time() + timeout
    while time.time() < end:
        app.processEvents()
        if cond():
            return True
        time.sleep(0.03)
    return False

It spins the Qt event loop until a condition becomes true or the timeout expires. Every asynchronous assertion in the suite — the batch finishing, the worker stopping, the quota pause firing — is written against it.

One caveat, stated honestly: a very small number of the asynchronous checks are timing-sensitive, and a heavily loaded machine can occasionally produce a spurious failure in the cancel path (the signal may not have been delivered by the time the condition is polled). Re-running the suite clears it. It is a harness race, not an application defect — a clean run is 302/302.

The check helper

def check(name, cond, extra=""):
    if cond:
        print(f"  PASS  {name}")
    else:
        failures.append(name)
        print(f"  FAIL  {name}  {extra}")

Every check prints its name. A failing check prints the extra diagnostic context that was passed with it — the actual value where the expected value did not match. The suite ends with:

if failures:
    print(f"FAILED: {len(failures)} -> {failures}")
    sys.exit(1)
print("ALL V3 TESTS PASSED")

and exits non-zero on any failure, so it works as a CI gate.

Running it

cd kaman-ai-translator
python tests/test_v3.py

The suite needs the GUI dependencies to be importable, because it instantiates real widgets:

pip install -r requirements.txt

requirements.txt covers PyQt6, Pillow, google-genai and pymupdf — and the pymupdf and Pillow imports are what let the suite build real PDF fixtures and real page images rather than mocking them.

Related pages

Clone this wiki locally