test: align corpus_search/rag tests with tests/README taxonomy#96
Merged
Conversation
Move the 11 tests from tests/unit/corpus_search/ into the rag tree to mirror the source layout under src/fireflyframework_agentic/rag/: - corpus, query_path, cli, benchmark_smoke → tests/unit/rag/ - expander, hybrid, reranker, answerer → tests/unit/rag/retrieval/ - pipeline, retry, ingest_with_real_vector → tests/unit/rag/ingest/ Drop the now-empty tests/unit/corpus_search/ package and update the benchmark README path reference.
Move stray tests under tests/unit/ to mirror src/fireflyframework_agentic
modules:
- tests/rag/ingest/test_ledger.py → tests/unit/rag/ingest/.
- tests/content/loaders/{test_markitdown.py,fixtures/} → tests/unit/content/loaders/.
- Drop tests/unit/ingestion/ (no matching source module).
Top-level tests/ now contains only the canonical categories
(unit, integration, functional, security, performance,
data_validation, responsible_ai, examples).
Per tests/README purpose-based taxonomy: - test_ingest_with_real_vectorstore.py → integration/ (drives real SqliteVecVectorStore + loader + chunker + ledger + ingest_one; docstring already labels it 'Integration test'). - test_benchmark_smoke.py → integration/ (multi-component end-to-end smoke of the ingest+retrieval pipeline; not a pytest-benchmark, so it does not belong in performance/). - test_cli.py, test_query_path.py, test_answerer.py → examples/corpus_search/ (these import from examples.corpus_search.* and exercise the example agent's CLI / facade / answerer, not the framework rag/ module). unit/rag/ is now framework-only: corpus, retrieval (expander, hybrid, reranker), and ingest (ledger, pipeline, retry).
- test_benchmark_smoke.py: mark the end-to-end mechanics-mode benchmark
as nightly (the corpus/queries sync check stays in PR-gate).
- test_ingest_with_real_vectorstore.py: mark as nightly — drives real
SqliteVecVectorStore + MarkitdownLoader (heavy deps).
- test_e2e_real_llm.py: move pytestmark file-level marker onto the
function per tests/README convention ('functions or classes only,
never via pytestmark at file level').
Per the rule that tests under tests/integration/ may only import the framework, relocate to tests/examples/corpus_search/: - test_e2e_real_llm.py — imports examples.corpus_search.agent. - test_benchmark_smoke.py — imports tests.examples.corpus_search.benchmark.runner. tests/integration/ now contains only framework-only integration tests (test_full_integration.py, test_ingest_with_real_vectorstore.py).
ancongui
pushed a commit
that referenced
this pull request
May 31, 2026
test: align corpus_search/rag tests with tests/README taxonomy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reorganize the corpus_search / rag tests to follow the
tests/READMEtaxonomy and the rule thattests/unit/<module>mirrorssrc/fireflyframework_agentic/<module>.Changes (5 commits)
test(rag)— distribute the 11 files intests/unit/corpus_search/undertests/unit/rag/(root +retrieval/+ingest/), mirroring the source layout.git mvpreserves history.test— aligntests/tree with the canonical module layout: movetests/rag/ingest/test_ledger.pyandtests/content/loaders/*intotests/unit/<module>/; drop the emptytests/unit/ingestion/(no matching source module).test— reclassify by purpose:test_ingest_with_real_vectorstore.pyand the benchmark smoke intotests/integration/(multi-component, real DB shape).test_cli.py,test_query_path.py,test_answerer.pyintotests/examples/corpus_search/(they exercise example code, not framework).test— apply@pytest.mark.nightlyto slow integration tests; convert the file-levelpytestmarkintest_e2e_real_llm.pyto per-function markers as the README requires.test— move example-importing tests out oftests/integration/. Nowtests/integration/only importsfireflyframework_agentic.*; anything that importsexamples.*lives undertests/examples/corpus_search/.Final shape
tests/unit/: one folder per src module (agents,content,rag, …);unit/rag/contains framework-only tests forcorpus,retrieval/,ingest/.tests/integration/: framework-only multi-component tests (test_full_integration.py,test_ingest_with_real_vectorstore.py).tests/examples/corpus_search/: anything that touches example code (test_agent,test_cli,test_query_path,test_answerer,test_benchmark_smoke,test_e2e_real_llm).Test plan
pytest --collect-onlysucceeds on the new tree.pytest -m "not nightly") green.pytest -m nightly) still picks up the marked tests.