feat: Flash with full optimization becomes the default local indexing mode - #404
Merged
Conversation
Move asyncio.run(coro) out of the except RuntimeError block so real errors no longer carry a bogus "no running event loop" context in their traceback.
… mode
Every entrance now defaults to Flash with the full optimize pass
(deterministic merge, then LLM expand), replacing the standard LLM-built
tree as the default:
- submit_document(): mode=None now means "flash"; pass mode="standard"
for the LLM-built tree. _index_flash runs optimize="full" with the
expand model = summary_model, and fails fast with the missing key
name(s) via litellm.validate_environment before any work.
- page_index_flash(): optimize takes "full" (default) / "merge" / False;
True is accepted as "full" for compatibility, unknown values raise
instead of silently degrading to merge-only. optimize_expand stays
honored for legacy callers.
- CLI: --mode {flash,standard} replaces --flash (kept as a hidden
compatibility alias that forces flash). --optimize defaults to full in
flash mode with an `off` choice; explicitly passing it outside flash
still errors. Standard-only tuning flags (--toc-check-pages,
--max-*-per-node, --if-add-*) now error in flash mode instead of being
silently ignored, mirroring the existing flash-only flag errors. The
key pre-check runs only when an LLM will actually be called, so
--no-summary --optimize off|merge works keyless. Output drops the
_structure_flash suffix — always <name>_structure.json.
On the Disney earnings PDF the optimized default is also faster than
unoptimized flash (fewer nodes to summarize) and fixes hierarchy
mistakes; both modes emit identical schemas end to end.
Docs updated to match (mode flag, defaults, LLM usage honesty); tests
pin the new defaults: stored mode == "flash", optimize passthrough, and
the unknown-optimize rejection.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports the two pending commits from
feat/local-chat(the v0.2.10 line) tomain, following the #402 pattern. After this merge,git diff main feat/local-chatis empty — main and the 0.2.10 branch are content-identical again (verified on the branch before opening this PR).Flash becomes the default local indexing mode, with full tree optimization
Every entrance now defaults to Flash with the full optimize pass (deterministic merge, then LLM expand), replacing the standard LLM-built tree as the default:
submit_document():mode=Nonenow means"flash"; passmode="standard"for the LLM-built tree._index_flashrunsoptimize="full"with the expand model =summary_model, and fails fast with the missing key name(s) vialitellm.validate_environmentbefore any work.page_index_flash():optimizetakes"full"(default) /"merge"/False;Trueis accepted as"full"for compatibility, unknown values raise instead of silently degrading to merge-only.optimize_expandstays honored for legacy callers.--mode {flash,standard}replaces--flash(kept as a hidden compatibility alias that forces flash).--optimizedefaults to full in flash mode with anoffchoice; explicitly passing it outside flash still errors. Standard-only tuning flags (--toc-check-pages,--max-*-per-node,--if-add-*) now error in flash mode instead of being silently ignored, mirroring the existing flash-only flag errors. The key pre-check runs only when an LLM will actually be called, so--no-summary --optimize off|mergeworks keyless. Output drops the_structure_flashsuffix — always<name>_structure.json.On the Disney earnings PDF the optimized default is also faster than unoptimized flash (fewer nodes to summarize) and fixes hierarchy mistakes; both modes emit identical schemas end to end. Docs updated to match; regression tests pin the new defaults (stored
mode == "flash", optimize passthrough, unknown-optimize rejection)._run_syncphantom-chain fixThe one commit #402's port missed (it was committed 20 minutes after #402 merged): moves
asyncio.runout of theexcept RuntimeErrorprobe inlocal_chat.py, so exceptions raised by the user's coroutine stop carrying the phantom "no running event loop" context in their tracebacks.Verification
269 tests green (3 skipped without the claude extra) on the ported branch; the flash-default work itself was verified with live end-to-end runs (real SDK submit → tree → agent tools; CLI keyless/keyed matrices) before landing on
feat/local-chat, where #400 carries it for review.