Fork network integration + LMDB growth fix (1GB map, 7-day retention) - #5
Closed
kahlos wants to merge 10 commits into
Closed
Fork network integration + LMDB growth fix (1GB map, 7-day retention)#5kahlos wants to merge 10 commits into
kahlos wants to merge 10 commits into
Conversation
… scripts - config.py: default_model, validate_session_on_init, allow_private_url_fetch, url_fetch_timeout; hot yaml reload; load_cached_1psidts - chat.py: default_model fallback; deep_research report return over HTTP (turns-RPC fallback when plan.research_id is missing), responses mirror - client.py: validate_session probe; fetch_deep_research_report + crawl - pool.py: _init_with_fallback with cached 1PSIDTS retry; boot probe - helper.py: reject_unsafe_url SSRF guard, fetch timeout/cap/HTML rejection - start-gemini-api.sh: start/stop/status launcher - .gitignore: config/config.yaml (local cookies), .DS_Store Verified 2026-08-12/13: AC1 unit 9/9, AC3 suites 35/35 + recovery, ruff/format/pyright 0/0/0, AC2 live DR run HTTP 200 (22,161-char report).
# Conflicts: # app/server/chat.py # app/utils/config.py # config/config.yaml
…+ nanobot-ops port)
…fault file strategy)
…tion inside) Resolved 5 conflict sites: _process_conversation_with_timeout now accepts allow_summary_compaction/reason and delegates to the C7 compaction wrapper, keeping C4's TimeoutError->503 handling and C7's per-path gating.
- storage.max_size default 256MB -> 1GB (code + example config): LMDB map is fixed at open; at the 256MB ceiling every write failed with MDB_MAP_FULL and session reuse silently stopped persisting. - retention_days default 14 -> 7 so the 6-hourly cleanup reclaims space sooner at current growth rates. - README: document max_size / retention_days env overrides and the MDB_MAP_FULL behavior (restart required after raising max_size). - Live deployment config/config.yaml (gitignored: credentials) updated to match and server restarted; verified store write/find/read/delete against the reopened 1GB environment.
Owner
|
@kahlos Thanks for contributing this PR. There are a few points I’m considering:
|
Author
|
@luuquangvu Thanks for the detailed review — all six points are fair, and we've reconciled the integration accordingly (then integrated your 71c0db2/c8e0a83 into our fork).
Also fixed a latent bug found while integrating: Closing #5 as superseded — the reconciled survivors (unobjected only) are in #6. |
Author
|
Closed as superseded by #6 (reconciled survivors). |
luuquangvu
pushed a commit
that referenced
this pull request
Aug 15, 2026
…ion recovery, script restore (#6) * Survivors from reconciled fork (PR #5 follow-up): C1 v1beta, SSRF guard, truncation recovery, model-resolution fix, start script vs luuquangvu/main: - app/server/gemini.py + app/models/gemini_models.py + app/models/__init__.py + app/main.py + app/server/middleware.py: native Gemini /v1beta REST surface (models, generateContent, streamGenerateContent, x-goog-api-key auth, Google error envelopes) - unobjected in review - app/utils/helper.py + app/utils/config.py + config/config.yaml: SSRF guard (reject_unsafe_url, allow_private_url_fetch, url_fetch_timeout); recovery_timeout knob - app/services/client.py + app/server/chat.py: truncated-stream recovery via LIST_CONVERSATION_TURNS polling; fix chat.py:688 calling nonexistent client.resolve_model (lib: _resolve_model_by_name) - scripts/start-gemini-api.sh: restore local start/stop helper (deleted in 71c0db2 with no replacement) - app/services/pool.py: extract _init_one/_init_attempt (behavior-preserving) * Drop unused video-generation helper from PR surface (dead against luuquangvu models) * Remove VideoGeneration import (dead on PR surface) * Fix CI: move MIME registry below imports and use contextlib.suppress Ruff E402/SIM105 in app/main.py (self-introduced by the survivors main.py edit): the mimetypes loop ran before module imports and used try/except-pass. Imports first, loop after (still at import time, before any upload), no noqa needed. All 4 CI steps now pass: ruff check, ruff format, ty, pyright. * Update the PR to align with the current flow: - removing _recover_stream_output, as it’s now handled upstream. - removing fetch_last_model_turn since it’s already included upstream. - eliminating the unnecessary looks_like_html function.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the merged fork-network work (C1/C3/C4/C7 ports, composed preprocess wrapper) together with a fix for the LMDB storage growth issue.
LMDB fix (f0232cd)
storage.max_sizedefault raised 256MB → 1GB. LMDB's map is fixed at open; at the 256MB ceiling every write fails withMDB_MAP_FULLand session-reuse persistence silently stops (non-fatal: HTTP still 200, warn + continue).retention_daysdefault 14 → 7 so the 6-hourly cleanup reclaims space sooner.CONFIG_STORAGE__MAX_SIZE/CONFIG_STORAGE__RETENTION_DAYSoverrides and the restart-after-raise requirement.Fork network integration (cf44679 and earlier merges)
filestrategy, behavior-preserving)No force-pushes; branch is a fast-forward of
luuquangvu/main(10 commits ahead, 0 behind).