config(sidecar): bump SIDECAR_CACHE_MAX_ENTRIES 10k -> 100k (#13)

[vrogojin]

Summary

Closes #13. Bumps SIDECAR_CACHE_MAX_ENTRIES from 10_000 to 100_000 to relieve the entry-count saturation observed on unicity-ipfs1.dyndns.org (9,995/10,000 entries, 90 MB / 1 GiB byte budget, ~33,405 GET /sidecar/blob 404 / 6 h, 3.3 % hit rate).

Config tuning only -- cache semantics (LRU eviction of in-kubo, 503 back-pressure for pending-only saturation, 24 h promotion-timeout WARN) are unchanged.

Old / new value

	old	new
DEFAULT_MAX_ENTRIES (Python)	10_000	100_000
SIDECAR_CACHE_MAX_ENTRIES (Dockerfile ENV)	"10000"	"100000"
SIDECAR_CACHE_MAX_ENTRIES (docker-compose default)	10000	100000
README env-var table	10000	100000

Env-var override path (docker-compose substitution / run-ipfs.sh passthrough) is unchanged -- operators can still set a custom value.

Files changed

• nostr-pinner/instant_pin_cache.py -- DEFAULT_MAX_ENTRIES + docstring
• Dockerfile -- baked-in ENV default
• docker-compose.yml -- compose default
• README.md -- env-var documentation table

Memory / footprint estimate

The cache is SQLite-backed with on-disk blob files (not in-memory). Footprint at 100k entries:

Component	Per-entry	At 100k entries
Blob bytes on disk (avg observed)	~9 KB	~900 MB -- still bounded by the existing 1 GiB SIDECAR_CACHE_MAX_BYTES cap
SQLite row (instant_pin_cache)	< 200 B	~20 MB (sqlite file, OS page-cacheable)
Resident process memory	trivial (no in-RAM dict of entries)	trivial

The 1 GiB byte cap remains the safety net: if average blob size grows, the byte cap will start triggering eviction before 100k entries are reached, exactly as today.

Eviction policy summary (unchanged)

InstantPinCache._evict_until_fits (in instant_pin_cache.py):

1. If new submit would exceed either cap, evict LRU in-kubo rows until both caps are satisfied.
2. If only pending rows remain and the cap is still exceeded, return SubmitOutcome.CACHE_FULL -> HTTP 503 back-pressure. Pending blobs are never evicted (writer would silently lose durably-acknowledged bytes).
3. Pending rows older than SIDECAR_CACHE_PROMOTION_TIMEOUT (24 h) trigger operator WARN logs but are still not auto-aged-out.

Test results

nostr-pinner/tests/test_instant_pin_cache.py parameterizes caps via _build_cache(max_entries=N, max_bytes=N) -- no test asserts the literal 10000, so the bump is test-safe.

Sandbox limitation: I could not install pytest/pytest-asyncio/httpx in this environment (no pip, no venv, network-restricted), so the full suite was not executed locally. Validation done:

• AST-parsed instant_pin_cache.py -- syntax clean, DEFAULT_MAX_ENTRIES == 100000 confirmed.
• grep audit confirmed no other tests, fixtures, or docs depend on the old 10000 literal.

Please verify CI runs the test suite on this PR.

Verification (after deploy, per issue #13)

curl -s https://unicity-ipfs1.dyndns.org/sidecar/cache-stats | jq '{confirmed: .confirmed_entries, max: .max_entries, bytes_used: .bytes_used_total, bytes_max: .max_bytes, miss_rate: (.cache_misses / (.cache_hits + .cache_misses))}'

Expected: max_entries == 100000; miss_rate drops from ~0.97 toward the byte-budget limit.

[gemini-code-assist]

Code Review

This pull request increases the maximum cache entry limit (SIDECAR_CACHE_MAX_ENTRIES) from 10,000 to 100,000 across the Dockerfile, docker-compose, documentation, and Python codebase. Feedback highlights that this ten-fold increase could introduce performance and memory bottlenecks during cache eviction, as the current implementation fetches all candidates into memory and lacks a composite index to optimize the query. It is recommended to batch or limit the eviction query and add a composite index on (state, last_accessed_at).

[gemini-code-assist]

Bumping DEFAULT_MAX_ENTRIES to 100_000 introduces potential performance and memory bottlenecks in the cache eviction logic under load:

1. Memory Footprint in Eviction: In _make_room_for (lines 371-376), the query SELECT cid, byte_size FROM instant_pin_cache WHERE state = 'in-kubo' ORDER BY last_accessed_at ASC is executed, and cursor.fetchall() is used to load all candidates into memory. With up to 100,000 entries, this can fetch tens of thousands of rows into memory on the write path, causing latency spikes and high memory usage.
2. Database Query Performance: The query filters by state and orders by last_accessed_at. Without a composite index on (state, last_accessed_at), SQLite may perform a full table scan or filesort, which becomes significantly slower as the table grows to 100,000 rows.

Recommendations for future improvements:

• Batch Eviction / Limit Query: Limit the SELECT query in _make_room_for using a LIMIT clause (e.g., only fetch the oldest N entries to evict, or evict in batches).
• Composite Index: Add a composite index on (state, last_accessed_at) in init_instant_pin_cache_schema to optimize the eviction query.