Skip to content

test: add core ingest coverage with httptest server#16

Merged
acoshift merged 1 commit into
mainfrom
test/ingest-core-coverage
May 25, 2026
Merged

test: add core ingest coverage with httptest server#16
acoshift merged 1 commit into
mainfrom
test/ingest-core-coverage

Conversation

@acoshift
Copy link
Copy Markdown
Member

Summary

Adds dedicated tests for the core ingest feature, exercised against an httptest server so no real Quickwit instance is required. Complements the existing regression tests by covering the happy-path mechanics and operational behaviors of the buffered ingest pipeline.

New file ingest_test.go (black-box quickwit_test package, matching existing style):

Test What it verifies
TestIngest_DeliversAllItemsInOrder All records reach the server exactly once, in order, across multiple batches
TestIngest_FlushTriggeredByBatchSize Reaching the batch size flushes immediately (timer pinned far out)
TestIngest_FlushTriggeredByTimer A partial batch flushes on the timer (batch size never reached)
TestIngest_FlushRemainingOnClose Close() flushes whatever is still buffered
TestIngest_RetriesFailedFlushUntilSuccess A failing flush retries until the server accepts it; no data loss
TestIngest_AppliesAuthHook The user auth hook is applied to ingest requests
TestIngest_ConcurrentWorkersDeliverEachItemOnce With multiple workers every record is delivered exactly once (no loss/dup)
TestIngest_DiscardsWhenBufferFull In discard mode, overflow items are dropped via OnDiscard instead of blocking
TestIngest_413ReducesBatchSizeAndChunks A 413 shrinks the batch and re-sends in smaller chunks, still in order

Test plan

  • go vet ./...
  • go test -race -count=5 ./... — passes (timing-sensitive tests stable across runs)

🤖 Generated with Claude Code

Cover the ingest path end-to-end against an httptest server (no real
Quickwit needed): full delivery without loss/duplication, both flush
triggers (batch size and timer), flush-on-Close, retry-until-success,
auth-hook application, concurrent-worker delivery, backpressure discard
when the buffer is full, and 413-driven batch-size reduction with
in-order chunked re-delivery.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@acoshift acoshift merged commit e391ee2 into main May 25, 2026
1 check passed
@acoshift acoshift deleted the test/ingest-core-coverage branch May 25, 2026 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant