fix(connectors): bring quickwit_sink up to convention

[mfyuce]

Summary

• Defer index_id YAML parse to open() — return InvalidConfigValue instead of panicking via expect()
• Use ClientWithMiddleware (build_retry_client) for ingest retries with exponential backoff on 5xx / connection errors
• check_connectivity_with_retry in open() against /health/livez
• Map 4xx responses (except 429) to PermanentHttpError so the circuit breaker is not tripped by bad payloads
• Add verbose_logging / max_retries / retry_delay / max_retry_delay / max_open_retries / open_retry_max_delay to QuickwitSinkConfig (all optional, forward-compatible)
• Downgrade per-batch consume() log to debug! (upgraded to info! when verbose_logging = true)
• Set Content-Type: application/x-ndjson on ingest requests
• Drop unused dashmap / once_cell deps; add reqwest-middleware
• 5 unit tests covering verbose flag, client init state, and index_id init state

Test plan

• cargo clippy -p iggy_connector_quickwit_sink --all-targets -- -D warnings — clean
• cargo test -p iggy_connector_quickwit_sink — 5 tests pass
• Integration tests: cargo test -p integration -- connectors::quickwit

🤖 Generated with Claude Code

https://claude.ai/code/session_01QBxbPbdKXzoMdvLBeugNBX

[github-actions]

Thanks for the PR. It is labeled S-waiting-on-review and queued for review.

Slash commands (own line, regular comment) move it around the queue:

• /ready - back to S-waiting-on-review after addressing feedback
• /author - flip to S-waiting-on-author while you finish changes
• /request-review @user-or-team - request a reviewer

See CONTRIBUTING.md for details.

[ryerraguntla]

lib.rs:ingest() — create_index() treats 409 Conflict as InitError; concurrent open() calls (multi-instance, restart race) both see has_index()=false, both POST, second gets 4xx → InitError → connector
never opens. Fix: absorb 409 (and 400 "already exists") as Ok(()) in create_index(). Retry middleware also retries a 5xx create that succeeded server-side; on retry, server returns 409 → same path. Same fix
covers both.

[ryerraguntla]

reqwest::Client::new() has no timeout; health probe and has_index()/create_index() can hang indefinitely under network partition or slow-starting Quickwit. Fix: reqwest::Client::builder().timeout(...) with configurable or sensible default (e.g. 30s)

[ryerraguntla]

@mfyuce - all the above are minors. I am not sure of quickwit s production data set distribution to mention about the need for circuit breakers (if there are huge number of records/documents for the same cursor key for a given batch size) . Please make a judgement call about the need for circuit breaker implementation . otherwise it is all set for second reviewer's comments before merging.

[ryerraguntla]

/author

[mfyuce]

Thanks for the thorough review @ryerraguntla! Both issues addressed in the latest push:

409 / "already exists" on create_index() — create_index() now absorbs 409 CONFLICT and 400 BAD_REQUEST whose body contains "already exists" as Ok(()) with an info! log. Covers the concurrent-open race and the retry-after-succeeded-5xx path.

No timeout on reqwest::Client — Added request_timeout: Option<String> to QuickwitSinkConfig (default 30s, configurable via TOML). The raw client is now built with Client::builder().timeout(request_timeout), bounding health probes, has_index(), and create_index() under network partition.

/ready

[mfyuce]

@ryerraguntla — judgment call on the circuit breaker:

The existing build_retry_client wraps a HttpRetryMiddleware that already retries 429 and 5xx with exponential backoff, and the PermanentHttpError mapping cuts retries for permanent 4xx errors (bad data won't hammer the backend). Ingest throughput is also naturally rate-bounded by batch_length and poll_interval.

QuickWit is typically an internal service in the same cluster, so prolonged partition is rare, and the runtime already isolates failures per connector. A full half-open / trip-threshold circuit breaker on top of this would add complexity without clear benefit for this specific sink. Happy to revisit if the second reviewer sees a concrete failure mode that the existing retry policy doesn't cover.