Skip to content

bug(ingest): unsafe/empty table-name rejections silently drop instead of routing to DLQ #151

Description

@EricAndrechek

Problem

internal/ingest/bento.go:108:

// Validate table name to prevent SQL injection.
if !safeIdentifierRe.MatchString(raw.TableName) {
    slog.WarnContext(msgCtx, "rejecting message with unsafe table name", "table", raw.TableName)
    // TODO: manually push to a DLQ subject with metadata for later analysis instead of silently dropping?
    if doubleAckErr := m.DoubleAck(msgCtx); doubleAckErr != nil {
        slog.WarnContext(msgCtx, "double ack failed for dropped message", "error", doubleAckErr)
    }
    continue
}

When the table name fails the safeIdentifierRe check, the message is logged at WARN and DoubleAck'd — i.e. permanently removed from JetStream — with no DLQ entry and no payload retained anywhere. Per AGENTS.md design decision #6 the DLQ exists precisely "to prevent silent data loss." This path violates that invariant.

The adjacent empty-table_name branch (bento.go:97-103) has the same shape and the same gap.

Proposed Solution

Route both rejection branches through the existing DLQ output (stream WAVEHOUSE_DLQ, subject dlq.<table> — falling back to dlq.__rejected__ when the table name itself is the reason for rejection). Include the rejection reason in headers (x-wh-drop-reason: unsafe-table-name / empty-table-name) so operators can grep the DLQ for these specifically.

Acceptance criteria

  • No code path in bento.go silently drops a message — every drop either lands in the DLQ or surfaces with full payload at ERROR.
  • Tests cover both the unsafe-table-name and empty-table-name DLQ paths.
  • AGENTS.md design decision ci: bump goreleaser/goreleaser-action from 6 to 7 #6 wording matches the new behavior (or stays as-is if the new behavior matches the stated invariant).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/ingestIngest pipeline (Bento, batching, DLQ)bugSomething isn't workingdocumentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions