Releases: agriffi10/log-forge
Release list
v1.0.0
The first release under semantic versioning. The public API — everything in
log_foundry.__all__, the Sink protocol, and every shipped sink class — is frozen for the whole
of 1.x: nothing is removed or renamed, and no signature changes in a way that breaks a caller
until 2.0.0. Behaviour is not frozen by that promise; a defect is still a defect, and fixing one
can change what a broken path does.
1.0.0 is not a feature release. It is a reliability release. Three audit arcs — SPEC-024..033,
SPEC-034..047 and SPEC-048..050 — worked through the paths where the library could lose an event,
fail its caller, block forever, or report health it could not back up, and SPEC-051 through
SPEC-055 closed out what those arcs left open before the tag. Each item below was reproduced by
running it before it was fixed. That is a description of the work done, not a claim that no such
path remains: the failure modes are the part that has been audited hardest, and the ones still
open are tracked in docs/architecture.md §12.
Signatures changed very little; behaviour changed a lot. Read Upgrading from 0.10.x
below before you bump — several of the changes are silent.
What the arcs fixed
- An HTTP redirect lost the batch and forwarded your credentials.
urlopen's default opener
follows a 301/302/303 on a POST by rewriting it as a body-less GET and keeping every header.
So anhttp://collector behind a load balancer that redirects tohttps://lost every batch it
was ever sent, sent theAuthorizationheader to a host you never configured, and read the
redirect target's200as a successful delivery — every counter clean. It reached every
HTTP-family sink and Sentry's HTTP fallback. A 3xx is now refused, counted and announced.
If you ship to an HTTP endpoint over a redirecting hop, treat any bearer token in that
configuration as disclosed to the redirect target and rotate it. - The AWS batch sinks duplicated events on a partial failure.
SQSSink,SNSSink,
KinesisSinkandFirehoseSinkcalled their client unguarded inside a chunk loop, so a fault on
chunk N escapedemitafter chunks 1..N-1 had landed and the worker's retry re-sent the whole
batch. The exit drain is one large batch by construction, so it is exactly this shape. A client
exception now costs its own chunk and nothing else. - Per-request context leaked between requests. Baggage and an adopted trace context were
written intocontextvarsand never taken back out, so on a thread serving requests
sequentially one request'suser_idreached the next request's events, and a warm container
kept joining a trace whose process had exited. Both are released when the root span closes —
baggage restored, an adopted context cleared, because an inbound context is a one-shot
handoff rather than a process default. flush()did not reach an open span, and the README's own serverless recipe was the case
that broke. Callingflush()inside a@tracefunction delivered zero of two events with
every counter clean. A flush now sweeps open spans and hands their buffers to the worker.- Sinks that lost everything reported success. A dead syslog socket produced a truthy
flush()and an all-zerohealth()while every message was lost. A sink that can prove it
delivered nothing now raises, so the retry engages and the loss is counted. Three cases stay
quiet deliberately — an unadjudicable batch response, a rejected SQS sender fault, and an
oversized event — because a retry there duplicates rather than recovers. - A sink's backoff paused all delivery, including through
shutdown().HTTPSinkpassed a
server-suppliedRetry-Afterstraight totime.sleep; measured,Retry-After: 8at the
defaultmax_retries=3blockedshutdown()for 22.01 s, and86400would have stalled
logging for a day. Every retry backoff is now bounded, clamped, and cut short by a shutdown —
while only ever shortening a wait, never skipping work. Read that as scoped to backoff
rather than to every wait:Sink.close()takes no timeout and is still unbounded on both
delivery paths, whichdocs/architecture.md§13 Known constraints records as accepted. The
backoff case that is simply still open —NATSSink's exit drain — is in §12 Open items,
which names what would close it, alongside two further open items about aclose()bound that
§13 accepts in general. Its connect loop is no longer one of them:
NATSSink(max_reconnect_attempts=0)raisesValueErrorbefore it connects at all. It blocked
the constructor indefinitely onmainbetween the spec that added the keyword and the one that
refused it, and never in a released version — the keyword does not exist in0.10.1, so on
this release's own baseline there is nothing to upgrade from. The refusal itself is listed
with the other construction-time refusals below; what is here is only the history. - The exit drain overwhelmed the whole HTTP family.
Worker._final_drainhands a sink the
entire exit backlog — 5,980 events, measured — and no sink in that family chunked it: five
overrodeemitand the sixth inherited a baseemitthat sent the batch in one request the
destination rejected whole.HTTPSinkis now a template method that owns the chunk loop, and
subclasses extend it through_render/_body/_handle_responseinstead. - Concurrent emitters could corrupt a sink. A level call with no active span emits on your
thread, against the same sink object the worker is draining into. Unlocked,SQLiteSinkdid not
merely lose rows — it killed the interpreter with a bus error. - Closed sinks silently accepted work.
KafkaSinkproduced into a batch nothing would flush,
GooglePubSubSinkappended a future nothing would resolve, and the Redis sinks succeeded by
reconnecting a client they had just disconnected. os.fork()was unhandled anywhere. A forked child inherited a worker whose drain thread does
not exist (six events never delivered,health()clean on every documented alert term) and locks
held by threads that do not exist (19 of 60 children hung permanently insideinfo(), on the
application's own thread). Both are repaired, in the child only.- The sink every event was going to could be closed by nobody. The record of which sinks were
owed a close was a single slot, so arming a second discarded the first. Measured with every call
sequential on one thread: the live sink'sclose()never ran and its buffer never delivered.
Then, once that record became a set,shutdown()cost one slow close times the number owed —
2.00 s for one, 8.02 s for four. Both are fixed; 200 owed sinks went 11.4 s → 0.06 s. - Diagnostics could leak your data. Twelve of the library's stderr sites printed
repr(exception), and a psycopg repr reprints the statement and its bound parameters. Every
line the library writes about itself now names an exception by type only.
Upgrading from 0.10.x
Ordered by how likely each is to reach you, not by how loudly it lands — items 18 to 22 are the
quietest in the list and are last. Read to the end.
1. Health is a frozen dataclass, not a NamedTuple. len(h), h[0] and
queued, dropped, failed_batches, stopped_reason = health() — the four-way unpack that worked
against 0.10.x — now raise TypeError. Read it by attribute. The same
applies to SinkLosses. This is the change most likely to break an upgrade, and it was made
before 1.0 precisely so the eight fields you are gaining — none of them present in 0.10.x,
which had the same four as v0.7.0 — sink, retired,
submitted_after_shutdown, incomplete_swaps, closing_sinks, inherited_sink, orphan_lost,
in_span_lost — could be plain appends with no position to preserve.
2. Config, ContinueResult, FlushResult, Health and SinkLosses are all keyword-only
dataclasses. Positional construction now raises TypeError. Item 1 already covers unpacking a
Health; this is the other half, and it was done before 1.0 precisely so field order never
becomes part of the frozen contract. It reaches structural pattern matching too, which is the half
with no exception to read at the call site you wrote: kw_only empties __match_args__, so case Health(a, b): no longer matches a Health — it raises TypeError: Health() accepts 0 positional sub-patterns (2 given) from the match statement itself. A keyword pattern, case Health(queued=q):, still matches. If you construct SinkLosses inside your own sink's
losses(), If you wrote your own sink below already tells you to use keywords — that is the
same rule stated once, not twice. Three smaller changes ride the same spec: context.__all__ is
trimmed to six names — current_baggage_header, current_trace_context, current_traceparent,
get_baggage, reset_context, set_baggage — with current_span, push_span and pop_span
withdrawn from the list against your 0.10.x baseline but still importable, so only from log_foundry.context import * loses anything, and it gains reset_context. trace(defaults=…)
now takes its copy at decoration rather than reading your mapping live per event, so mutating it
afterwards no longer reaches later spans, and the parameter widened from dict to Mapping so a
typed dict[str, str] is accepted either way. And the seven HTTP platform sinks (DatadogSink,
LokiSink and their siblings) now type their keyword arguments through Unpack[TypedDict], so
DatadogSink("k", timeout="not-a-float") is a mypy error instead of a first-request failure — a
typed consumer's build can newly fail here; that is the point. Additive and safe:
GroupIdSource/DedupIdSource (SQSSink's message_group_id/message_deduplication_id) and
Backend (SentrySink's backend) are now exported from their sink modules, and `flush_...
v0.10.1
What's Changed
- fix(SPEC-023): attach the SBOM before the release becomes immutable by @agriffi10 in #91
Full Changelog: v0.10.0...v0.10.1