Skip to content

HostLoom 0.12.0

Choose a tag to compare

@kidoz kidoz released this 27 Sep 13:45

The twelfth release of HostLoom. Four threads run through it, three of them in HostLoom.Logging. The first is output that a service moving from Serilog can keep. Its log queries were written against the records of Serilog's CompactJsonFormatter, and the provider differed in ways such a query notices: a collection in a plain hole such as {Ids} was written as its type name, a byte array was Base64 when destructured and its type name otherwise, the JSON writer trimmed trailing zeros from @t so a fixed-pattern timestamp parser rejected it, standard ILogger calls had no @r, destructured objects carried no "$type", and a caller's hole named SourceContext, ThreadId, or EventId lost to the formatter's own. Collections now keep their structure, byte arrays are uppercase hex with Serilog's summary beyond 1,024 bytes, @t carries all seven fractional digits, @r holds one invariant rendering per formatted token, the opt-in DestructuringOptions.TypeTags adds "$type", IncludeFields can leave public fields out as Serilog does, the caller's holes win, and string-keyed dictionaries and (string, value) tuples passed to BeginScope become fields. The same work closed paths that wrote protected values: destructuring read a property with a private getter such as public string Password { private get; set; }, walked a delegate's closure through Target, and rendered @r for {@Order:j} from the value's own ToString(), which printed members that [NotLogged] and [LogMasked] hide. It now reads public getters only, writes delegates and reflection objects as their names, and renders @r from the masked JSON, and a legacy LogReplaced attribute masks its member whole instead of being ignored.

The second thread is records that were lost and log calls that failed. A ToString() that threw in a plain hole, MEL's own formatter throwing, or a template with fewer arguments than holes escaped the log call, and LoggerFactory rethrew it to the caller; the value now reads "[DestructuringFailed]", what was captured before the failure is kept, and the failure is counted. With every thread-pool thread busy, a queued record never reached the sink and neither EnqueueTimeout nor ShutdownTimeout fired, measured with 100 ms bounds over 10 s; producers now wake the writer directly, a blocked caller waits on a monitor the writer signals, and the shutdown deadlines run on a thread of their own. One transient IOException from a sink used to fault the pipeline and drop every later record; it now costs the batch being written, counted as sink_failed. Environment.Exit, Main returning, and an unhandled exception ended the process around the background writer and lost up to a full queue, the last lines before a crash among them; the provider now flushes from AppDomain.ProcessExit and AppDomain.UnhandledException within ShutdownTimeout, and a crash handler registered after it can call the new HostLoomLoggerProvider.Flush. Since .NET 10 a SIGTERM to an application without the Generic Host raises no event at all, which the documentation now states. A stdout that nobody read stalled every thread that logged a warning for as long as it stalled, so EnqueueTimeout now defaults to one second. Records with thousands of fields, oversized field names, and LogFast values that refuse every buffer are bounded before they cost the caller or the writer; a nested destructuring failure replaces only the failing value; the built-in formatters are safe to share between providers; returned log entries no longer keep exception graphs alive; shutdown counts every abandoned record exactly once; concurrent disposals await one shutdown; and StreamLogSink releases an owned stream even when its final flush throws.

The third thread is new surface and Native AOT. AddHostLoomLogging accepts a Func<IServiceProvider, ILogSink>: each container built from the service collection calls it when it first resolves logging and gets a sink of its own, where a sink instance is shared by all of them, and a call site that creates its sink there no longer needs a CA2000 suppression. HostLoom.Logging is now annotated for trimming and Native AOT. Under Native AOT the configuration overloads failed at startup, because the reflection binder found no options and counted every key as unknown; the binding is now compiled. Destructuring wrote every type a native application had not preserved as an empty object, with no signal; DestructuringOptions.Preserve<T>() keeps a type's members, NotLogged<T> and Mask<T> now keep theirs, and a type still left without members is counted once under component=destructurer. A legacy LogMasked attribute whose options a native application cannot read masks its member whole instead of failing the whole object. examples/HostLoom.Examples.LoggingAot publishes natively, checks its own output, and runs in the release workflow next to the Valkey sample.

The fourth thread is metrics across the framework. Every histogram now advises its bucket boundaries: operation durations from 0.1 ms to 30 s, spans and lags from 1 ms to a day, and WebSocket queue bytes from 256 B to 16 MiB. An exporter's defaults, which end at 10 000, put every sub-millisecond cache hit into one bucket.

Upgrading from 0.11.0 changes behaviour an application can observe; each change is stated under Changed or Fixed. EnqueueTimeout defaults to one second, so a record that waits longer for room on a full queue is dropped and counted as enqueue_timeout; set it to null to wait without limit. A sink failure drops the batch being written as sink_failed and logging goes on, and writer_fault now means a defect in the writer itself. Log stores see new field shapes: a collection or dictionary in a plain hole keeps its structure, byte arrays and byte memory are uppercase hex, LogFast through an injected logger keeps numeric and boolean types instead of strings, and string-keyed dictionaries and (string, value) tuples passed to BeginScope become fields. Destructuring reads only public getters and writes delegates and reflection objects as their names. A custom ILogFormatter must allow Format to run concurrently, because the formatter passed to AddHostLoomLogging is shared by the provider of every container. A Native AOT application preserves the types it writes with {@...} through DestructuringOptions.Preserve<T>(). Exported histograms get new bucket boundaries. No package is published for the first time.

No package is published for the first time; the set is the same as 0.11.0. @hostloom/websocket-client is not part of this NuGet set; that package carries its own version and ships when a websocket-client-vX.Y.Z tag is published, and it has no changes since its 0.3.0 release.

Added

  • Every histogram advises its bucket boundaries: operation durations from 0.1 ms to 30 s, spans and lags from 1 ms to a day, and WebSocket queue bytes from 256 B to 16 MiB. An exporter's defaults, which end at 10 000, put every sub-millisecond cache hit into one bucket.
  • DestructuringOptions.TypeTags adds Serilog's "$type" member, the type's short name, to every destructured object except anonymous and other compiler-generated types. Off by default.
  • DestructuringOptions.IncludeFields, on by default, can leave public fields out of destructured objects, as Serilog does; value tuples are written as sequences of their items either way.
  • ClefLogFormatter writes @r for standard ILogger calls: one rendering per template token that has a format, in template order, rendered in the invariant culture the way Serilog's CompactJsonFormatter renders it. LogRecord.TemplateRenderings exposes them to other formatters.
  • AddHostLoomLogging overloads that take a Func<IServiceProvider, ILogSink>. Each container built from the service collection calls the factory when it first resolves logging and so gets a sink of its own, where a sink instance is shared by all of them. A call site that creates its sink in the factory no longer needs a CA2000 suppression.
  • HostLoomLoggerProvider.Flush(TimeSpan) waits until every record logged before the call is written and the sink flushed, for a crash handler registered after the provider: handlers run in registration order, so such a handler logs after the provider's own exit flush.
  • HostLoom.Logging is annotated for trimming and Native AOT, and examples/HostLoom.Examples.LoggingAot publishes natively and checks its own output. DestructuringOptions.Preserve<T>() keeps a type's members for {@...} in such an app, as NotLogged<T> and Mask<T> now do for theirs. Under Native AOT a destructured type without readable members is counted once under component=destructurer instead of passing unnoticed.

Changed

  • A collection or dictionary in an event hole without an operator ({Ids}) is captured with its structure, as Serilog captures it, instead of as its type name. An element that is neither a scalar nor a collection is written as its invariant ToString(), and the destructuring caps and record budget apply.
  • A byte array is written as Serilog's uppercase hex in every hole, summarized as its first 16 bytes and its length beyond 1024 bytes. A destructured byte array was Base64, and one in a plain hole was its type name.
  • EnqueueTimeout defaults to one second instead of no limit. A log call that waits for room on a full queue, under Block or for Warning and above under DropBelowWarning, drops its record as enqueue_timeout after a second instead of waiting for as long as the sink stalls; a stdout nobody read stalled every thread that logged a warning. Set it to null to wait without limit.
  • A sink failure costs the batch being written instead of stopping logging for the rest of the process. The batch's records are counted as dropped with the new reason sink_failed, and the writer goes on with the next batch, so a sink that recovers gets every record after the failure. One transient IOException used to fault the pipeline and drop every later record as writer_fault, which now means a defect in the writer itself. A failed flush is counted and costs nothing further.

Fixed

  • Field-name limits reject oversized names before allocating their encoded buffers or queueing them, including names prefixed by @ or $. Rejected fields keep their masked values for message and scope rendering. Wrapped logging respects each provider's configured name limit; caller-name references are released before enqueueing, and field drops remain counted.
  • Concurrent logger disposal calls await the same bounded drain and sink cleanup operation.
  • LogFast through a wrapped or injected logger preserves numeric and boolean field types, matching direct HostLoom logging. These fields previously travelled as strings; downstream mappings relying on that representation need to accept the corresponding JSON types.
  • The AddHostLoomLogging overloads that bind options from configuration work under Native AOT. The reflection binder found no options there, so every key counted as unknown and startup failed; the binding is now compiled.
  • A legacy LogMasked attribute whose options cannot be read, as under Native AOT, masks its member whole. The exception it raised made the whole object "[DestructuringFailed]".
  • LogFast bounds UTF-8 formatting buffers and retries, keeps canonical numeric fields intact even when a display format throws, and isolates throwing value formatters with a counted failure sentinel.
  • Nested destructuring failures preserve valid JSON. Unreadable memory replaces only its own value, preserving sibling members and collection items. Failed walks release their temporary object references.
  • Returned log entries clear exception graphs and event metadata before entering the pool.
  • Shutdown loss accounting includes records being formatted and excludes flush markers; late writer cleanup cannot count abandoned records twice.
  • StreamLogSink attempts to dispose owned streams even when its final flush throws.
  • A log call no longer throws because of the caller's values. A ToString() that threw in a plain hole, MEL's own formatter throwing, and a template with fewer arguments than holes all escaped the call, and LoggerFactory rethrew them to the caller; the bootstrap logger dropped such an event silently. The value now reads "[DestructuringFailed]", a state that throws part-way keeps what was captured before it, including the template, and the message is rendered from the captured fields or written as [MessageUnavailable]. Failures are counted under the new capture component and under destructurer.
  • A destructured value no longer produces an invalid JSON line when a nested type's members cannot be reflected, for example because a member attribute's constructor throws. The failure sentinel was written inside the object that had already been opened; it now replaces the value, and the type is not reflected again on later events.
  • Destructuring reads only properties with a public getter; public string Password { private get; set; } was written out. Delegates, Type and other reflection objects, assemblies, and modules are written as their names instead of being walked: a delegate exposed its closure's captured locals through Target, and a destructured exception's TargetSite filled the record budget. Memory<byte> and ReadOnlyMemory<byte> are written as hex, like byte arrays.
  • @r renders a destructured token ({@Order:j}) from its captured, masked JSON; it rendered the value's own ToString(), which printed members that [NotLogged] and [LogMasked] hide. A legacy LogReplaced attribute now masks its member whole instead of being ignored.
  • A destructured object no longer carries a key twice: a property hidden with new is written once, from the most derived type, and dictionary keys that render alike are written once, the omission marked with "…": "[Truncated]".
  • ClefLogFormatter keeps a caller's hole named SourceContext, ThreadId, or EventId, as Serilog does; it used to drop the caller's value for its own.
  • A collection in a plain hole is enumerated once. The MEL formatter enumerated it again to render the message, so a lazy query ran twice; the message is now rendered from the captured fields.
  • A dictionary with string keys and a (string, value) tuple passed to BeginScope become fields, as under Serilog, instead of the scope object's type name. @r skips template tokens Serilog does not accept as holes, so its renderings line up with the tokens that have them.
  • Logging no longer depends on the thread pool anywhere. With every pool thread busy, a queued record never reached the sink, EnqueueTimeout and ShutdownTimeout did not fire, and a provider's disposal did not return, measured with 100 ms bounds over 10 s. Producers now wake the writer directly, a caller blocked on a full queue waits on a monitor the writer signals instead of the channel, which completes a blocked write through the pool whatever its options say, and the shutdown deadlines are timed waits on a thread of their own. DisposeAsync returns to its caller at once.
  • A record with thousands of fields no longer costs its caller and the log writer in proportion. Capture stops at four times MaxFieldsPerRecord, counting the rest as over the cap; resolving duplicate names is linear instead of quadratic; and a pooled entry sheds a field table grown by such a record.
  • The built-in formatters are safe to share. The formatter passed to AddHostLoomLogging goes to the provider of every container built from the service collection, and two providers formatting through one instance corrupted each other's lines; the ILogFormatter contract now says Format may run concurrently.
  • A record dropped because the provider was being disposed or had faulted is counted as such; when the channel closed between the state check and the write, it was counted as a full queue.
  • A message rendered from captured fields writes a template's }} as }, as it already wrote {{ as {.
  • A process that ends without disposing the logging provider no longer loses the records still queued. Environment.Exit, Main returning, and an unhandled exception each ended the process around the background writer, silently losing up to a full queue, the last lines before a crash among them; a Generic Host app calling Environment.Exit lost them too, because the host is not disposed then. The provider now flushes the queue from AppDomain.ProcessExit and AppDomain.UnhandledException, within ShutdownTimeout. Environment.FailFast and a killed process still lose what is queued, and so does a SIGTERM to an application without the Generic Host that does not handle the signal: since .NET 10 the runtime raises no event for it.
  • ClefLogFormatter writes @t with all seven fractional digits. The JSON writer trimmed trailing zeros, so a timestamp could lose digits or its whole fraction, which a fixed-pattern timestamp parser rejects.