Skip to content

Releases: lsm/fiso

v0.19.0

Choose a tag to compare

@lsm lsm released this 03 Apr 17:43
d40394a

[0.19.0] — 2026-04-03

Added

  • Configurable Kafka commit policies (errorHandling.commitPolicy).
    Three modes are available for Kafka-source flows:

    Policy Offset committed when…
    sink Sink delivery succeeds (strict; pipeline stalls on failure)
    sink_or_dlq Sink succeeds or DLQ write succeeds (default)
    kafka_transaction Atomically with the sink produce (EOS)

    The default when commitPolicy is omitted is sink_or_dlq, preserving
    backward-compatible behaviour for existing flows.

  • Transactional exactly-once semantics (EOS) for Kafka-to-Kafka flows
    via commitPolicy: kafka_transaction. Uses franz-go's
    GroupTransactSession to wrap the consumer-offset commit and the sink
    Produce in a single Kafka transaction. Requires:

    • source.type: kafka and sink.type: kafka on the same cluster.
    • errorHandling.transactionalId set to a unique, stable identifier
      per consumer instance.
  • errorHandling.transactionalId config field — required when
    commitPolicy: kafka_transaction; validated at startup.

  • internal/delivery package — new package exposing:

    • CommitPolicy type with constants sink, sink_or_dlq,
      kafka_transaction.
    • NormalizeCommitPolicy / ValidateCommitPolicy helpers.
    • WithKafkaTransactionalProducer / KafkaTransactionalProducerFromContext
      for propagating a transactional producer through the call stack via
      context.Context.
  • E2E tests for all commit policy modes in
    test/e2e/kafka-commit-policies/:

    • sink — verifies delivery and consumer-group lag reaches zero.
    • sink_or_dlq — verifies good messages reach the sink and deliberately
      failed messages land in the DLQ topic.
    • kafka_transaction — verifies committed messages appear in the sink
      topic under read_committed isolation.
  • all-tests-passing CI gate job — lightweight job that depends on
    every other CI job. The branch-protection ruleset for main now
    requires only this single check, automatically covering all current and
    future jobs.

  • Example flow configs in examples/flow/kafka-commit-policies/ with
    thorough inline comments explaining each policy's trade-offs and
    operational requirements.

Fixed

  • fiso-wasmer-aio sink type switch — added missing default branch;
    previously an unrecognised sink type would silently leave the sink as
    nil, causing a nil-pointer panic inside pipeline.New.

  • fiso-wasmer-aio DLQ publisher — replaced the no-op DLQ publisher
    stub with a real kafka_source.NewPublisher for Kafka-source flows,
    matching the behaviour of fiso-flow and fiso-flow-wasmer.

  • fiso-wasmer-aio WASM factory — moved wasmruntime.NewFactory()
    outside the interceptor loop so the factory is created once and reused
    across all WASM interceptors instead of once per interceptor.

  • Lint (SA1012)internal/delivery/tx_context_test.go nil-context
    guard test now carries a //nolint:staticcheck directive; the function
    explicitly handles nil as a valid input and the test is intentional.

Security

  • Go 1.25.8 — upgraded the Go toolchain in go.mod and all CI jobs
    from 1.25.7 to 1.25.8, which fixes three standard-library CVEs:
    • GO-2026-4601: incorrect parsing of IPv6 host literals in net/url.
    • GO-2026-4602: FileInfo can escape from a Root in os.
    • GO-2026-4603: URLs in <meta> content attribute not escaped in
      html/template.

Changed

  • All Go dependencies updated to their latest available versions
    (go get -u ./... && go mod tidy). Notable upgrades include
    go.opentelemetry.io/otel v1.35 → v1.43,
    google.golang.org/grpc v1.74-dev → v1.80,
    go.temporal.io/sdk v1.39 → v1.41, and
    sigs.k8s.io/controller-runtime v0.22 → v0.23.

Full Changelog: v0.18.0...v0.19.0

v0.18.0

Choose a tag to compare

@github-actions github-actions released this 16 Feb 05:23
57e37e1

Changelog

  • 7725e0c Add Debezium CDC integration with E2E tests
  • 2390028 Add E2E tests for all WASM deployment modes
  • c25c4d9 Add WASM interceptor support to Fiso-Link
  • 502c8c3 Add WASM runtime abstraction with wazero and wasmer support
  • 14fb71e Add Wasmer binary entry points and update Makefile
  • 4582c88 Add go.sum for user-service in Debezium E2E test
  • 84e9836 Add internal/wasmer manager package for long-running apps
  • de00de9 Add wasmer-app interceptor type and build targets
  • e09aaaf Fix CI issues and improve test coverage
  • 7337a8d Fix Debezium E2E test configuration issues
  • 0606221 Fix Debezium Server Docker image version in E2E test
  • 1c42d41 Fix Dockerfile to include go.sum for reproducible builds
  • ac85e92 Fix PostgreSQL initdb error in Debezium E2E test
  • 5cc299c Fix errcheck in factory_test.go
  • ae1705e Fix franz-go API usage in Debezium E2E test user-service
  • 67784f6 Fix gofmt formatting issues in gRPC and HTTP sources
  • e24317e Fix lint issues and add Wasmer integration tests
  • 85d17a5 Implement combined Wasmer binaries (flow-wasmer, wasmer-link, wasmer-aio)
  • 36f36bf Implement full Wasmer runtime with WASI support
  • 57e37e1 Improve Wasmer integration: add validation, health checks, context propagation, and docs
  • 4cc5a37 chore: go mod tidy
  • ffd1217 ci(e2e): add all wasm deployment mode jobs in parallel
  • 8d3b8cf feat(aio): implement full Flow+Link+Wasmer orchestration and generic interceptor loading
  • 10d6a94 feat(wasm): implement WasmerAppRuntime and port injection
  • 55a4dd1 fix(ci): resolve gofmt and lint issues in tests and wasmer code
  • 159d4a9 fix(docker): bundle libwasmer.so in wasmer runtime images
  • 804c748 fix(docker): copy libwasmer from correct module subdir
  • f789eb1 fix(docker): resolve libwasmer path via go list
  • 25af3c2 fix(docker): use glibc-based images for wasmer builds
  • d91f168 fix(link+wasmer): support target port/basePath and harden wasm e2e modules
  • 7a844aa fix(test): correct path to wasm module in interceptor registry test
  • 34b5c3e fix(wasm): improve wasmer runtime concurrency and stdin handling
  • 08687ce fix(wasmer): pass mapped stdin path without leading slash
  • 2f8abfc fix(wasmer): support app-mode via host http bridge and robust stdin-file args
  • 3b35e01 fix(wasmer): treat WASI exit code 0 as successful execution
  • a0a2a94 test(config): make create-watch assertion resilient to event ordering
  • 2044a00 test(flow-wasmer): relax header assertion under cloudevents transport
  • 7a74257 test(link): cover port/basePath parsing and URL path helpers

v0.17.1

Choose a tag to compare

@lsm lsm released this 13 Feb 21:06

What's Changed

Enhancements

  • Add trace context extraction to HTTP source - Enables distributed tracing context propagation through HTTP headers for improved observability
  • Add trace context extraction to gRPC source - Enables distributed tracing context propagation through gRPC metadata for improved observability
  • Add INFO logging to fiso-link Kafka handler - Adds informational logging to match the HTTP proxy pattern, providing better visibility into Kafka message processing

Details

This release adds distributed tracing enhancements to fiso-link, enabling trace context extraction from both HTTP and gRPC sources. This allows traces to span across services seamlessly. Additionally, INFO-level logging has been added to the Kafka handler to provide consistent logging patterns across all fiso-link components.

Full Changelog: v0.17.0...v0.17.1

v0.17.0

Choose a tag to compare

@lsm lsm released this 13 Feb 14:29

What's New in v0.17.0

OpenTelemetry Distributed Tracing Support

This release adds comprehensive OpenTelemetry distributed tracing support to Fiso, enabling end-to-end visibility across your data pipelines.

Features

Opt-in Tracing

  • Enable tracing by setting FISO_OTEL_ENABLED=true
  • Configure the OTLP exporter endpoint via OTEL_EXPORTER_OTLP_ENDPOINT
  • Default endpoint: http://localhost:4317 (standard OTLP gRPC endpoint)

W3C Trace Context Propagation

  • Full support for W3C Trace Context standard
  • Automatic trace context propagation across all Fiso components
  • Seamless integration with existing distributed tracing infrastructure

Automatic Instrumentation

  • HTTP clients and servers
  • gRPC services
  • Kafka producers and consumers

Log Correlation

  • Trace ID and span ID automatically included in logs when tracing is active
  • Easy correlation between traces and log entries for debugging

Configuration

Enable tracing with environment variables:

# Enable OpenTelemetry tracing
export FISO_OTEL_ENABLED=true

# Set your OTLP exporter endpoint (optional, defaults to localhost:4317)
export OTEL_EXPORTER_OTLP_ENDPOINT=http://your-otel-collector:4317

# Optional: Set service name for tracing
export OTEL_SERVICE_NAME=fiso

Improvements

  • Build artifacts added to .gitignore for cleaner repository state

Upgrade Notes

This release is backward compatible. Tracing is disabled by default, so existing deployments will continue to work without any changes. To take advantage of the new tracing features, simply set the environment variables listed above.


For detailed documentation on configuring and using the new tracing features, see the updated README and configuration guides.

v0.16.1

Choose a tag to compare

@lsm lsm released this 12 Feb 23:57

v0.16.1

This is a patch release with CI/lint fixes.

Changes

  • Fix lint error: use t.Setenv instead of os.Setenv in tests
  • Fix gofmt formatting

v0.16.0

Choose a tag to compare

@lsm lsm released this 12 Feb 18:06

What's New in v0.16.0

This release introduces configurable logging and comprehensive data flow tracing with correlation IDs, making it easier to debug and monitor event processing in production environments.

New Features

Configurable Log Levels

All Fiso binaries now support configurable log levels:

  • --log-level flag: Set the logging verbosity directly via command line

    fiso-flow --log-level debug
    fiso-link --log-level info
    fiso-operator --log-level warn
  • FISO_LOG_LEVEL environment variable: Configure log levels without modifying command-line arguments

    export FISO_LOG_LEVEL=debug
    fiso-flow

Supported log levels: trace, debug, info, warn, error

Comprehensive Data Flow Logging with Correlation IDs

Fiso now tracks events throughout the entire processing pipeline using correlation IDs:

  • Correlation ID Propagation: Events carry correlation IDs through the entire event pipeline, from ingestion to delivery
  • INFO-level Logging: Event processing and delivery operations are logged at INFO level for production visibility
  • Latency Tracking: Logs include latency measurements for performance monitoring

Supported Correlation Headers

Fiso automatically extracts and propagates correlation IDs from standard headers:

  • fiso-correlation-id (Fiso native)
  • x-correlation-id (common standard)
  • x-request-id (HTTP standard)
  • traceparent (W3C Trace Context)

Upgrade Notes

No breaking changes in this release. Simply update your binaries to take advantage of the new logging capabilities.

Commits Included

  • 2a289e0 - Add configurable log levels for all Fiso binaries
  • bf5d170 - Add comprehensive data flow logging with correlation IDs

v0.15.0

Choose a tag to compare

@lsm lsm released this 11 Feb 23:59

v0.15.0 - Azure Workload Identity Authentication for Temporal Sink

🚀 New Feature: Azure Workload Identity for Temporal

Adds Azure Workload Identity authentication support for the Temporal sink, enabling Fiso Flow to authenticate to Temporal Cloud or self-hosted Temporal servers using Azure Entra ID when running on Azure Kubernetes Service (AKS).

What's New

Azure Workload Identity Authentication:

  • Seamless authentication to Temporal using Azure Entra ID
  • Support for Azure Workload Identity (AKS pod identity)
  • Support for Azure Managed Identity (DefaultAzureCredential)
  • Support for Service Principal credentials
  • Support for Azure CLI credentials (for local development)

Configuration:

temporal:
  client:
    host: "your-tenant.tmprl.cloud:7233"
    namespace: "your-namespace"
    tls:
      enabled: true
    auth:
      azure:
        scope: "api://{temporal-app-id}/.default"

Use Cases

This feature is ideal for:

  • AKS Deployments: Use Workload Identity to authenticate without managing secrets
  • Azure Integration: Leverage Azure's identity platform for Temporal authentication
  • Zero-Trust Security: Eliminate hardcoded credentials in favor of managed identity

Security

  • No client secrets required when using Workload Identity or Managed Identity
  • Automatic token refresh handled by Azure SDK
  • Follows Azure best practices for workload identity

Testing

  • Unit tests for Azure credential configuration
  • Integration tests for Temporal authentication
  • All existing tests pass (backward compatible)

📦 Installation

# Download binaries from the assets below

# Or use Docker
docker pull ghcr.io/lsm/fiso-flow:v0.15.0
docker pull ghcr.io/lsm/fiso-link:v0.15.0

🔗 Links

🙏 Contributors

Thanks to everyone who contributed to this release!

v0.14.0

Choose a tag to compare

@lsm lsm released this 11 Feb 20:44

v0.14.0 - Kafka SASL/OAUTHBEARER with Azure Entra

🚀 New Feature: Kafka OAuth Authentication

Adds SASL/OAUTHBEARER authentication support for Kafka clients (fiso-flow and fiso-link) using Azure Entra (formerly Azure AD) OAuth2 tokens. This enables fiso to connect to Azure-integrated Kafka clusters including Confluent Cloud with Azure identity integration.

What's New

OAUTHBEARER SASL Mechanism:

  • Azure Entra OAuth2 client credentials flow
  • Automatic token caching and refresh (via Azure SDK)
  • Support for Azure Managed Identity (DefaultAzureCredential) for AKS deployments
  • Confluent Cloud extensions support (logicalCluster, identityPoolId)

Configuration:

kafka:
  clusters:
    azure-confluent:
      brokers:
        - pkc-xxxxx.westeurope.azure.confluent.cloud:9092
      tls:
        enabled: true
      auth:
        mechanism: OAUTHBEARER
        oauth:
          provider: azure
          tenantId: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
          clientId: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
          clientSecretEnv: "AZURE_CLIENT_SECRET"  # Env var
          scope: "api://<app-id>/.default"
          extensions:
            logicalCluster: "lkc-xxxxx"
            identityPoolId: "pool-xxxx"

Security:

  • Client secrets via environment variables only (never hardcoded)
  • Follows same pattern as Temporal authentication (v0.13.0)
  • Token automatic refresh before expiration

Dependencies

  • Added github.com/Azure/azure-sdk-for-go/sdk/azidentity@v1.13.1
  • Added github.com/Azure/azure-sdk-for-go/sdk/azcore@v1.21.0

Testing

  • 10 new unit tests (validation + mechanism creation)
  • Integration test for Azure OAuth (optional, manual)
  • All existing tests pass (backward compatible)

Supported Auth Methods

fiso now supports 4 Kafka SASL mechanisms:

  1. ✅ PLAIN
  2. ✅ SCRAM-SHA-256
  3. ✅ SCRAM-SHA-512
  4. OAUTHBEARER (Azure Entra) ← NEW!

📦 Installation

# Download binaries
curl -L https://github.com/lsm/fiso/releases/download/v0.14.0/fiso-flow-linux-amd64 -o fiso-flow
curl -L https://github.com/lsm/fiso/releases/download/v0.14.0/fiso-link-linux-amd64 -o fiso-link
chmod +x fiso-flow fiso-link

# Or use Docker
docker pull ghcr.io/lsm/fiso-flow:v0.14.0
docker pull ghcr.io/lsm/fiso-link:v0.14.0

🔗 Links

🙏 Contributors

Thanks to everyone who contributed to this release!

v0.13.0

Choose a tag to compare

@lsm lsm released this 11 Feb 13:26

🔐 Temporal Authentication Support

This release adds comprehensive authentication support for Temporal workflows, enabling secure connections to Temporal Cloud and self-hosted clusters.

New Authentication Methods

1. Static API Key

sink:
  type: temporal
  config:
    hostPort: namespace.tmprl.cloud:7233
    auth:
      apiKey: "your-api-key"

2. Dynamic API Key (from environment)

sink:
  type: temporal
  config:
    auth:
      apiKeyEnv: "TEMPORAL_API_KEY"

3. Token from File

sink:
  type: temporal
  config:
    auth:
      tokenFile: "/var/secrets/temporal-token"

4. OIDC (OAuth2 Client Credentials)

sink:
  type: temporal
  config:
    auth:
      oidc:
        tokenURL: "https://auth.example.com/oauth/token"
        clientID: "your-client-id"
        clientSecretEnv: "OIDC_CLIENT_SECRET"
        scopes:
          - "temporal:write"

5. Mutual TLS (mTLS)

sink:
  type: temporal
  config:
    tls:
      certFile: "/path/to/client.crt"
      keyFile: "/path/to/client.key"
      caFile: "/path/to/ca.crt"  # optional

TLS Configuration

Disable TLS (for local development):

sink:
  type: temporal
  config:
    tls:
      disabled: true  # Allows auth without TLS (dev only)

Server-only TLS (verify server certificate):

sink:
  type: temporal
  config:
    tls:
      enabled: true
      caFile: "/path/to/ca.crt"

Testing & Quality

  • ✅ 12 unit tests covering all credential builders
  • ✅ 4 integration tests (API key, token file, OIDC)
  • ✅ Full E2E test with mock OAuth2 server
  • ✅ All CI tests passing

Other Improvements

  • Enhanced CloudEvents SDK integration (from v0.12.2)
  • Better YAML config parsing for Temporal settings
  • Improved test infrastructure with Docker Compose patterns
  • Coverage threshold adjusted to 94.5% (more realistic for integration-heavy code)

Full Changelog: v0.12.2...v0.13.0

v0.12.2

Choose a tag to compare

@github-actions github-actions released this 08 Feb 06:39
ff3ab97

Changelog

  • ff3ab97 fix: add missing CloudEvents fields to CloudEventsConfig struct
  • c941014 fix: wire all 7 CloudEvents fields from config to pipeline