Some comments after some usage of redb

[manutiedra]

Hello,

I'd like to give you some feedback after playing redb for some hours:

1. The async methods do not have a CancellationToken.

2. It would be nice that RedbObject has a DebuggerDisplay attribute or an overriden ToString so when you are debugging you can see more useful information of the underlying TProps in order to get more information without having to expand the objects. This does not help:

Imagine if it shows the Id, Name and ParentId values, as well as the ToString output of the TProps. That would be very useful.

3. In my company we use Impersonation a lot. A lot of customers call and the operator uses the application impersonating the customer. I think that it will be a nice addition to add impersonation support to _objects, because you want to know who did something and if it was impersonated or not. Also, if I don't specify any user, everything is stored using the user 0 (sys). I'd expect a null or maybe the default user (-1)

4. The audit information is very basic (who created/modified it and when). As in the pro version you have the diff, it would be nice to be able to configure full audit information (so the pro version can save that diff somewhere).

5. The soft delete seems to be a delayed delete more than a soft delete as I see no way to really undelete anything that has been soft deleted.

6. If I create a RedbOject where a property of A is of type RedbOject, when I save RedbOject, 2 entries in _objects are created, one for A and one for B. Also, an entry in _values for A references B using the _Object column. However, if I want RedbOject with A having a property of type RedbOjbect (non-generic, as I only want to use a basic value (byte[] in this case), it fails generating the _structure:

You can reproduce it adding this code to the Product class in the default template:

public RedbObject? ByteVal1 { get; set; }

I was hopping that to work and then use the ValueBytes of ByteVal1.

Hopefully you find some of those comments useful. Best regards.

[reliktbk]

Thanks for the detailed feedback. Some of it (interceptors, CancellationToken) was already on the 4.0 roadmap, and your write-up moved it to the front of the queue: both are in the tree now. The rest turned out to be diagnostics or documentation defects, and we fixed those right away. Point by point:

1. CancellationToken. This was planned for the major release (there won't be another window for signature changes for years), so we pulled it forward and did it in full: every async method of the public contracts now takes a trailing CancellationToken cancellationToken = default. That covers storage, LINQ queries (including tree/grouped/window forms), schemes, trees, permissions, users and roles: about 270 methods. Existing code compiles unchanged. The semantics are pinned down and covered by tests on all three providers. Cancellation surfaces as OperationCanceledException and nothing else (the SQL Server driver's SqlException-on-cancel is normalized). A cancelled SaveAsync rolls back completely, all or nothing. The point of no return is the commit: after a successful commit the token is never consulted again, so "cancelled" can never mean "but actually saved".

2. Debugger and ToString. RedbObject now carries [DebuggerDisplay]: the debugger shows {name} (id=…, scheme=…) with no expansion and no database round-trips, and a lazy stub is never woken by it. ToString() returns the object's canonical JSON, exactly the same contract as the in-database SQL function get_object_json(id). That's handy both in logs and for comparing what's in memory against what's in the database.

3. Impersonation / "who really made the change". Covered by the interceptors (see point 4): in SavingAsync/SavedAsync your application sees both the objects and the effective user of the operation. Next to whatever real/effective pair you track yourself, that's enough to write an audit journal in any shape you want.
   A practical recommendation from our own production apps: keep the audit journal as a plain flat table, not as redb objects. An audit log is an append-only stream; what it needs is cheap inserts and indexes on timestamp/user/event type, not schemes and props. IRedbService.Context gives you raw SQL access to the same database, so the journal lives right next to your redb data: create a table like audit_log (id BIGSERIAL, event_type, "timestamp", user_login, entity_type, entity_id, details JSONB) once at startup via redb.Context.ExecuteAsync(ddl), and let your interceptor insert into it. On top of that, every redb object also carries service fields (key, value_long and friends) for your own lookup keys.

4. EF-style interceptors. Also from the roadmap, arrived early: meet IRedbSaveInterceptor, four hooks with default implementations, deliberately EF-shaped so it feels familiar. SavingAsync fires after the object graph is collected but BEFORE ids and hashes are assigned: you may mutate Props (EF's SavingChanges contract), objects being created show Id = 0 (temporary-key semantics), and an exception cancels the whole save. SavedAsync fires after the commit, with the final ids of created objects; under the ChangeTracking strategy it also carries the row-level diff the save actually applied (PropertyPath in model terms: "Title", "Items[1].Price", plus the old and new value). DeletingAsync/DeletedAsync apply the same rules around hard deletes (veto before the SQL, fact after). Register any number of them through DI; with no subscribers the pipeline pays nothing.

5. Soft delete. The important thing here is calling it by its real name: SoftDeleteAsync is not a recoverable recycle bin, it's a deferred deletion mechanism (the Background Deletion System). The mark is instant and atomic, and the object disappears from queries immediately, while the physical removal of a large subtree is done in batches in the background. That's exactly why the scheme is cleared and unique keys are released on the spot: a key becomes reusable right away, without waiting for the background purge to finish.
   And the purge runs out of the box: UsePostgres/UseMsSql/UseSqlite each register BackgroundDeletionService (a hosted service) automatically. The database itself is the queue: the worker atomically claims pending trash containers (cluster-safe, so two instances never fight over one container) and purges them batch by batch. Nothing to launch by hand; it just needs your app to run under the Generic Host. If you want explicit control and progress reporting, there's DeleteWithPurgeAsync(ids, batchSize, progress); see example E087 in redb.Examples.
   If what you need is an actual recoverable trash bin, that's a different pattern: move the object under your own "trash" container using the ordinary tree operations, without SoftDelete. We've strengthened the documentation on this distinction.

6. Bare RedbObject as a property / binary data. What you hit (the IX__structures duplicate) was a diagnostics defect: scheme sync now rejects a bare RedbObject property loudly, with a hint ("declare RedbObject; for raw bytes use byte[]"). For what it's worth, RedbObject with an empty TProps would have worked even then. But the original task is now solved directly: byte[] in Props is stored as a scalar BLOB (one _values row, not a row per byte) and round-trips correctly on all three providers, pinned by a dedicated suite of 78 checks. And if a large blob shouldn't be pulled on every load of its parent, put it into a child RedbObject and hold it as a reference: lazy loading (it's in 4.0) will fetch it only when actually touched.

Everything above is already in the tree and will ship with 4.0.0 (not released yet, coming soon).

[manutiedra]

Thank you very much for the detailed explanations. You don't need to hurry the release for me. I still have plenty of things to try and learn to be able to use redb.Route and redb.Tsak.

Regarding the interceptors... will it be possible to insert in a custom table like audit_log in the same transaction Redb uses for SaveAsync/DeleteAsync?

I saw the hosted service for the Soft Delete but I wanted to point to you that the name of the method (SoftDeleteAsync) can be a source of confusions for newcomers like me.

About the Bare RedbObject... I thought that I could add a property of type bare RedbObject to my domain model (because I only wanted to use one of the value properties, so using RedbObject felt bad as if I didn't want to have TProps, but I see that's not allowed. I welcome the change that now byte[] in Props is stored as a BLOB in _values. Thank you.

BTW, I wanted to get some objects by Name but I didn't find how to do it. Finally I found the method WhereRedb. Maybe it was me but I didn't find that easily. So Maybe the documentation is not very clear about it. In the Examples, there's only one example of WhereRedb. I think that adding another one in the first places of the query section would be helpful.

[reliktbk]

Good questions, all three. In order:

1. Audit records in the same transaction: yes, and the recipe is one line. By default the hooks deliberately do not share the save's transaction. SavingAsync runs before BEGIN, so a veto cancels the save without any rollback; SavedAsync runs after COMMIT, so a hook can never unwind a commit that already happened. But when the audit row must be atomic with the write, wrap the call in your own transaction:

await redb.Context.ExecuteAtomicAsync(async () =>
{
    await redb.SaveAsync(obj); // the whole pipeline, SavedAsync included, runs inside this transaction
});

Inside that scope an insert from SavedAsync via redb.Context.ExecuteAsync(...) joins the same transaction and commits or rolls back together with the save. The same works for DeleteAsync and its DeletedAsync hook. Combined with the flat audit table from the previous answer, that gives you an atomic journal: one transaction, business write plus audit row, all or nothing.

2. On the SoftDeleteAsync name: point taken, and the name stays. It is a shipped contract, and renaming a contract to fix a first impression is a worse trade than documenting it properly. What we did instead: the documentation now leads with what the method actually is (deferred deletion: an instant atomic mark, background batched purge, unique keys released on the spot) and with the recoverable-bin alternative (move the object under your own trash container using the ordinary tree operations). Worth adding: the purge runs out of the box. UsePostgres/UseMsSql/UseSqlite each register the background deletion worker automatically, the database itself is the queue, and the claim is cluster-safe. Nothing to launch by hand.

3. WhereRedb docs: done, and it ships with 4.0.0 like the rest. We've added a dedicated example (E205 in redb.Examples) covering the three production patterns: a lookup by the service value_string key, an id-set prefilter with Contains, and the composite WhereRedb(system predicate).Where(props predicate) in one query, with both predicates translated to SQL. The service fields (key, value_long, value_string and friends) live on the _objects row itself, so they make good lookup keys next to your business data with no scheme changes. Until the release, the short version: Where filters your Props, WhereRedb filters the system fields of the object row, and they chain freely in one query.

Thanks again for pushing on the details: two of your three follow-ups turned into shipped improvements (the atomicity recipe is now spelled out in the interceptor docs, and E205 exists because of this thread).

[reliktbk]

4.0.0 is out, so here is what each of your six points turned into. Everything below is on nuget.org now.

1. CancellationToken. Shipped as described: a trailing CancellationToken cancellationToken = default on every async method of the public contracts, about 270 of them, and existing calls compile unchanged. Cancellation surfaces as OperationCanceledException and nothing else, a cancelled SaveAsync rolls back completely, and after a successful commit the token is never consulted again.

2. Debugger and ToString. RedbObject carries [DebuggerDisplay], and ToString() returns the object's canonical JSON, the same contract as get_object_json(id) in the database. Neither wakes a lazy stub.

3 and 4. Impersonation and richer audit. IRedbSaveInterceptor is in: four hooks with default implementations, EF-shaped. SavingAsync can mutate Props and veto the save, SavedAsync carries the final ids and, under the ChangeTracking strategy, the row-level diff the save actually applied. The atomicity recipe from the follow-up round is documented: wrap the call in ExecuteAtomicAsync and an insert from SavedAsync joins the same transaction, so a business write and its audit row commit or roll back together.

5. Soft delete. The name stays, and the documentation now leads with what the method actually is: an instant atomic mark, a background batched purge, unique keys released on the spot. The recoverable bin remains a different pattern, your own trash container built from ordinary tree operations.

6. Bare RedbObject as a property. Scheme synchronisation refuses it loudly now, with a hint, instead of producing the duplicate-index crash you hit. The original task is solved directly: byte[] in Props is stored as the scalar BLOB it is, one _values row instead of one per byte, and a database carrying the old layout converts itself on the next synchronisation. [RedbUnique] on a byte[] property is allowed, so dedup by content works. And if a large blob should not travel with its parent, hold it as a reference: lazy references are in, with LoadAsync(depth: 1) as the default.

E205 is in redb.Examples as promised, covering the three WhereRedb patterns.

Two things worth knowing beyond your list. Pro is free on the whole 4.x line, no key, commercial use included. And if your _values table is large, read the upgrading section before the first start: the V4 schema block takes exclusive locks for the length of the bundle, and there is a quiet-window path for it (GetUpgradeScript() or redb schema --upgrade, with AutoApplyDatabaseUpgrades = false on the application nodes).

Changelogs for the four products, all released together on 11 September:

• redb: https://github.com/redbase-app/redb/blob/main/CHANGELOG.md
• redb.Route: https://github.com/redbase-app/redb-route/blob/main/CHANGELOG.md
• redb.Tsak: https://github.com/redbase-app/redb-tsak/blob/main/CHANGELOG.md
• redb.Identity: https://github.com/redbase-app/redb-identity/blob/main/CHANGELOG.md

The announcement, with the breaking changes and the upgrade notes: https://github.com/orgs/redbase-app/discussions/13

Thanks again for this thread. Two of the six were roadmap items your write-up moved to the front of the queue, and the rest were defects worth fixing on their own.

[reliktbk]

Good news first: the routes working over SFTP means the module, the database and the transactions are all fine. What is left is the API key, and that one is on us — the failure is silent where it should not be.

Why every dashboard page fails

KeyHash must be the HMAC in upper case hex. The worker computes Convert.ToHexString(...), which is upper case, and looks the key up by that exact string. A hash written in lower case is still accepted into the store at startup — you even get Seeded API key ... from config in the log — and then never matches, so every call comes back as invalid or expired API key. openssl dgst -hmac prints lower case, which is how this usually happens.

I reproduced your error here with nothing but a lower-cased hash, and the log was line for line yours.

Compute it like this:

printf %s "$TSAK_API_KEY" | openssl dgst -sha256 -hmac "$TSAK_AUTH_SECRET" | awk '{print toupper($NF)}'

[Convert]::ToHexString(
  [Security.Cryptography.HMACSHA256]::new([Text.Encoding]::UTF8.GetBytes($secret)).ComputeHash(
    [Text.Encoding]::UTF8.GetBytes($key)))

The secret is Tsak__Auth__Secret, at least 16 characters; the key is whatever random string you put in Tsak__Web__Nodes__0__ApiKey. The worker stores only the hash.

One more thing in your file, which would have bitten you next: roles are one string, not an array.

Tsak__Auth__Keys__0__Roles: "admin"          # right, comma-separated for several
Tsak__Auth__Keys__0__Roles__0: "admin"       # wrong: the key ends up with no roles

With EnforceRoles: "true" and RolelessKeysAreAdmin: "false" a roleless key is refused everywhere.

About the CLI: the command is tsak auth create --name ci --roles admin, with tsak auth list and tsak auth revoke next to it. Our README writes them as auth keys create, which is simply wrong — thank you for finding it, we are fixing the README. Note also that the CLI talks to the same authenticated API, so the first key still has to come from configuration, the way you did it.

One consequence of the store: keys live in the database from the first start. A new KeyHash under the same Id replaces the stored hash in place, so the previous key stops working; a different Id adds a second key. tsak auth list shows what the worker actually has.

All of this stops being a trap in the next release: the hash is normalised, so either case works, roles are accepted as a string and as an array, and a KeyHash that is not a hash at all is refused with an error naming the key instead of silence. The README command is already corrected.

A compose file we ran end to end

This is the one I started here against Tsak 4.0.0 with authentication on, and checked: the management API answers with the key and 401 without it, the dashboard logs in, and the pages that failed for you (/, /node/local, /node/local/routes, /node/local/modules) render. It also creates the database, which redb does not do for you.

name: serials-tsak

services:
  sqlserver:
    image: mcr.microsoft.com/mssql/server:2025-latest
    container_name: serials-sqlserver
    restart: unless-stopped
    environment:
      ACCEPT_EULA: "Y"
      MSSQL_SA_PASSWORD: "${SQL_SA_PASSWORD:?SQL_SA_PASSWORD required}"
      MSSQL_PID: "Developer"
    ports:
      - "${SQL_PORT:-1433}:1433"
    volumes:
      - sqlserver_data:/var/opt/mssql
    healthcheck:
      test: ["CMD-SHELL", "/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P \"$$MSSQL_SA_PASSWORD\" -C -Q 'SELECT 1' || exit 1"]
      interval: 10s
      timeout: 5s
      retries: 20
      start_period: 30s
    networks: [serials]

  # redb creates its tables, not the database. One shot, then exits.
  db-init:
    image: mcr.microsoft.com/mssql/server:2025-latest
    container_name: serials-db-init
    depends_on:
      sqlserver:
        condition: service_healthy
    environment:
      MSSQL_SA_PASSWORD: "${SQL_SA_PASSWORD:?SQL_SA_PASSWORD required}"
    entrypoint: ["/bin/bash", "-c"]
    command:
      - >
        /opt/mssql-tools18/bin/sqlcmd -S sqlserver,1433 -U sa -P "$$MSSQL_SA_PASSWORD" -C
        -Q "IF DB_ID('serials_demo') IS NULL CREATE DATABASE serials_demo;"
    restart: "no"
    networks: [serials]

  sftp:
    image: atmoz/sftp:latest
    container_name: serials-sftp
    restart: unless-stopped
    command: "${SFTP_USER:-testuser}:${SFTP_PASSWORD:?SFTP_PASSWORD required}:::upload"
    ports:
      - "${SFTP_PORT:-2222}:22"
    volumes:
      - ./runtime/sftp:/home/${SFTP_USER:-testuser}/upload
    networks: [serials]

  tsak:
    image: ghcr.io/redbase-app/redb-tsak-stack:${TSAK_VERSION:-4.0.0-net10}
    container_name: serials-tsak
    restart: unless-stopped
    depends_on:
      sqlserver:
        condition: service_healthy
    ports:
      - "${DASHBOARD_PORT:-8085}:8085"
      - "${API_PORT:-9090}:9090"
    environment:
      # The worker's own storage: redb on SQL Server. The provider decides the dialect; the
      # connection string alone does not. The image ships sqlite so that it starts with no server.
      Tsak__Storage__Type: "Redb"
      Tsak__Redb__Provider: "mssql"
      Tsak__Redb__UsePro: "true"
      # Writes only the properties that actually changed, instead of rewriting all of them on every
      # save. Needs the Pro tier above, which is free.
      Tsak__Redb__PropsSaveStrategy: "ChangeTracking"
      ConnectionStrings__MSSql: "Server=sqlserver,1433;Database=serials_demo;User Id=sa;Password=${SQL_SA_PASSWORD};TrustServerCertificate=true"

      Tsak__Modules__AssemblyPaths__0: "/app/worker/modules"
      Tsak__HotReload__Enabled: "true"

      # KeyHash: HMAC-SHA256 of the key, secret as the HMAC key, hex, UPPER case.
      # Roles: one string, not an array.
      Tsak__Auth__Enabled: "true"
      Tsak__Auth__Secret: "${TSAK_AUTH_SECRET:?TSAK_AUTH_SECRET required}"
      Tsak__Auth__Keys__0__Id: "svc-tsak-web"
      Tsak__Auth__Keys__0__Name: "tsak-web-dashboard"
      Tsak__Auth__Keys__0__KeyHash: "${TSAK_WEB_API_KEY_HASH:?TSAK_WEB_API_KEY_HASH required}"
      Tsak__Auth__Keys__0__Roles: "admin"

      # The dashboard reaches the worker in the same container, with that same key.
      Tsak__Web__AdminLogin: "admin"
      Tsak__Web__AdminPassword: "${TSAK_WEB_ADMIN_PASSWORD:?TSAK_WEB_ADMIN_PASSWORD required}"
      Tsak__Web__Mode: "standalone"
      Tsak__Web__Nodes__0__Id: "local"
      Tsak__Web__Nodes__0__Url: "http://localhost:9090"
      Tsak__Web__Nodes__0__ApiKey: "${TSAK_WEB_API_KEY:?TSAK_WEB_API_KEY required}"

      # Module settings, through the Override layer of its context.
      Tsak__Contexts__serial-numbers__Override__Sftp__Host: "sftp"
      Tsak__Contexts__serial-numbers__Override__Sftp__Port: "22"
      Tsak__Contexts__serial-numbers__Override__Sftp__Username: "${SFTP_USER:-testuser}"
      Tsak__Contexts__serial-numbers__Override__Sftp__Password: "${SFTP_PASSWORD}"
      Tsak__Contexts__serial-numbers__Override__Archive__Directory: "/data/archive"
      Tsak__Contexts__serial-numbers__Override__Report__Directory: "/data/reports"
      Tsak__Contexts__serial-numbers__Override__As2__CertificateDirectory: "/data/certs"
      Tsak__Contexts__serial-numbers__Override__As2__CertificatePassword: "${AS2_CERT_PASSWORD:-demo}"
    volumes:
      - ./modules:/app/worker/modules
      - ./runtime/certs:/data/certs:ro
      - ./runtime/archive:/data/archive
      - ./runtime/reports:/data/reports
      - dataprotection:/root/.aspnet/DataProtection-Keys
    networks: [serials]

volumes:
  sqlserver_data:
  dataprotection:

networks:
  serials:

.env next to it:

SQL_SA_PASSWORD=
SFTP_USER=testuser
SFTP_PASSWORD=
TSAK_AUTH_SECRET=              # at least 16 characters
TSAK_WEB_API_KEY=              # any random string
TSAK_WEB_API_KEY_HASH=         # the command above, UPPER case
TSAK_WEB_ADMIN_PASSWORD=

The dataprotection volume matters once you run it for longer than an afternoon: without it the dashboard's keys are regenerated on every recreate and everyone is logged out.

What it looks like when it works

The Tsak dashboard, here with the Identity image from my previous message, so you can compare against yours — contexts, routes, start and stop:

And that image's own Identity admin UI, to show what a module can put on top:

[reliktbk]

[reliktbk]

4.0.1 is out, and with it the new demo. I ran the whole thing against a real Tsak worker before writing this, so the list below is what the log says, not what we hope.

The fixes you ran into

• A stopped /api/echo answers 503 naming the route to start, instead of falling through to the authenticated API and answering 401.
• A context in which no module came up is no longer started, and the log says so instead of the cheerful "all 0 endpoints operational".
• API keys: the KeyHash is normalised, so the case of the hex no longer matters, roles are accepted both as a string and as an array, and a KeyHash that is not a hash is refused with an error naming the key instead of silence. The README command is corrected: it is tsak auth create.
• File consumers have poll backoff, the Camel options plus one of our own: backoffOnFailedExchanges also counts a file that failed to process, not only a failed poll. In the demo the SFTP consumer uses it, so a hub whose database is down stops hammering the partner's folder.
• InitRoute.main may be async, which the demo now uses.

What the demo does now, and what changed since you last read it

The demo: https://github.com/redbase-app/redb-route/tree/main/demo/SerialNumbersDemo

Take it again rather than patching what you have: besides the move to 4.0.1 and the async main, it now ships a compose file for Tsak, a SerialNumbers.Worker seed step that prepares the database, and the second spelling of the module in XML. Two changes matter for the code you already know.

Duplicates are caught by the data, not by a registry of delivered files. The IdempotentConsumer step is gone. The request object carries {partner}/{request id} as its unique key, written inside the same transaction: a second request with that id is answered as a duplicate, a concurrent twin fails on the key, and a delivery that rolled back leaves no trace and is simply processed again. One fact, one place. A registry of processed files keys the same fact by file name and size, which a corrected file changes and a re-sent request does not.

The message trail is printed, not just collected. .MessageHistory() times every step, but the trace lives in the exchange, so something has to print it: the parked-message handler logs it at Warning, and the request route logs it at Debug. Raise the module to Debug and every request shows the path it took, with timings.

The run

A Tsak 4.0.1 worker, redb on SQL Server, the module dropped into modules/ while the worker was running:

Loaded entry point SerialNumbers.Core.dll from package SerialNumbers into ALC pkg:SerialNumbers
Package SerialNumbers: 1 entry points, 5 companion DLLs loaded
Applied Override layer for context serial-numbers (4 keys)
SerialNumbers module ready: 2 partners, archive at ..., product API on port 8090
Initialized module SerialNumbers.Core in context serial-numbers
Context 'serial-numbers' started successfully: all 12 endpoints operational

Then one request through SFTP, for a product that was not active yet:

acme: received SNREQ_ACME_9001.xml over SFTP
acme: SNREQ_ACME_9001.xml archived as acme/2026/09/18/004532759_SNREQ_ACME_9001.xml
acme: SNREQ_ACME_9001.xml is a serial number request
acme: request ACME-2026-9001 is on hold until its product is activated

Activating the product over the module's own REST endpoint returns what it did, and releases the request:

PUT /api/products/04607001234567/status  {"status":"Active","changedBy":"you"}
{"gtin":"04607001234567","status":"Active","outcome":"Unchanged","requestsReleased":1}

04607001234567: 1 request(s) on hold to release
acme: releasing request ACME-2026-9001 from acme/2026/09/18/004532759_SNREQ_ACME_9001.xml
acme: request ACME-2026-9001 Accepted, response queued
acme: delivered SNRESP_ACME-2026-9001_1040173.xml
Quota report written: quota_usage_20260918_005000.csv

Here is the worker console through the whole of it, from the hot-loaded package to the quota report:

The partner never re-sends anything: the hub reads its archived file back and decides the same request object again, against the product and the quota as they are now.

The response carries 30 000 serial numbers, all distinct, one per block of 10 000 and at a random position inside its block: 000000005495, 000000017122, 000000022206, 000000034264, 000000049123. That is the anti-counterfeit property you asked for, and it is one set-based INSERT ... SELECT over GENERATE_SERIES, not a loop.

Two notes on running it:

• The compose file from my previous message is the one I used, with TSAK_VERSION=4.0.1-net10.
• If you keep the database from the earlier version of the demo, drop the demo's own tables first (serial_numbers, serial_blocks, serial_allocations, outbox, product_status_changes). The schema script creates tables that are missing but does not alter tables that exist, so an old table shows up later as Invalid column name. The redb objects are not affected.

The same integration, written in XML

There is a second spelling of the module in the demo, SerialNumbers.Xml. Same domain, same services, same database, same worker; the flow lives in context.xml and routes/*.route.xml instead of RouteBuilder classes. The heart of it reads like this:

<route id="serial-number-request" description="A serial number request: validate, decide, respond">
  <from uri="direct://serial-number-request"/>
  <messageHistory/>

  <tryCatch>
    <try>
      <validateXsd file="SerialNumberRequest.xsd"/>
      <unmarshal format="application/xml"
                 target="SerialNumbers.Core.Integration.Xml.SerialNumberRequestXml, SerialNumbers.Core"/>
    </try>
    <catch exceptions="redb.Route.Validation.ValidationException, redb.Route">
      <to uri="bean:#intake?method=RecordInvalid"/>
      <log level="Warning">${header.serials.partner}: ${header.serials.fileName} violates the schema, recorded as Invalid</log>
      <stop/>
    </catch>
  </tryCatch>

  <transaction>
    <to uri="bean:#serial-requests?method=Register"/>
    <choice>
      <when expr="${header.serials.decision} == 'Accepted'">
        <to uri="bean:#serial-requests?method=Allocate"/>
        <to uri="bean:#serial-requests?method=QueueResponse"/>
      </when>
      <when expr="${header.serials.decision} == 'Rejected'">
        <to uri="bean:#serial-requests?method=Reject"/>
        <to uri="bean:#serial-requests?method=QueueResponse"/>
      </when>
    </choice>
  </transaction>

  <log level="Debug">${messageHistory()}</log>
</route>

What the markup buys: the pipeline is readable without a compiler. What <transaction> encloses is what commits together, and the branch names of a <choice> are the decision table. The package deploys as a .tpkg the worker hot-reloads, and the VS Code extension draws the graph from the same file.

What it costs, honestly, two things:

• Partner topology. The C# module generates one SFTP consumer and one delivery route per partner read from the database. Markup is static, so there every partner is written out: acme over SFTP, globex over AS2. Choose by how your partners change: by deploy, or at runtime. With per-partner formats, as in your case, that is the question to answer first.
• The bootstrap. Creating and seeding the database is code either way; the XML module reuses the C# module's InitRoute.main for it.

Certificates used to be on that list and are not any more: a <property> takes a nested anonymous <bean>, and factoryMethod= calls a static creator, so an AS2 partner is declarable in full, key material included:

<bean name="globex" type="redb.Route.As2.As2ConnectionFactory, redb.Route.As2">
  <property key="OurCertificate">
    <bean type="System.Security.Cryptography.X509Certificates.X509CertificateLoader, System.Security.Cryptography.X509Certificates"
          factoryMethod="LoadPkcs12FromFile">
      <constructorArg value="{{As2.CertificateDirectory}}/hub.pfx"/>
      <constructorArg value="{{As2.CertificatePassword}}"/>
    </bean>
  </property>
  <property key="As2From" value="{{As2.Id}}"/>
  <property key="As2To" value="GLOBEX"/>
  <property key="Sign" value="true"/>
</bean>

Run one spelling or the other, not both at once: they share the demo database, so they would race for the same partner folders. Everything else about them is already distinct (context name, API port, AS2 port, archive and report folders), so switching is dropping one .tpkg and removing the other.

[manutiedra]

Thank you very much for your help and detailed explanations.

I will go through the new version of the demo and the redb* 4.0.1 version and hopefully I can have the auth and the observability (Grafana, Prometheus, Jaeger UI).

[reliktbk]

One line worth adding to that compose file: Tsak__Redb__PropsSaveStrategy: "ChangeTracking".

The default is DeleteInsert, which rewrites every property of an object on every save. ChangeTracking compares with what is stored and writes only what actually changed. It needs the Pro tier, which is free and which the compose already turns on with Tsak__Redb__UsePro.

In this hub the difference is not academic: a request is saved several times over its life, recorded on arrival, put on hold, decided again when the product is activated, and a product row changes status on top of that. Only the serial numbers themselves are pure inserts, and those go through one set-based statement anyway.

That default is our oversight, not a recommendation, and we are changing it. Until then it is that one line, which I have also added to the compose file in the earlier message.

And one breaking change to know before you move to 4.0.1, if you have sql: statements of your own: parameters are now written :#name, as in Apache Camel, instead of @name. In 4.0.1 @ belongs to the database and reaches it as written, so T-SQL variables and EXEC proc @arg = :#value now work, but an old WHERE id = @id fails with "Must declare the scalar variable". The demo's outbox route shows the change: WHERE id = @id became WHERE id = :#id, and @redbError became :#redbError. It is also why the new demo does not run on 4.0.0.

[reliktbk]

One more thing that belongs to point 6, and it matters if you actually stored bytes on 3.x.

In 4.0.0 the whole binary path works, on all three providers and both tiers: byte[] as a property of Props, now stored as the scalar BLOB it is (one _values row instead of one row per byte), and the root ValueBytes field you wanted to reach. It is pinned by a round-trip suite on the six Free and Pro fixtures, covering null, the empty array, all three padding classes, all 256 byte values and a megabyte. Reads on SQL Server got about 2.5 times cheaper along the way, FOR XML PATH(''), BINARY BASE64 instead of an XML instance per value, so an object carrying a 1 MB byte[] reads in 134 ms server-side instead of 336 ms.

On 3.7.2 that was not the case, and one combination deserves a look because it was the silent one: SQL Server on the Free tier never wrote ValueBytes into the column. No exception, nothing in the log, the column stayed NULL. The other broken combinations at least announced themselves, byte[] in Props failed at load on every provider and PostgreSQL Free threw when reading the root field, and Pro was correct everywhere, since it materialises from the typed columns instead of the JSON projection.

So if a 3.x SQL Server database on Free stored anything through ValueBytes, check the column before you migrate:

SELECT _id, _name,
       CASE WHEN _value_bytes IS NULL THEN 'NULL'
            ELSE CAST(DATALENGTH(_value_bytes) AS varchar(10)) END AS bytes
FROM _objects WHERE _name = 'your object';

Those bytes never reached the database, and 4.0.0 cannot recover what was never written. A byte[] property in Props is the safer thing to reach for anyway, and in 4.0.0 it is also the cheaper one.