Skip to content

[RUM-16912] Refactor transport layer for extensibility#140

Merged
cdn34dd merged 1 commit into
mainfrom
carlosnogueira/RUM-16912/refactor-transport-layer-for-extensibility
Jun 23, 2026
Merged

[RUM-16912] Refactor transport layer for extensibility#140
cdn34dd merged 1 commit into
mainfrom
carlosnogueira/RUM-16912/refactor-transport-layer-for-extensibility

Conversation

@cdn34dd

@cdn34dd cdn34dd commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

Motivation

The batch transport currently has a single concrete BatchProducer/BatchConsumer pair with one strategy hardcoded into it: serialize events as NDJSON to disk, then POST them as a JSON array with DD-API-KEY auth. That works fine for RUM and spans, but we're about to need other strategies — session replay being the next one, and its intake contract is quite different (multipart body, query-param auth, segments rather than line-delimited events).

As things stand there's nowhere clean to add that. You'd either fork the classes or bolt conditionals onto code that also handles file rotation, the write queue, and upload scheduling — none of which actually changes between strategies. This PR is the groundwork to avoid that: separate the shared plumbing from the per-track specifics so a new strategy can slot in without touching any of the scheduling/rotation logic.

Changes

  • BatchProducer and BatchConsumer are now abstract base classes. They keep everything that's identical across tracks, the write queue, directory setup, orphaned-.tmp recovery, file naming, .log discovery, and the send-then-delete-on-success loop. Subclasses implement only the two things that genuinely differ: writeData() (how an event is serialized to disk) and buildRequest() (how a batch of lines becomes an HTTP request).
  • New standard/ implementation (StandardBatchConsumer / StandardBatchProducer). This is the existing behavior moved behind the two hooks, NDJSON on disk, JSON-array POST. No behavior change for RUM or spans.
  • BatchManager now builds its pair through a createProducerConsumerPair() factory. Today it only returns the standard pair; adding a strategy later is a new subdirectory plus one branch here, and nothing else in the manager has to change.
  • Tests were reorganized to follow the new layout, base-class behavior is covered through a small test subclass, and the standard implementation has its own producer/consumer specs.

Test instructions

Checklist

  • Tested locally (playground)
  • Added unit tests for this change.
  • Added e2e/integration tests for this change.
  • Updated related documentation.

@cdn34dd cdn34dd requested a review from a team as a code owner June 19, 2026 10:04
@bcaudan bcaudan requested a review from Copilot June 22, 2026 14:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the batch transport layer to separate shared batching/file-rotation/upload plumbing from track-specific serialization and request construction, enabling future transport strategies (e.g., session replay) without modifying scheduling/rotation logic.

Changes:

  • Made BatchProducer / BatchConsumer abstract base classes with strategy hooks (writeData, buildRequest) and shared initialization/upload logic.
  • Added “standard” concrete implementations (StandardBatchProducer / StandardBatchConsumer) preserving the existing NDJSON-on-disk + JSON-array POST behavior.
  • Updated BatchManager to construct producer/consumer via a factory and reorganized tests to cover base vs. standard behaviors.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/transport/batch/standard/StandardBatchProducer.ts Implements standard NDJSON file writing + size-based rotation on top of abstract producer.
src/transport/batch/standard/StandardBatchProducer.spec.ts Updates/extends producer tests for the new standard implementation and filename format.
src/transport/batch/standard/StandardBatchConsumer.ts Implements standard request construction (JSON array + DD-API-KEY header).
src/transport/batch/standard/StandardBatchConsumer.spec.ts Adds tests validating standard consumer request headers/body and file selection/order.
src/transport/batch/BatchProducer.ts Converts producer into an abstract base with shared init/recovery and unique filename generation.
src/transport/batch/BatchManager.ts Introduces factory-based producer/consumer creation; wires standard pair for now.
src/transport/batch/BatchManager.spec.ts Updates manager tests to mock standard producer/consumer and validate wiring/scheduling.
src/transport/batch/BatchConsumer.ts Converts consumer into an abstract base delegating request construction to buildRequest.
src/transport/batch/BatchConsumer.spec.ts Refactors tests to use a minimal concrete subclass to exercise base upload/send/delete logic.
src/transport/batch/batchConfig.types.ts Introduces a unified BatchConfig type used by BatchManager.create.

Comment thread src/transport/batch/BatchConsumer.spec.ts
Comment thread src/transport/batch/BatchProducer.ts
@cdn34dd cdn34dd force-pushed the carlosnogueira/RUM-16912/refactor-transport-layer-for-extensibility branch from dd5da55 to cd306bf Compare June 23, 2026 06:51
@cdn34dd cdn34dd merged commit a8c7b64 into main Jun 23, 2026
15 checks passed
@cdn34dd cdn34dd deleted the carlosnogueira/RUM-16912/refactor-transport-layer-for-extensibility branch June 23, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants