Skip to content

docs: Clarify input transformation support for EventBridge Pipes #287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 23, 2025
Merged

docs: Clarify input transformation support for EventBridge Pipes #287

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions src/content/docs/aws/services/pipes.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,11 +198,12 @@ or Amazon S3 logs.

The EventBridge Pipes implementation in LocalStack is currently in preview stage and has the following limitations:

* Lack of input transformers.
* Lack of concurrency support (i.e., ParallelizationFactor), resulting in slower processing in high-throughput scenarios.
* Lack of lifecycle management for pipe states (i.e., missing tests for state transitions).
* Lack of re-sharding support when polling from Kinesis and DynamoDB streams.
* Batch handling behavior may have parity issues (e.g., batch flushing rules by size, length, time, etc. are not implemented).
- Input transformation is supported **for targets only**. Enrichment input transformation is not yet available.
- Support for wildcards (`*`) in JSONPath in input transformation is not implemented.
- Lack of concurrency support (i.e., ParallelizationFactor), resulting in slower processing in high-throughput scenarios.
- Lack of lifecycle management for pipe states (i.e., missing tests for state transitions).
- Lack of re-sharding support when polling from Kinesis and DynamoDB streams.
Copy link
Contributor

@gregfurman gregfurman Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can rather phrase this as "limited support", since we do have re-sharding functionality... it's just rough around the edges 😅

- Batch handling behavior may have parity issues (e.g., batch flushing rules by size, length, time, etc. are not implemented).
Copy link
Contributor

@gregfurman gregfurman Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We handle batching and have quite a few parity tests AFAIK 🤔 Unsure if it's worth mentioning this here.

See the ESM docs https://docs.localstack.cloud/aws/services/lambda/#behaviour-coverage (the pipes implementation is based off of the ESM one).

Copy link

@tiurin tiurin Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gregfurman this line used to be in the docs before. This PR's change only clarifies input transformation. If batch behavior is not described correctly I'd suggest to raise a separate docs ticket about it and clarify batching behavior there, without blocking this PR.


## API Coverage

Expand Down