Skip to content

Add parallel_connections to unix_stream#1408

Closed
blt wants to merge 6 commits intomainfrom
blt/add_parallel_connections_to_unix_stream
Closed

Add parallel_connections to unix_stream#1408
blt wants to merge 6 commits intomainfrom
blt/add_parallel_connections_to_unix_stream

Conversation

@blt
Copy link
Copy Markdown
Collaborator

@blt blt commented Jul 3, 2025

What does this PR do?

This commit ports the behavior of unix_datagram to unix_stream. Each
parallel connection maintains its own separate block cache, meaning
for large numbers of parallel connections startup time may be high.
However users will no longer have to make independent unix_stream
generators within a single configuration.

With the parallel_connection addition to unix_stream we now must solve
the problem of high connection count causing non-trivial cache duplication.
This is done by allowing for a 'handle' on the cache which maintains a separate
index per handle, meaning we do not need to synchronize the cache indexes between
peers and peers are able to reference the singular cache.

Motivation

https://github.com/DataDog/saluki use-case

Copy link
Copy Markdown
Collaborator Author

blt commented Jul 3, 2025

@blt blt marked this pull request as ready for review July 3, 2025 17:22
@blt blt requested a review from a team as a code owner July 3, 2025 17:22
@blt blt force-pushed the blt/add_parallel_connections_to_unix_stream branch 2 times, most recently from a69f29c to 15e2eb6 Compare July 7, 2025 23:40
@blt blt changed the base branch from main to graphite-base/1408 July 7, 2025 23:46
@blt blt changed the base branch from graphite-base/1408 to blt/fix_dogstatsd_regression_test July 7, 2025 23:46
Base automatically changed from blt/fix_dogstatsd_regression_test to main July 8, 2025 00:03
@blt blt force-pushed the blt/add_parallel_connections_to_unix_stream branch from 2975308 to 2dc22d6 Compare July 8, 2025 00:04
@blt blt requested a review from scottopell July 8, 2025 00:30
blt added 6 commits July 9, 2025 11:13
This commit ports the behavior of unix_datagram to unix_stream. Each
parallel connection maintains its own separate block cache, meaning
for large numbers of parallel connections startup time may be high.
However users will no longer have to make independent unix_stream
generators within a single configuration.

Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
@blt blt force-pushed the blt/add_parallel_connections_to_unix_stream branch from 2dc22d6 to c7a3220 Compare July 9, 2025 18:14
@blt blt marked this pull request as draft July 10, 2025 01:06
@blt
Copy link
Copy Markdown
Collaborator Author

blt commented Jul 14, 2025

I merged the stack weirdly, closing to rescue the separate PRs.

@blt blt closed this Jul 14, 2025
blt added a commit that referenced this pull request Jul 14, 2025
This commit is a rescue of #1408
without the accidental merge-in of #1412.
The goal is to reach configuration pairity with unix_datagram.

Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants