Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[v23.3.x] transform: create smp and scheduling groups #16139

Merged
merged 8 commits into from
Jan 19, 2024
Merged

[v23.3.x] transform: create smp and scheduling groups #16139

merged 8 commits into from
Jan 19, 2024

Commits on Jan 17, 2024

  1. rpc: pass requests by value 

    Pass RPC requests by value, as we start to write some handlers using
coroutines, it can be a footgun to access have a suspension point, then
the request is destroyed. An example that motivated this:

```c++
ss::future<reply> service::handle(request&& r, rpc::streaming_context&) {
  co_await ss::coroutine::switch_to(get_scheduling_group());
  co_return co_await handle_request(std::move(r));
}
```

In this case `r` is destroyed because the coroutine does not own the
value, and handle request would have a stack-use-after-return issue.

Now that we pass by value, we don't have to remember to move the value
into the current method.

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
(cherry picked from commit 5becd89)
    @rockwotj
    rockwotj committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    0eebe15 View commit details
    Browse the repository at this point in the history

  2. ssx/work_queue: support scheduling group 

    Add a scheduling group parameter for all the work executed on the queue.

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
(cherry picked from commit 566e2b1)
    @rockwotj
    rockwotj committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    e227778 View commit details
    Browse the repository at this point in the history

  3. resource_mgmt/cpu: add transform scheduling group 

    Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
(cherry picked from commit 4b3251b)
    @rockwotj
    rockwotj committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    4c2c5d0 View commit details
    Browse the repository at this point in the history

  4. transform: run transforms on a scheduling group 

    Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
(cherry picked from commit 6928acc)
    @rockwotj
    rockwotj committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    8c5a9f9 View commit details
    Browse the repository at this point in the history

  5. transform/rpc: use scheduling group 

    When RPCs come in for transform requests, ensure that they are running
on the transform scheduling groups. Node local service requests are
assumed to be already running on the transform scheduling group, which
is mostly true except for deploys.

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
(cherry picked from commit 895af5f)
    @rockwotj
    rockwotj committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    d286bb2 View commit details
    Browse the repository at this point in the history

  6. resource_mgmt/smp: add smp group for transforms 

    Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
(cherry picked from commit d018c59)
    @rockwotj
    rockwotj committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    ce0e4ec View commit details
    Browse the repository at this point in the history

  7. transform/rpc: use smp_group for cross shard calls 

    The transform/rpc subsystem makes cross shard calls when accessing the
correct partition for a transform, use a `smp_group` to manage this,
like other subsystems.

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
(cherry picked from commit cb310c5)
    @rockwotj
    rockwotj committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    b52f272 View commit details
    Browse the repository at this point in the history

  8. resource_mgmt/smp: seperate implementation 

    Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
(cherry picked from commit 99a3ece)
    @rockwotj
    rockwotj committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    c8171d4 View commit details
    Browse the repository at this point in the history