-
Couldn't load subscription status.
- Fork 232
[Feat] Error handler in streams #451
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Other ecosystem call it https://doc.rust-lang.org/std/result/enum.Result.html#method.inspect_err Eitherway, fine by me |
In this case, side effects aren't necessarily linked to an error; side effects in this context are any operation that shouldn't affect the flow. Think of a call to an external system (e.g., sending an email) that shouldn't affect the processing of the main flow, and is therefore a side effect. Then |
Co-authored-by: Paulo Valente <16843419+polvalente@users.noreply.github.com>
Co-authored-by: Paulo Valente <16843419+polvalente@users.noreply.github.com>
Co-authored-by: Paulo Valente <16843419+polvalente@users.noreply.github.com>
…into feat/error-handler
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚀 💜
| GRPC.Stream.unary(request, materializer: materializer) | ||
| |> GRPC.Stream.map(fn %HelloReply{} = reply -> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| GRPC.Stream.unary(request, materializer: materializer) | |
| |> GRPC.Stream.map(fn %HelloReply{} = reply -> | |
| request | |
| |> GRPC.Stream.unary(materializer: materializer) | |
| |> GRPC.Stream.map(fn %HelloReply{} = reply -> |
| In this example: | ||
|
|
||
| * The function inside `map/2` raises an exception for the value `2`. | ||
| * map_error/2 captures and transforms that error into a structured `GRPC.RPCError` response. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| * map_error/2 captures and transforms that error into a structured `GRPC.RPCError` response. | |
| * `map_error/2` captures and transforms that error into a structured `GRPC.RPCError` response. |
| end | ||
|
|
||
| @doc """ | ||
| Applies a **side-effecting function** to each element of the stream **without altering** its values. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| Applies a **side-effecting function** to each element of the stream **without altering** its values. | |
| Applies a side-effect function to each element of the stream without altering its values. |
Let's remove all of these bold highlights
| ## Returns | ||
| - Updated stream if successful. | ||
| - `{:error, item, reason}` if the request fails or times out. | ||
| - `{:error, reason}` if the request fails or times out. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a bit redundant with the typespec
| GRPCStream.t() | {:error, :timeout | :process_not_alive} | ||
| def ask(%GRPCStream{flow: flow} = stream, target, timeout \\ 5000) do | ||
| mapper = fn item -> do_ask(item, target, timeout, raise_on_error: false) end | ||
| # mapper = fn item -> do_ask(item, target, timeout, raise_on_error: false) end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
stray comment
| %GRPCStream{ | ||
| stream | ||
| | flow: | ||
| Flow.map(flow, fn flow_item -> | ||
| tap(flow_item, fn item -> safe_invoke(effect_fun, item) end) | ||
| end) | ||
| } | ||
| end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's break this up a bit for readability
| Flow.map(flow, fn | ||
| {:error, _reason} = item -> | ||
| res = safe_invoke(func, item) | ||
|
|
||
| case res do | ||
| {:error, %GRPC.RPCError{} = new_reason} -> | ||
| {:error, new_reason} | ||
|
|
||
| {:error, new_reason} -> | ||
| msg = "#{inspect(new_reason)}" | ||
| {:error, GRPC.RPCError.exception(message: msg)} | ||
|
|
||
| {:ok, other} -> | ||
| other | ||
|
|
||
| other -> | ||
| other | ||
| end | ||
|
|
||
| {:ok, other} -> | ||
| other | ||
|
|
||
| other -> | ||
| other | ||
| end) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we extract this to a defp?
| Stream.cycle(1..5) | ||
| |> Stream.take(5) | ||
| |> GRPC.Stream.from() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| Stream.cycle(1..5) | |
| |> Stream.take(5) | |
| |> GRPC.Stream.from() | |
| 1..5 | |
| |> GRPC.Stream.from() |
Doesn't this work? If not, we should make it work in a separate PR. Suggestion for if enumerables aren't allowed directly:
| Stream.cycle(1..5) | |
| |> Stream.take(5) | |
| |> GRPC.Stream.from() | |
| 1..5 | |
| |> Stream.take(5) | |
| |> GRPC.Stream.from() |
| describe "ask/3 error handling" do | ||
| test "returns timeout error if response not received in time" do | ||
| pid = | ||
| spawn(fn -> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| spawn(fn -> | |
| spawn_link(fn -> |
| # do not send any response | ||
| receive do | ||
| _ -> :ok | ||
| end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| # do not send any response | |
| receive do | |
| _ -> :ok | |
| end | |
| Process.sleep(:infinity) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also instead of spawning we could have self() be the pid in ask and maybe even have an assert_receive
🎯 Context
This PR introduces a new error handling and side-effect model for the
GRPC.Streammodule, covering both unary and streaming flows.The implementation improves safety, consistency, and expressiveness across Flow-based pipelines used in gRPC message processing, ensuring that exceptions, errors, and inconsistent return values are handled uniformly and predictably.
🚀 New Features
🧩 1. effect/2
A new utility function to safely perform side effects within a stream pipeline without modifying the values being processed.
⚙️ 2. map_error/2
Adds a declarative mechanism for mapping or transforming errors (
{:error, reason}) within a stream into structuredGRPC.RPCErrorvalues.Behavior:
This enables localized error recovery and translation, useful for both input validation and unexpected runtime exceptions.
🔄 3. Unified Error Matching and Propagation
All stream operators now use the internal safe_invoke/2 wrapper, which standardizes how functions inside the pipeline behave.
This makes all pipelines exception-safe — any operator can raise or throw, but the stream will continue gracefully with well-defined {:error, reason} items.
💡 4. Integration with run/1 and run_with/3
Both stream finalizers (run/1, run_with/3) have been updated to:
✅ 5. Test Coverage
New and extended tests added in GRPC.StreamTest include:
effect/2: ensures side effects are applied safely and do not affect data flow, even when exceptions occur inside the callback.map_error/2: verifies correct error mapping and recovery.map,flat_map,filter,ask, etc.).:process_not_alive,:timeout) inask/3.🧠 Impact
This refactor:
🔮 Next Steps