Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
docs(SPEC): add error handling to the SPEC #586
There is a lot in the PR, to help focus the discussion let me explain a few things.
This PR uses a new sytnax for describing types, this was necessary to be able to talk about Flux types in any meaningful way. Please do not discuss/fret about the syntax being used. I'll explain it shortly here and then I will create a follow up PR that updates the grammar for this syntax where we can debate its merits there.
A quick overview of the type syntax. A type expression reads like any normal Flux expression except that it must always describe a literal value. A few new operators were added:
OK with that out of the way, just go with it. The purpose of this PR is to discuss how error handling works.
The basic concept is that each stream has two channels, a
aanthony1243 left a comment
I think we are like 90% of the way there, but this work will require a delicate balance between making error handling easy to use and not making generalized assumptions about how users want to handle errors. Overall, I like the design, and I can see how this will smoothly fit into our existing data/execution model which is good.
wolffcm left a comment
I dig it. Errors terminate by default, but if you want other behavior it is intuitive to get it. I guess that execution subsystem implicitly calls the error handler after each source or transform?
This looks great! However, I have some perplexities.
Suppose you have some function with side effects. With this model where you don't have an order relationship between
What is the purpose of making the
Maybe, what we need is simpler than this: a global error channel and one meta channel per transformation. If one wants to correlate stats/meta, he can extract every meta channel and perform joins, etc.
If one wants to populate the global error stream, he can write down the errorHandler so that it writes the error to the global error channel. We would need a
So, the error handler should be a function that takes an error as input
This is still fragile, because in the case of task parallelism, we cannot even unsure that while you get an error, another stage of the data processing pipeline has started. Se it could be that the error is processed after that some upstream transformation has started operating on fresh data.
Example of global error channel: