Error handling

[bachdavi]

Currently we panic if anything is out of order, which leads to the shutdown of the process.
In the future we need a resilient structure that notifies the clients via the web socket that some computation in a single dataflow went crazy, but continues the others.

[comnik]

It would be nice to send general command feedback (e.g. 'success!') via the WS connection as well.

[theronic]

Super excited for this project! I think it's the most important work affecting the Clojure ecosystem right now.

panic! on user error seems like the lowest-hanging issue preventing people from building up real-world usage of the broker (even if it is not production-ready as-is).

[comnik]

Thank you for the encouraging words. It is unfortunately not as low-hanging as one would hope, because we need to make sure that changes to the overall dataflow are atomic. But working a bit on this now (6215894).

[theronic]

To elaborate why panic! on invalid error is a nefarious usability issue: at this stage it only affects people who are simultaneously willing to tolerate bugs and who's eyes and encouragement are most valuable to the project. Strained by their day-jobs, folks are less likely to keep eyes on it if they have to keep restarting the broker (despite being incomplete). Maybe I'm wrong about that :).

Either way, good luck @comnik! Do reach out if and when you're ever interested in pursuing a commercial application - graph-based spreadsheets being the most saleable application that requires defensible UI work.

[comnik]

@theronic We are looking into commercial applications – happy to talk!

[comnik]

The server should now be returning error messages on all non-critical errors. It is not perfect yet, because we have to rely on checking potentially fallible things before synthesis. Once we've committed to synthesising a rule, the server will panic on any exceptions, to avoid inconsistent dataflow states.

Closing this issue now. New, more specific issues should be created once we uncover problematic cases.

[theronic]

Server now seems to halt processing silently after any error, e.g. reasserting the same attribute :/. I lost half a day trying to debug why my code wasn't triggering queries. Maybe I'm doing something wrong?

[bachdavi]

I had a look into it and it turns out its just a typo in the error channel when reregistering web socket connections.

Progress was made but the channel to send results to clients was in some sort of weird loop because we used the results channel instead of the error channel.

I'll fix it now.

Can you tell me if things work as expected?

[comnik]

@theronic re lost time spent debugging: we could do something like let the server send ack's on any command?