Naiad: A Timely Dataflow System
- Is Naiad/timely dataflow in use in production environments anywhere? If not, and what would be barriers to adoption?
- The paper says that Naiad performs better than Spark. In what cases would one prefer which one over the other?
- What was the reasoning for implmenting Naiad in C#? It seems like this language choice introduced lots of pain, and performance would be better in C/C++.
- I'm confused by the OnRecv/OnNotify/SendBy/NotifyAt API. Does the user need to tell the system when to notify another vertex?
- What do the authors mean when they state systems like Spark "requires centralized modifications to the dataflow graph, which introduce substantial overhead"? Why does Spark perform worse than Naiad for iterative computations?