Add logic to stop a model evaluation run#3421
Conversation
shati-patel
left a comment
There was a problem hiding this comment.
Thanks! In terms of sensible manual testing, things work generally as expected for me too.
(I haven't been able to test "Stop variant analysis from the variant analysis results view page" since we don't auto-open the view any more)
In terms of ✨ chaos ✨ testing: If I hammer the Evaluate/Stop Evaluation button enough, I can get into a place where everything lags, and then multiple evaluation runs happen and things get a bit confusing. It seems like we kick off multiple MRVA runs, but can only cancel the latest one. It's hard to see what's actually going on since we don't display the results anywhere, so I'm not too worried for now! 🙃
That's true, we do have a limitation that you can only cancel the last evaluation run (we create a new token so the old one gets disposed. That, in combination with the fact that query bundling isn't truly cancelable, make it laggy. Since we're working on a prototype phase I think this is okay for now. Thanks for testing! 😄 |
Updated the
ModelEvaluatorlogic to deal with stopping of an evaluation run. As part of this I extracted the logic that creates and cancels a variant analysis into a privaterunVariantAnalysisfunction.The code isn't super straight forward so I've added a bunch of comments.
I've also started adding some tests for the
ModelEvaluatorbut only focused on the stopping aspect for now.I've done the following manual tests:
Checklist
ready-for-doc-reviewlabel there.