Writer Daemon freezing up

[istathar]

Something about ff7bf80 has caused the writer daemon to deadlock (?). Maybe it's only getting one capability, thereby messing up the multi-threaded runtime... but that's not supposed to happen.

AfC

[christian-marie]

I'll improve the robustness here as part of libraritising the storage back end.

This was a ceph lock that got stuck on.

[istathar]

@christian-marie I didn't get a chance to thank you for your hard work last night tracking this down.

Obviously locking isn't supposed to bite us like this; as far as I know we never had such a problem in v1 so it will be interesting to see what behaviour assumptions we've made recently turned out to be wrong. That said, the codebase has been refactored so much (which is good) that previous expectations are really out the window.

It'll be good to review the design around the daymaps et al.

I daresay close this one when we can run multiple writers concurrently again.

AfC