Resolve an O(n!)-ish behavior in work claiming. Previously, clusters with a large number of work units may have observed high levels of ZooKeeper activity due to unnecessary claim cycles being triggered. On launch, Ordasity's attempt to claim work results in VerifyIntegrityListener being fired, which triggers an additional recursive call to claimWork(). This cycle would continue until all work units were claimed. In a local test of 2600 work units, this took over an hour to complete and issued 3.5 million fruitless writes against ZK. Sincere apologies for this bug, which may have been with us since the very beginning.
Guard against a distributed race condition in which a node shuts down… … a work unit immediately after receiving handoff without deleting the ZNode. Implemented by performing one final check of a ZNode's ownership before shutting it down. This check requires making a GET call to ZK and introduces IO in a method I'd prefer there be none, but it is the final condition checked and is quite rare.