os/bluestore: drop deferred_submit_lock

This lock serves no purpose.  It only protects the call to the aio_submit
on the OpSequencer running batch, but we are the only caller who will do
that submission, and a new batch won't be moved into place until ours is
completed and retired (by the completion for the aio we are queueing).

More importantly, this fixes a deadlock:

- thread A submits aio, but the queue is full, and blocks
- thread B blocks taking deferred_submit_lock while holding deferred_lock
- aio completion thread blocks trying to take deferred_lock, and thus
no further aios are drained.

Fixes: http://tracker.ceph.com/issues/21171
Signed-off-by: Sage Weil <sage@redhat.com>

src/os/bluestore/BlueStore.cc

@@ -8697,8 +8697,6 @@ void BlueStore::_deferred_submit_unlock(OpSequencer *osr)
     ++i;
   }
 
-  // demote to deferred_submit_lock, then drop that too
-  std::lock_guard<std::mutex> l(deferred_submit_lock);
   deferred_lock.unlock();
   bdev->aio_submit(&b->ioc);
 }

src/os/bluestore/BlueStore.h

@@ -1841,7 +1841,7 @@ class BlueStore : public ObjectStore,
   interval_set<uint64_t> bluefs_extents;  ///< block extents owned by bluefs
   interval_set<uint64_t> bluefs_extents_reclaiming; ///< currently reclaiming
 
-  std::mutex deferred_lock, deferred_submit_lock;
+  std::mutex deferred_lock;
   std::atomic<uint64_t> deferred_seq = {0};
   deferred_osr_queue_t deferred_queue; ///< osr's with deferred io pending
   int deferred_queue_size = 0;         ///< num txc's queued across all osrs