`thingbuf::mpsc::Sender` hanging up for parallel `try_send_ref` and `send` / `send_ref` from sync thread and async `tokio::task`

[sgasse]

Hi! We are using thingbuf in a performance-sensitive application for its speed while supporting sync and async interaction on the same sender type. Recently I ran into an issue where thingbuf seems to hang up our async application completely in busy loops. The profiling in the degraded state showed almost only calls to thingbuf.

I was able to reproduce the issue in a minimum example. You can run several variations of it yourself by checking out this repo:
https://github.com/sgasse/thingbuf_hangup/

The initial setup (binary thingbuf_sendref) to get into the hangup was this:

• One std::thread sends with try_send_ref in a loop every 10ms.
• One tokio::task receives on the channel with recv_ref().await in a loop. After 10s, there is a delay of 10s introduced between receive calls. This simulates badly handled backpressure from a downstream task.
• One tokio::task starts sending with send(..).await in a loop after 20s.
• One tokio::task logs an alive message every second.

Once the second sender becomes active, we no longer see any alive logs. Introducing logs to thingbuf shows that two threads (one tokio worker and the self-spawned thread) are both stuck in this loop in push_ref.

Here is some log output with line numbers from src/lib.rs from thingbuf for the hang-up scenario, this is infinitely:

...
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 202: push_ref loop starts
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 294: if
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 227: idx: 77, state: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 301: tail at the end of the loop: 7757
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 286: head: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 202: push_ref loop starts
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 294: if
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 227: idx: 77, state: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 301: tail at the end of the loop: 7757
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 286: head: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 202: push_ref loop starts
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 294: if
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 227: idx: 77, state: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 301: tail at the end of the loop: 7757
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 286: head: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 202: push_ref loop starts
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 294: if
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 227: idx: 77, state: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 301: tail at the end of the loop: 7757
...

The initial setup mimics the behavior of a part of our real application. However I varied the setup in other examples, here are some findings:

• Whether we try_send_ref() (sync), send_ref().await() or send(..).await (async) does not seem to matter, see examples thingbuf_sendref, thingbuf_sendref_pure, thingbuf_send_recvref and thingbuf_send_no_try_recvref which all hang-up.
• When using recv().await instead of recv_ref().await, there is no hangup, see example thingbuf_send.
• Switching from 1 worker thread to e.g. 3 sometimes leads to a few more logs printed immediately after the second sender starts but then, the example also hangs up.
• My system is x86_64-unknown-linux-gnu, but I initially found it on aarch64-linux-android so I guess it does not depend on the platform.
• The issue is reproducible with both rustc in version 1.76 and nightly, so probably not related to the compiler.
• To check that there is nothing obviously wrong with the example setup, I replaced thingbuf::mpsc with tokio::mpsc in one example, which works as expected: It still logs the alive messages and does not hang up.

I would have expected the same behavior as I see with tokio::mpsc, so I guess it is a bug. But please let me know if there is a limitation which I overlooked or if I can provide further info.

[niceskylei]

I encountered the same problem.

[fslongjin]

We are also using StaticThingbuf in the DragonOS kernel and have encountered the exact same problem. The issue appears randomly and is difficult to reproduce after the system kernel is restarted.

In our use case:

• https://code.dragonos.org.cn/xref/DragonOS/kernel/src/debug/klog/mm.rs?r=b8ed38251dc255b0c525801b5dbf37d3b0d0d61e#48
• https://code.dragonos.org.cn/xref/DragonOS/kernel/crates/klog_types/src/lib.rs?r=840045af94ea3391f29e87e968db5d9c48316981#189

Once the buffer is full, there is a very sluggish phenomenon. After a period of time, the performance drops off a cliff. Ultimately, it freezes at the push_ref.

[hawkw]

Okay, v0.1.5 has been published to crates.io, including @tukan's changes from PR #81, which should fix this.

Let me know if you continue seeing performance issues after updating to v0.1.5!

[xiaolin2004]

After updating the repository, the performance issue still remains unresolved. Below is the backtrace of the program execution:

The push_ref() method eventually hangs at:

We made some minor modifications to the code to adapt it to our requirements. Below is the repository link after merging the latest version:https://github.com/xiaolin2004/thingbuf.git

[hawkw]

Hmm, that's unfortunate. Thanks for following up --- I'll keep looking.

[hawkw]

It looks like, after 11089327 iterations under loom, the tests added in PR #81 seem to enter into an infinite spin loop.

Full logs from that iteration

---- mpsc::tests::mpsc_blocking::mpsc_test_skip_slot iteration 11089327 ----
[ThreadId(0)           src/mpsc/blocking.rs:1612] self.inner.core.tx_count.fetch_add(1, Ordering::Relaxed) = 1
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 0
[ThreadId(0)                     src/lib.rs:326] idx = 0
[ThreadId(0)                     src/lib.rs:327] gen = 0
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 0
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 0
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
 INFO iter{11089327}:thread{id=1}: loom::rt::execution: ~~~~~~~~ THREAD 1 ~~~~~~~~
[ThreadId(1)                     src/lib.rs:201] push_ref
[ThreadId(1)                     src/lib.rs:203] self.tail.load(Relaxed) = 0
[ThreadId(1)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(1)                     src/lib.rs:209] idx = 0
[ThreadId(1)                     src/lib.rs:210] gen = 0
[ThreadId(1)                     src/lib.rs:228] slot.state.load(SeqCst) = 0
[ThreadId(1)                     src/lib.rs:229] clear_has_reader(raw_state) = 0
[ThreadId(1)                     src/lib.rs:231] state == tail = true
[ThreadId(1)                     src/lib.rs:234] self.tail.compare_exchange_weak(tail, next_tail, SeqCst, Acquire) = Ok(0)
[ThreadId(1)                     src/lib.rs:238] check_has_reader(raw_state) = false
[ThreadId(1)                     src/lib.rs:257] advanced tail 0 to 1; claimed slot [0]
[ThreadId(1)                     src/lib.rs:272] -> initialized
[ThreadId(1)                     src/lib.rs:560] drop Ref<usize> (push), new_state = 1
[ThreadId(1)                     src/lib.rs:565] self.new_state = 1
[ThreadId(1)                     src/lib.rs:565] self.slot.state.store(test_dbg!(self.new_state), Release) = ()
[ThreadId(1)                    src/mpsc.rs:562] notifying rx (loom::thread::Thread)
[ThreadId(1)               src/wait/cell.rs:161] notifying; close=WAITING;
[ThreadId(1)               src/wait/cell.rs:163] bits = NOTIFYING
[ThreadId(1)               src/wait/cell.rs:164] self.fetch_or(bits, AcqRel) = WAITING
[ThreadId(1)               src/wait/cell.rs:168] self.fetch_and(!State::NOTIFYING, AcqRel) = NOTIFYING
[ThreadId(1)               src/wait/cell.rs:170] waiter = None
 INFO iter{11089327}:thread{id=2}: loom::rt::execution: ~~~~~~~~ THREAD 2 ~~~~~~~~
[ThreadId(2)                     src/lib.rs:201] push_ref
[ThreadId(2)                     src/lib.rs:203] self.tail.load(Relaxed) = 0
[ThreadId(2)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(2)                     src/lib.rs:209] idx = 0
[ThreadId(2)                     src/lib.rs:210] gen = 0
[ThreadId(2)                     src/lib.rs:228] slot.state.load(SeqCst) = 0
[ThreadId(2)                     src/lib.rs:229] clear_has_reader(raw_state) = 0
[ThreadId(2)                     src/lib.rs:231] state == tail = true
[ThreadId(2)                     src/lib.rs:234] self.tail.compare_exchange_weak(tail, next_tail, SeqCst, Acquire) = Err(1)
[ThreadId(2)                     src/lib.rs:290] failed to advance tail 0 to 1
[ThreadId(2)                    src/util.rs:40 ] hint::spin_loop() (x1)
 INFO iter{11089327}:thread{id=0}: loom::rt::execution: ~~~~~~~~ THREAD 0 ~~~~~~~~
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 0
[ThreadId(0)                     src/lib.rs:326] idx = 0
[ThreadId(0)                     src/lib.rs:327] gen = 0
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 1
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = true
[ThreadId(0)                     src/lib.rs:352] self.head.compare_exchange_weak(head, next_head, SeqCst, Acquire) = Ok(0)
[ThreadId(0)                     src/lib.rs:357] advanced head 0 to 1
[ThreadId(0)                     src/lib.rs:358] claimed slot [0]
[ThreadId(0)                     src/lib.rs:361] new_state = 9223372036854775816
[ThreadId(0)                     src/lib.rs:361] slot.state.store(test_dbg!(new_state), SeqCst) = ()
[ThreadId(0)                     src/lib.rs:557] drop Ref<usize> (pop)
 INFO iter{11089327}:thread{id=2}: loom::rt::execution: ~~~~~~~~ THREAD 2 ~~~~~~~~
[ThreadId(2)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(2)                     src/lib.rs:209] idx = 1
[ThreadId(2)                     src/lib.rs:210] gen = 0
[ThreadId(2)                     src/lib.rs:228] slot.state.load(SeqCst) = 1
[ThreadId(2)                     src/lib.rs:229] clear_has_reader(raw_state) = 1
[ThreadId(2)                     src/lib.rs:231] state == tail = true
[ThreadId(2)                     src/lib.rs:234] self.tail.compare_exchange_weak(tail, next_tail, SeqCst, Acquire) = Ok(1)
[ThreadId(2)                     src/lib.rs:238] check_has_reader(raw_state) = false
[ThreadId(2)                     src/lib.rs:257] advanced tail 1 to 8; claimed slot [1]
[ThreadId(2)                     src/lib.rs:272] -> initialized
[ThreadId(2)                     src/lib.rs:560] drop Ref<usize> (push), new_state = 2
[ThreadId(2)                     src/lib.rs:565] self.new_state = 2
[ThreadId(2)                     src/lib.rs:565] self.slot.state.store(test_dbg!(self.new_state), Release) = ()
[ThreadId(2)                    src/mpsc.rs:562] notifying rx (loom::thread::Thread)
[ThreadId(2)               src/wait/cell.rs:161] notifying; close=WAITING;
[ThreadId(2)               src/wait/cell.rs:163] bits = NOTIFYING
[ThreadId(2)               src/wait/cell.rs:164] self.fetch_or(bits, AcqRel) = WAITING
[ThreadId(2)               src/wait/cell.rs:168] self.fetch_and(!State::NOTIFYING, AcqRel) = NOTIFYING
[ThreadId(2)               src/wait/cell.rs:170] waiter = None
 INFO iter{11089327}:thread{id=1}: loom::rt::execution: ~~~~~~~~ THREAD 1 ~~~~~~~~
[ThreadId(1)                     src/lib.rs:201] push_ref
[ThreadId(1)                     src/lib.rs:203] self.tail.load(Relaxed) = 8
[ThreadId(1)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(1)                     src/lib.rs:209] idx = 0
[ThreadId(1)                     src/lib.rs:210] gen = 8
[ThreadId(1)                     src/lib.rs:228] slot.state.load(SeqCst) = 9223372036854775816
[ThreadId(1)                     src/lib.rs:229] clear_has_reader(raw_state) = 8
[ThreadId(1)                     src/lib.rs:231] state == tail = true
[ThreadId(1)                     src/lib.rs:234] self.tail.compare_exchange_weak(tail, next_tail, SeqCst, Acquire) = Ok(8)
[ThreadId(1)                     src/lib.rs:238] check_has_reader(raw_state) = true
[ThreadId(1)                     src/lib.rs:239] advanced tail 8 to 9; has an active reader, skipping slot [0]
[ThreadId(1)                     src/lib.rs:246] slot.state.fetch_update(SeqCst, SeqCst,
        |state|
            {
                Some(state & HAS_READER | next_state)
            }).unwrap_or_else(|_| unreachable!()) = 9223372036854775816
[ThreadId(1)                    src/util.rs:40 ] hint::spin_loop() (x1)
 INFO iter{11089327}:thread{id=0}: loom::rt::execution: ~~~~~~~~ THREAD 0 ~~~~~~~~
[ThreadId(0)                     src/lib.rs:558] self.slot.state.fetch_and(!HAS_READER, SeqCst) = 9223372036854775824
[ThreadId(0)                    src/mpsc.rs:570] notifying tx (loom::thread::Thread)
[ThreadId(0)              src/wait/queue.rs:371] WaitQueue::notify()
[ThreadId(0)              src/wait/queue.rs:377] state = 0
[ThreadId(0)              src/wait/queue.rs:378] self.state.compare_exchange_weak(state, WAKING, SeqCst, SeqCst) = Ok(0)
 INFO iter{11089327}:thread{id=1}: loom::rt::execution: ~~~~~~~~ THREAD 1 ~~~~~~~~
[ThreadId(1)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(1)                     src/lib.rs:209] idx = 0
[ThreadId(1)                     src/lib.rs:210] gen = 8
[ThreadId(1)                     src/lib.rs:228] slot.state.load(SeqCst) = 16
[ThreadId(1)                     src/lib.rs:229] clear_has_reader(raw_state) = 16
[ThreadId(1)                     src/lib.rs:231] state == tail = false
[ThreadId(1)                     src/lib.rs:306] self.head.fetch_or(0, SeqCst) = 1
[ThreadId(1)                     src/lib.rs:307] wrapping_add(head, self.gen) == tail = false
[ThreadId(1)                    src/util.rs:57 ] hint::spin_loop() (x2)
[ThreadId(1)                    src/util.rs:60 ] thread::yield_now()
 INFO iter{11089327}:thread{id=2}: loom::rt::execution: ~~~~~~~~ THREAD 2 ~~~~~~~~
[ThreadId(2)                     src/lib.rs:201] push_ref
[ThreadId(2)                     src/lib.rs:203] self.tail.load(Relaxed) = 9
[ThreadId(2)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(2)                     src/lib.rs:209] idx = 1
[ThreadId(2)                     src/lib.rs:210] gen = 8
[ThreadId(2)                     src/lib.rs:228] slot.state.load(SeqCst) = 2
[ThreadId(2)                     src/lib.rs:229] clear_has_reader(raw_state) = 2
[ThreadId(2)                     src/lib.rs:231] state == tail = false
[ThreadId(2)                     src/lib.rs:306] self.head.fetch_or(0, SeqCst) = 1
[ThreadId(2)                     src/lib.rs:307] wrapping_add(head, self.gen) == tail = true
[ThreadId(2)                     src/lib.rs:308] channel full
[ThreadId(2)              src/wait/queue.rs:189] WaitQueue::start_wait(0x7f3475749900)
[ThreadId(2)              src/wait/queue.rs:193] self.state.compare_exchange(WAKING, EMPTY, SeqCst, SeqCst) = Ok(2)
[ThreadId(2)           src/mpsc/blocking.rs:2163] core.tx_wait.start_wait(node, &thread) = Notified
[ThreadId(2)                    src/util.rs:57 ] hint::spin_loop() (x1)
[ThreadId(2)                    src/util.rs:60 ] thread::yield_now()
 INFO iter{11089327}:thread{id=1}: loom::rt::execution: ~~~~~~~~ THREAD 1 ~~~~~~~~
[ThreadId(1)                     src/lib.rs:313] self.tail.load(Acquire) = 9
[ThreadId(1)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(1)                     src/lib.rs:209] idx = 1
[ThreadId(1)                     src/lib.rs:210] gen = 8
[ThreadId(1)                     src/lib.rs:228] slot.state.load(SeqCst) = 1
[ThreadId(1)                     src/lib.rs:229] clear_has_reader(raw_state) = 1
[ThreadId(1)                     src/lib.rs:231] state == tail = false
[ThreadId(1)                     src/lib.rs:306] self.head.fetch_or(0, SeqCst) = 1
[ThreadId(1)                     src/lib.rs:307] wrapping_add(head, self.gen) == tail = true
[ThreadId(1)                     src/lib.rs:308] channel full
[ThreadId(1)              src/wait/queue.rs:189] WaitQueue::start_wait(0x7f3475752900)
[ThreadId(1)              src/wait/queue.rs:193] self.state.compare_exchange(WAKING, EMPTY, SeqCst, SeqCst) = Err(0)
[ThreadId(1)              src/wait/queue.rs:209] WaitQueue::start_wait_slow(0x7f3475752900)
[ThreadId(1)    src/util/mutex/loom_impl.rs:15 ] locking thingbuf::wait::queue::List<loom::thread::Thread>...
[ThreadId(1)    src/util/mutex/loom_impl.rs:17 ] -> locked thingbuf::wait::queue::List<loom::thread::Thread>!
[ThreadId(1)              src/wait/queue.rs:221] state = 0
[ThreadId(1)              src/wait/queue.rs:225] self.state.compare_exchange_weak(EMPTY, WAITING, SeqCst, SeqCst) = Ok(0)
[ThreadId(1)              src/wait/queue.rs:280] node.state.swap(WAITING, Release) = 0
[ThreadId(1)              src/wait/queue.rs:549] List::enqueue(0x7f3475752900)
[ThreadId(1)           src/mpsc/blocking.rs:2163] core.tx_wait.start_wait(node, &thread) = Wait
 INFO iter{11089327}:thread{id=0}: loom::rt::execution: ~~~~~~~~ THREAD 0 ~~~~~~~~
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 1
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 0
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 2
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = true
 INFO iter{11089327}:thread{id=2}: loom::rt::execution: ~~~~~~~~ THREAD 2 ~~~~~~~~
[ThreadId(2)                     src/lib.rs:201] push_ref
[ThreadId(2)                     src/lib.rs:203] self.tail.load(Relaxed) = 9
[ThreadId(2)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(2)                     src/lib.rs:209] idx = 1
[ThreadId(2)                     src/lib.rs:210] gen = 8
[ThreadId(2)                     src/lib.rs:228] slot.state.load(SeqCst) = 2
[ThreadId(2)                     src/lib.rs:229] clear_has_reader(raw_state) = 2
[ThreadId(2)                     src/lib.rs:231] state == tail = false
[ThreadId(2)                     src/lib.rs:306] self.head.fetch_or(0, SeqCst) = 1
[ThreadId(2)                     src/lib.rs:307] wrapping_add(head, self.gen) == tail = true
[ThreadId(2)                     src/lib.rs:308] channel full
[ThreadId(2)              src/wait/queue.rs:189] WaitQueue::start_wait(0x7f3475749900)
[ThreadId(2)              src/wait/queue.rs:193] self.state.compare_exchange(WAKING, EMPTY, SeqCst, SeqCst) = Err(1)
[ThreadId(2)              src/wait/queue.rs:209] WaitQueue::start_wait_slow(0x7f3475749900)
[ThreadId(2)    src/util/mutex/loom_impl.rs:15 ] locking thingbuf::wait::queue::List<loom::thread::Thread>...
[ThreadId(2)    src/util/mutex/loom_impl.rs:17 ] -> locked thingbuf::wait::queue::List<loom::thread::Thread>!
[ThreadId(2)              src/wait/queue.rs:221] state = 1
[ThreadId(2)              src/wait/queue.rs:280] node.state.swap(WAITING, Release) = 0
[ThreadId(2)              src/wait/queue.rs:549] List::enqueue(0x7f3475749900)
[ThreadId(2)           src/mpsc/blocking.rs:2163] core.tx_wait.start_wait(node, &thread) = Wait
 INFO iter{11089327}:thread{id=0}: loom::rt::execution: ~~~~~~~~ THREAD 0 ~~~~~~~~
[ThreadId(0)                     src/lib.rs:352] self.head.compare_exchange_weak(head, next_head, SeqCst, Acquire) = Ok(1)
[ThreadId(0)                     src/lib.rs:357] advanced head 1 to 8
[ThreadId(0)                     src/lib.rs:358] claimed slot [1]
[ThreadId(0)                     src/lib.rs:361] new_state = 9223372036854775817
[ThreadId(0)                     src/lib.rs:361] slot.state.store(test_dbg!(new_state), SeqCst) = ()
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 8
[ThreadId(0)                     src/lib.rs:326] idx = 0
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 16
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = false
[ThreadId(0)                     src/lib.rs:396] raw_state == head = false
[ThreadId(0)                     src/lib.rs:405] self.head.compare_exchange(head, next_head, SeqCst, Acquire) = Ok(8)
[ThreadId(0)                     src/lib.rs:407] skipped head slot [0], new head=9
[ThreadId(0)                     src/lib.rs:324] head = 8
[ThreadId(0)                     src/lib.rs:326] idx = 0
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 16
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = false
[ThreadId(0)                     src/lib.rs:396] raw_state == head = false
[ThreadId(0)                     src/lib.rs:405] self.head.compare_exchange(head, next_head, SeqCst, Acquire) = Err(9)
[ThreadId(0)                     src/lib.rs:410] failed to skip head slot [0], head=8, actual=9
[ThreadId(0)                    src/util.rs:40 ] hint::spin_loop() (x1)
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 9223372036854775817
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = false
[ThreadId(0)                     src/lib.rs:385] self.tail.fetch_or(0, SeqCst) = 9
[ThreadId(0)                     src/lib.rs:386] tail & !self.closed == head = true
[ThreadId(0)                     src/lib.rs:387] tail & self.closed != 0 = false
[ThreadId(0)                     src/lib.rs:390] --> channel empty!
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
thread 'mpsc::tests::mpsc_blocking::mpsc_test_skip_slot' panicked at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/path.rs:204:13:
Model exceeded maximum number of branches. This is often caused by an algorithm requiring the processor to make progress, e.g. spin locks.
stack backtrace:
   0: std::panicking::begin_panic
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:686:12
   1: loom::rt::path::Path::branch_thread
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/path.rs:204:13
   2: loom::rt::execution::Execution::schedule
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/execution.rs:188:20
   3: loom::rt::branch::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/mod.rs:120:22
   4: loom::rt::scheduler::Scheduler::with_execution::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/scheduler.rs:48:34
   5: loom::rt::scheduler::Scheduler::with_state::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/scheduler.rs:130:28
   6: scoped_tls::ScopedKey<T>::with
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/scoped-tls-1.0.1/src/lib.rs:171:13
   7: loom::rt::scheduler::Scheduler::with_state
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/scheduler.rs:130:9
   8: loom::rt::scheduler::Scheduler::with_execution
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/scheduler.rs:48:9
   9: loom::rt::execution
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/mod.rs:171:5
  10: loom::rt::branch
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/mod.rs:118:25
  11: loom::rt::object::Ref<T>::branch_action
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/object.rs:351:9
  12: loom::rt::atomic::Atomic<T>::branch
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/atomic.rs:389:11
  13: loom::rt::atomic::Atomic<T>::load
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/atomic.rs:229:9
  14: loom::sync::atomic::atomic::Atomic<T>::load
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/sync/atomic/atomic.rs:28:9
  15: loom::sync::atomic::int::AtomicUsize::load
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/sync/atomic/int.rs:58:17
  16: thingbuf::Core::pop_ref
             at ./src/lib.rs:346:39
  17: thingbuf::mpsc::ChannelCore<N>::try_recv_ref
             at ./src/mpsc.rs:383:9
  18: thingbuf::mpsc::blocking::Receiver<T,R>::try_recv_ref
             at ./src/mpsc/blocking.rs:1888:9
  19: thingbuf::mpsc::tests::mpsc_blocking::mpsc_test_skip_slot::{{closure}}
             at ./src/mpsc/tests/mpsc_blocking.rs:111:19
  20: thingbuf::loom::inner::run_builder::{{closure}}
             at ./src/loom.rs:125:13
  21: loom::model::Builder::check::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/model.rs:185:17
  22: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
  23: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/boxed.rs:2015:9
  24: loom::rt::scheduler::spawn_threads::{{closure}}::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/scheduler.rs:149:21
  25: generator::gen_impl::GeneratorImpl<A,T>::init_code::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/generator-0.7.5/src/gen_impl.rs:336:21
  26: generator::stack::StackBox<F>::call_once
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/generator-0.7.5/src/stack/mod.rs:139:13
  27: generator::stack::Func::call_once
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/generator-0.7.5/src/stack/mod.rs:121:9
  28: generator::gen_impl::gen_init::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/generator-0.7.5/src/gen_impl.rs:552:9
  29: core::ops::function::FnOnce::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
  30: std::panicking::try::do_call
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
  31: std::panicking::try
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
  32: std::panic::catch_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
  33: generator::gen_impl::catch_unwind_filter
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/generator-0.7.5/src/gen_impl.rs:543:5
  34: generator::gen_impl::gen_init
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/generator-0.7.5/src/gen_impl.rs:570:25
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
[PANIC                     src/lib.rs:557] drop Ref<usize> (pop)
[PANIC                     src/lib.rs:558] self.slot.state.fetch_and(!HAS_READER, SeqCst) = 9223372036854775817
[PANIC                    src/mpsc.rs:570] notifying tx (loom::thread::Thread)
[PANIC              src/wait/queue.rs:371] WaitQueue::notify()
[PANIC              src/wait/queue.rs:377] state = 0
[PANIC              src/wait/queue.rs:378] self.state.compare_exchange_weak(state, WAKING, SeqCst, SeqCst) = Err(1)
[PANIC              src/wait/queue.rs:377] state = 1
[PANIC              src/wait/queue.rs:399] WaitQueue::notify_slow(state: 1)
[PANIC    src/util/mutex/loom_impl.rs:15 ] locking thingbuf::wait::queue::List<loom::thread::Thread>...
[PANIC    src/util/mutex/loom_impl.rs:17 ] -> locked thingbuf::wait::queue::List<loom::thread::Thread>!
[PANIC              src/wait/queue.rs:578] List::dequeue(2) -> 0x7f3475752900
[PANIC              src/wait/queue.rs:581] last.state.swap(new_state, Release) = 1
[PANIC              src/wait/queue.rs:419] list.is_empty() = false
[PANIC                    src/wait.rs:66 ] NOTIFYING Thread { id: ThreadId(1), name: None } (from Thread { id: ThreadId(0), name: None })
[PANIC                     src/lib.rs:185] Core::close
[PANIC              src/wait/queue.rs:302] WaitQueue::continue_wait(0x7f3475752900)
[PANIC              src/wait/queue.rs:305] node.state.load(Acquire) = 2
[PANIC           src/mpsc/blocking.rs:2165] core.tx_wait.continue_wait(node, &thread) = Notified
[PANIC                    src/util.rs:57 ] hint::spin_loop() (x1)
[PANIC                    src/util.rs:60 ] thread::yield_now()

[tukan]

It looks like, after 11089327 iterations under loom, the tests added in PR #81 seem to enter into an infinite spin loop.

Full logs from that iteration

I have an idea what's wrong with the test but I have to debug it. Will try to find some time this week to work on it.

[tukan]

I investigated the test and the code in the https://github.com/sgasse/thingbuf_hangup/.

The test fails because of an edge case, and my initial thought is to adjust the test itself.

Here's a breakdown of the issue: We have a buffer with two slots. Initially, we write to slot [0], then to slot [1], and start reading from slot [0]. If we try writing again, slot [0] is unavailable (as it's still being read), so we skip it. Once the reading from slot [0] is completed, the head pointer moves to slot [1]. We attempt to write to slot [1] but can't, since it's not yet read, and the head pointer is already on slot [1]. Consequently, we declare the buffer as full. The problem arises in my test when starting to read from slot [1] – it never gets released, making it perpetually empty for reading and full for writing.

However, I identified a genuine issue within the code at https://github.com/sgasse/thingbuf_hangup/, which I've temporarily fixed on my local setup.

I executed cargo run --bin thingbuf_sendref from the https://github.com/sgasse/thingbuf_hangup repository (using the latest thingbuf), and it revealed an issue in accurately determining when the buffer is full.

I will re-examine my findings and aim to submit a PR later this week.