Stop accepting new queries after initiating shutdown

[dominiklohmann]

Before this change, the index continued accepting queries after initiating shutdown, which could lead to an infinite hang because queries were coming in faster than the index was able to process them.

📝 Checklist

• All user-facing changes have changelog entries.
• The changes are reflected on docs.tenzir.com/vast, if necessary.
• The PR description contains instructions for the reviewer, if necessary.

🎯 Review Instructions

Try locally:

T1> vast [--use-legacy-query-scheduler] start
T2> while true; do { sleep 0.5; vast export null & }; done
T3> vast stop

[tobim]

I got a live demo that the fix works.

[dominiklohmann]

@tobim I've re-requested your review after the last two commits.

[tobim]

Before we accept this additional complexity, can we test whether bidirectional linkage between all 3 actors would work? I mean the sink command calls self->link_to(snk) and self->link_to(exporter), and the exporter links itself to the sink? The rest of the lifetime management in the sink command can probably ripped out.

[dominiklohmann]

Before we accept this additional complexity, can we test whether bidirectional linkage between all 3 actors would work?

It does work, but we lose the ability to give a good exit code and error message if there's an unexpected shutdown. So I think listening for down messages is still preferred.

[tobim]

I can see that this resolves a timing issue with early shutdowns or crashes of the sink process. We should evaluate a solution based on the remote spawn functionality in CAF 0.18 once we are able to upgrade.