[Bug] Unsafe threads

[hopeyen]

Problem statement

We ran into situations in which the radio gets stuck or unable to respond to query requests. For instance,

• content topic update (described in Main event silent failure #115)
• comparison result GraphQL query
• logging of summary

We might somehow introduced data races (this should be nearly impossible in Rust, very curious to find out the exact cause). To start with, our current practice of using global variables wrapped in OnceCell and Arc with std::sync::mutex or tokio::sync::mutex might not be effectively preventing deadlock possibilities.

Expectation proposal

In theory, using a Mutex to access a global variable before passing it into async functions can be a valid approach to ensure safe concurrent access.

• Review the approach to access global variables.
• Redesign the concurrent aspects; Potentially create a global struct to contain all the variables
• Make sure that async tasks acquire and release locks asynchronously and do not access global var directly.
• Add timeouts to async functions to avoid threads sitting around forever

Alternative Considerations

• Use RwLock: for global variable that is read-heavy and has infrequent writes, as RwLock multiple readers or a single writer.
• Atomic Types: for simple primitive or atomic type.
• Message Passing: to communicate and share data between async tasks.(tokio::sync::mpsc) to send data between tasks, as MPSC ensures that only one task has ownership of the data at a time and preventing concurrent access issues.
• Setup tokio_console