How to deal with thread spawned outside of madsim simulation

[bsbds]

Hi, thanks for maintaining this fantastic project! I have a question regarding the scenario of invoking external C libraries that may create system threads.

For instance, I've been using rocksdb on madsim runtime, the main thread of rocksdb operates inside the madsim context while the threads it spawned do not, thereby leading to an inconsistency.

Any suggestions on this? Currently I have to allow spawning a system thread and call rocksdb functions inside it.

[wangrunji0408]

Hi! Glad to see madsim is being used in Xline.

It is intended to forbid creating system thread in the simulation in order to prevent non-determinism.

We have encountered a similar issue, where a third-party library, moka, created a system thread in the background for cache invalidation. Our solution was to replace it by a BTreeMap in the simulation.

Generally, I would suggest replace third-party libraries by mocks in the simulation, so that we could avoid any potential uncertainty. However, if this library is critical to functionality and you believe the system thread it creates won't break the determinism, I can introduce a new API to temporarily disable this check.

What do you think?

[bsbds]

Thanks for the suggestions! That makes sense. I'll try replacing the rocksdb with a mock one in the simulation.