st.experimental_rerun() raises exception if run in thread

[blipk]

Checklist

• I have searched the existing issues for similar issues.
• I added a very descriptive title to this issue.
• I have provided sufficient information below to help reproduce this issue.

Summary

Calling st.experimental_rerun() in a thread raises an exception and doesnt rerun streamlit.

Looking at the source for experimental_rerun() it seems it raises the exception in any case, and this is the method by which it signals the rerun, being caught by the main thread.

I'm not sure which part of streamlit handles this, but I am passing the correct context manager to the thread so I figured there should be a way to also catch the exception from a thread.

This is the error that is raised is provided below.

Reproducible Code Example

import time
import threading
import streamlit as st
from streamlit.runtime.scriptrunner import add_script_run_ctx, get_script_run_ctx, RerunException

def main():
    thread = threading.Thread(target=thread_func, args=(5,))
    add_script_run_ctx(thread, get_script_run_ctx())
    thread.start()
    st.write("Hi")

def thread_func(s):
    time.sleep(s)
    st.experimental_rerun()

if __name__ == "__main__":
    main()

Steps To Reproduce

streamlit run main.py on the provided example.

Expected Behavior

The context manager is aware of the (intended?) exception in the thread and Streamlit is rerun at the end of the thread.

Current Behavior

Exception in thread Thread-6 (thread_func):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/user/dev/project/test.py", line 13, in thread_func
    st.experimental_rerun()
  File "/home/user/.local/lib/python3.10/site-packages/streamlit/__init__.py", line 542, in experimental_rerun
    raise _RerunException(
streamlit.runtime.scriptrunner.script_runner.RerunException: RerunData(query_string='', widget_states=None, page_script_hash='b284a28710cce90d9d9be3a7f4cabc8e', page_name='')

Is this a regression?

• Yes, this used to work in a previous version.

Debug info

• Streamlit version: 1.13.0
• Python version: 3.10
• Operating System: Arch64
• Browser: Chrome/Firefox
• Virtual environment: None

Additional Information

No response

Are you willing to submit a PR?

• Yes, I am willing to submit a PR!

[kajarenc]

Hey, @blipk, thank you for opening this issue. I can confirm that RerunException has been raised, but the script doesn't rerun.

CC: @tconkling

[blipk]

I did some research and I'm not really sure it's possible to raise the RerunException from the thread to the main thread.

The best I could find was to do something like below, but that still requires polling the PropagatingThread.exc property or catching the exception when polling with a join() on the thread - but polling would block my main streamlit page thread, so really something like this needs to be built into streamlit if more support for threads is needed.

class PropagatingThread(threading.Thread):
    def run(self):
        self.exc = None
        try:
            self.ret = self._target(*self._args, **self._kwargs)
        except RerunException as e:
            self.exc = e
        except BaseException as e:
            self.exc = e

    def join(self, timeout=None):
        super(PropagatingThread, self).join(timeout)
        if self.exc:
            raise self.exc
        return self.ret

Adding the context does give me correct access to st.session_state etc though, so maybe I could directly send the exception to the main script runner via some other way? Some developer insight would be appreciated.

Right now as an alternative, I am writing to a token .py file at the end of the thread, and import that file in my main page and have runOnSave = true in my config, this achieves the same result as calling st.experimental_rerun() but feels hackish, and has a bug where button callbacks are fired randomly after the rerun (which also happens when rebooting a server on streamlit share)

[tconkling]

Hey @blipk - yeah, as you've noted, this won't work when called from another thread. None of Streamlit's APIs are safe to call from any thread other than your script's main thread; code that does so has ventured into the realm of undefined behavior, and we strongly recommend against it :)

If you need to run a separate thread that invokes Streamlit APIs, you'll need to do something like pushing requests to a shared queue that's processed by your script thread.

(This isn't something that's particularly easy to do with Streamlit, unfortunately. But we do have some - still vague - plans to make more complex use cases, like this one, less tricky!)

I'm closing this issue because Streamlit is working as expected, but feel free to re-open if I've misunderstood!