Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

direct.stdpy.threading: Fatal Python error upon shutdown since Python 3.8 #758

Closed
kamgha opened this issue Oct 19, 2019 · 4 comments
Closed

direct.stdpy.threading: Fatal Python error upon shutdown since Python 3.8 #758

kamgha opened this issue Oct 19, 2019 · 4 comments
Assignees
Labels
bug
Milestone

Comments

@kamgha
Copy link
Contributor

@kamgha kamgha commented Oct 19, 2019

Since Python 3.8.0, the following sample code will cause a "Fatal Python error" upon shutdown of the application (e. g. by closing the main Panda window):

from math import pi, sin, cos
from direct.showbase.ShowBase import ShowBase
from direct.stdpy import threading
#from direct.stdpy import threading2 as threading
#import threading
from queue import Queue
from direct.showbase.MessengerGlobal import messenger
from direct.task.TaskManagerGlobal import Task, taskMgr
 
 
class MyApp(ShowBase):
    def __init__(self):
        ShowBase.__init__(self)
 
        # Load the environment model.
        self.scene = self.loader.loadModel("models/environment")
        # Reparent the model to render.
        self.scene.reparentTo(self.render)
        # Apply scale and position transforms on the model.
        self.scene.setScale(0.25, 0.25, 0.25)
        self.scene.setPos(-8, 42, 0)
 
        # Add the spinCameraTask procedure to the task manager.
        self.taskMgr.add(self.spinCameraTask, "SpinCameraTask")
 
    # Define a procedure to move the camera.
    def spinCameraTask(self, task):
        angleDegrees = task.time * 6.0
        angleRadians = angleDegrees * (pi / 180.0)
        self.camera.setPos(20 * sin(angleRadians), -20.0 * cos(angleRadians), 3)
        self.camera.setHpr(angleDegrees, 0, 0)
        return Task.cont
 
 
class ConsolePrompt:
 
    def __init__(self):
        self.entered_queue = Queue()
        self.entered_queue_lock = threading.Lock()
 
 
    def console_prompt(self):
        """Opens my poor-mans in-game console."""
 
        taskMgr.add(self.check_for_input, 'ConsolePrompt-checkForInputTask', 100)
 
        thread = ConsolePromptThread(self)
        thread.start()
 
    def check_for_input(self, task=None):
        """A Task calls this every frame while the Console Prompt is active."""
 
        self.entered_queue_lock.acquire()
        while not self.entered_queue.empty():
            entered = self.entered_queue.get()
            self.entered_queue_lock.release()
 
            if entered == '':
                print('Console quit by user.')
                return Task.done
 
            print('something has been entered')
 
            self.entered_queue_lock.acquire()
 
            # re-open command prompt
            thread = ConsolePromptThread(self)
            thread.start()
 
        self.entered_queue_lock.release()
        return Task.cont
 
 
class ConsolePromptThread(threading.Thread):
    """The prompt itself will be run inside a Thread to avoid freezing the main window."""
 
    def __init__(self, cp_object):
        threading.Thread.__init__(self)
        self.console_prompt = cp_object
 
    def run(self):
        entered = input('INPUT HERE> ')
        entered = entered.strip()  # Strip whitespaces
 
        self.console_prompt.entered_queue_lock.acquire()
        self.console_prompt.entered_queue.put(entered)  # Put the entered string into the Queue to be processed
        self.console_prompt.entered_queue_lock.release()  # in the main Thread.
 
 
app = MyApp()
cp = ConsolePrompt()
cp.console_prompt()
app.run()

The main point in this sample is, that an input() is waiting in a separate Thread in the terminal window (e. g. for a simple game console) while the user wants to close the application. The problem can be observed in Windows when the sample code is run from an IDE, in this case PyCharm. It can not be reproduced when running it directly from the command line.

Steps to reproduce:

  1. Run the sample program
  2. Close the main Panda window

Error message when using direct.stdpy.threading:

Fatal Python error: could not acquire lock for <_io.BufferedReader name='<stdin>'> at interpreter shutdown, possibly due to daemon threads
Python runtime state: finalizing (tstate=00000000005843E0)
 
Thread 0x00000880 (most recent call first):
  File "C:/Users/Kamran/PycharmProjects/threadtest/panda.py", line 81 in run
  File "C:\panda3d\direct\stdpy\threading.py", line 108 in call_run
 
Current thread 0x00001758 (most recent call first):
<no Python frame>
 
Process finished with exit code 3

Error message when using direct.stdpy.threading2 as threading:

Fatal Python error: could not acquire lock for <_io.BufferedReader name='<stdin>'> at interpreter shutdown, possibly due to daemon threads
Python runtime state: finalizing (tstate=00000000004143E0)
 
Thread 0x00001bac (most recent call first):
  File "C:/Users/Kamran/PycharmProjects/threadtest/panda.py", line 81 in run
  File "C:\panda3d\direct\stdpy\threading2.py", line 466 in __bootstrap
  File "C:\panda3d\direct\stdpy\thread.py", line 107 in threadFunc
 
Current thread 0x00000e24 (most recent call first):
<no Python frame>
 
Process finished with exit code 3

Note that Python 3.8 introduces threading.excepthook() (and threading.get_native_id()), which may be related.

@rdb

This comment has been minimized.

Copy link
Member

@rdb rdb commented Dec 11, 2019

I'm unable to reproduce this on Linux. Going to assign the windows tag for now.

@rdb rdb added the windows label Dec 11, 2019
@rdb

This comment has been minimized.

Copy link
Member

@rdb rdb commented Dec 28, 2019

You can actually reproduce the same error with the Python built-in threading module by passing in daemon=True to the Thread constructor. So the issue here appears to be a disagreement between Python's and Panda's threading over whether to the thread should be daemonic.

However, Panda does not support the daemon parameter. I will look into this further.

@rdb

This comment has been minimized.

Copy link
Member

@rdb rdb commented Dec 28, 2019

Correction: all Panda-created threads are apparently "daemon threads" because Python explicitly joins any threads that were created with the threading module, but not with Panda's threading module.

We can probably fix this with an atexit handler.

@rdb rdb added this to the 1.10.5 milestone Dec 28, 2019
@rdb rdb self-assigned this Dec 28, 2019
@rdb rdb added the bug label Dec 28, 2019
@rdb

This comment has been minimized.

Copy link
Member

@rdb rdb commented Dec 28, 2019

Since the direct.stdpy.threading2 module promises to have the same semantics as Python's threading module, I've fixed this for that module. I'm going to leave the semantics of the direct.stdpy.threading module intact for 1.10.5 (for fear of causing breakage in applications that rely on the current behaviour), but I'm open to revisiting it for 1.11.0; I think these modules could well do with a revision anyway.

To work around the problem, register an atexit handler or override base.exitfunc to join your threads before interpreter shutdown.

@rdb rdb removed the windows label Dec 28, 2019
@rdb rdb closed this in c0fd600 Dec 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.