[PyInstaller] Workers don't shutdown when started from within a script

[hash3liZer]

When you start & stop proxy.py from command line, it works all fine. However, when using embedded proxy.py in your script, it doesn't totally destroy its workers. The workers are still shown on the Task Manager. I've to manually end the processes.

Consider this example:

import ipaddress
import proxy

if __name__ == "__main__":
    proxy.main(
       hostname = ipaddress.IPv4Address("127.0.0.1"),
       port=8080,
       num_workers=1
    )

Now, compile this script:

$ pyinstaller --onefile script.py

Now, when you run the compiled script:

$ .\agent.exe

And then stop the agent, it would stop the main process which started proxy.py but the proxy.py processes still remains there until forcefully stopped from the task manager.

[abhinavsingh]

Do you force kill the agent? If yes, then I can see why worker processes keep running because they are set as daemon i.e. keep running detached from the main process. This was done to enforce proxy is running even if dispatcher thread errors out. But is not what you want, but I think it will depend how are you terminating your agent. Give it a signal 15 instead of kill 9 and it should also terminate the worker processes. Let me know

[hash3liZer]

I am using CTRL+C to exit from the script. The workers should stop after the signal i suppose.

Anyhow, you are right, if i set the processes as daemon=True, the workers do shut down as they are suppose to. What i did was this in file: proxy.core.acceptor.pool.py in function start_workers:

    def start_workers(self) -> None:
        """Start worker processes."""
        for acceptor_id in range(self.flags.num_workers):
            work_queue = multiprocessing.Pipe()
            acceptor = Acceptor(
                idd=acceptor_id,
                work_queue=work_queue[1],
                flags=self.flags,
                work_klass=self.work_klass,
                lock=LOCK,
                event_queue=self.event_queue,
            )
            acceptor.daemon = True                    ## -- ## Added this statement here
            acceptor.start()
            logger.debug(
                'Started acceptor#%d process %d',
                acceptor_id,
                acceptor.pid)
            self.acceptors.append(acceptor)
            self.work_queues.append(work_queue[0])
        logger.info('Started %d workers' % self.flags.num_workers)

It worked fine after that. If you say so, i can open a pull request for this.

[abhinavsingh]

Frankly I will have to get on a Windows (w/ PyInstaller) to see what exactly is happening. But, since passing daemon flags seems to do the trick here, how about we just add a flag to control this aspect. By default this flag can be False. You can override it when bundling within PyInstaller. Wdyt?

[abhinavsingh]

@hash3liZer Can you share your pyinstaller setup? So that I can reproduce and try the same locally. I gave it a try and it runs into a multiprocessing import error loop when started. Thank you!!!

[abhinavsingh]

@hash3liZer Hey, wanted to post an update about this issue. #908 now registers signal handler and proxy instance will shutdown whenever the parent process detaches. This may or may not work reliably on Windows, but I haven't tried it on Windows yet.