Can't get launchctl working

[sddgit]

I hope it's ok to ask here for help. I can run the script very successfully from MacOS Terminal, using

python3 /users/serveruser/Envoy_scripts/envoy_mqtt_script.py

but I'm having a lot of trouble getting it to run via launchctl. My plist is:

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Disabled</key>
	<false/>
	<key>EnvironmentVariables</key>
	<dict>
		<key>PATH</key>
		<string>/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/usr/local/sbin</string>
	</dict>
	<key>KeepAlive</key>
	<true/>
	<key>Label</key>
	<string>envoy</string>
	<key>ProgramArguments</key>
	<array>
		<string>/usr/bin/python3</string>
		<string>/users/serveruser/Envoy_scripts/envoy_mqtt_script.py</string>
	</array>
	<key>RunAtLoad</key>
	<true/>
	<key>StandardErrorPath</key>
	<string>/Users/serveruser/Library/logs/Envoy mqtt script/stderr.log</string>
	<key>StandardOutPath</key>
	<string>/Users/serveruser/Library/logs/Envoy mqtt script/stdout.log</string>
</dict>
</plist>

When I load it, the stderr.log shows

  File "/users/serveruser/Envoy_scripts/envoy_mqtt_script.py", line 12, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

Now when I first started playing with this, the "requests" module was missing on my system, and I installed it and all was well via Terminal (or Python3 Launcher). But the problem is back when I use launchctl. Any ideas?

[vk2him]

It sounds like you've installed the requests module as a different user that the lauchctl user is using and it can't find it.

Can you confirm when you installed the requests module that you were logged in as /users/serveruser ?

Which user are you logged in as when you can successfully run the script from terminal - is it serveruser?

What does "pip3 show requests" display when you are able to run it from terminal - this will show where requests has been installed.

Hopefully this might give you some clues. Cheers

[sddgit]

Thank you for trying to help.

Everything I've done has been under serveruser. I first of all installed Python3, then pip via curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py, then python3 get-pip.py (got those instructions from a web site somewhere!). Then I followed your instructions pip install paho-mqtt, and running the script gave me the error about missing the "requests" module. So I installed that using pip install requests, and the script ran fine in Terminal, but doesn't using launchctl.

pip3 show requests (or pip show requests) gives:

Version: 2.26.0
Summary: Python HTTP for Humans.
Home-page: https://requests.readthedocs.io
Author: Kenneth Reitz
Author-email: me@kennethreitz.org
License: Apache 2.0
Location: /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages
Requires: urllib3, charset-normalizer, certifi, idna
Required-by: 

Thanks again.

[sddgit]

Fixed it. Instead of loading Python3 from /usr/bin, I now load it from /usr/local/bin, and everything's fine. I have absolutely no idea why it exists in both places!

[vk2him]

Great to see that you fixed it. It would be interesting to see what’s displayed when you type “which Python3” from terminal. Macs really are a royal pain in the rear lol, I’ve had so many issues over the years cause it’s not a real Unix system and Apple do things differently . I’m seriously considering dumping mine and going for a Linux system. I might enable bootcamp on the Mac and see if I can just run Mint or another similar OS.

[sddgit]

That gets /Library/Frameworks/Python.framework/Versions/3.9/bin/python3. Completely different again, but kind of matches where the “requests” module is installed, as above. I got the usr/local/bin/python3 by starting the Python Launcher that was installed when I installed Python3, and that defaulted to that path. I did end up installing Xcode command line tools at some stage too to fix something, but can’t for the life of me remember why! No idea what’s going on…

If you live in Unix/Linux, i can understand why you might want a native install. I’m sure someone’s done that using Boot Camp.

EDIT: Ah, everything Python-related in /usr/local/bin is an alias to the real thing in the Frameworks path, so at least that makes sense. No idea what the executable is in the /usr/bin path is.

[vk2him]

Interesting - which python3 reports /usr/bin/python3 for me - which python reports /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
I hate MACOS lol

[sddgit]

But what’s in your /usr/bin/ folder? Is Python3 an alias to Python3 in the Frameworks path or an executable? In my /usr/bin/ Python is an alias to Python (2) in the frameworks path, but Python3 is an executable (but doesn’t work properly). Do you see Python3 (alias) in /usr/local/bin? All very confusing. To be fair, the placements would be down to the installer wouldn’t they? I used 3.9 from https://www.python.org/downloads/. I presume the installer sets up the aliases to point to the latest release installed.

[vk2him]

My Mac is a hot mess - /usr/bin/ has a combination of symlinks and executables - python3 is executable but python is an alias. I've even got multiple versions of pip3. Maybe I should wipe the mac and start from scratch - I've had the MAC for over 15 years and have installed/deleted a lot of applications and projects, it's been replaced a few times in that time but each new one was restored from time-machine. I'm sure it's got a lot of orphaned repos and junk. I have a spare MacMini so I might build that again from scratch.

[sddgit]

Yeh, probably a good time to rebuild after 15 years, painful as that can be sometimes.

However, I don't think your setup is that different from mine. /usr/bin has the python3 executable and an alias for python (pointing to V2.7). pip3 is an executable. How does your /usr/local/bin look? The key to why I had the problem in the first place is my PATH variable. It's:

PATH=/Library/Frameworks/Python.framework/Versions/3.9/bin:/Users/serveruser/.nvm/versions/node/v14.15.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

So the Python and Node installations seem to add the latest version to the PATH, but I originally launched python3 from /usr/bin. Also note that /usr/local/bin is before /usr/bin, so usr/local/bin is the "truth" as far as Python aliases etc are concerned.

[vk2him]

Yes, PATH adds another layer of complexity lol - /usr/local/bin also has a mix of executables and alias - it ain't pretty.

I wasn't able to install paho-mqtt via pip install paho-mqtt, hence why I listed an alternative - no double my issue is related to multiple versions of python/pip/gosh know what else :)

[sddgit]

Well mine’s not that pretty either, and I’ve only just installed Python!

[vk2him]

Well that didn't take long. I now have Home Assistant and this script running on a MacMini that I formatted and is running Linux Mint as the only OS. I had it up and running in a few hours, it was so much easier than fighting MACOS.

[sddgit]

Nice. Horses for courses!

[sddgit]

Just a thought. Do you think it would be possible for this script to run as an add-on or integration in home assistant?