Execute s3a on Mac computer

[HackEduca]

Hi Alan, I just tried to support a couple of teachers in how to install s3-extend in a Mac computer.
Firstly it was my first contact (remote) in a Mac so far and they are not familiar with this kind of thing.

They just share their screen with me and I guide them (as they don´t speak english).
So I followed all the steps but in the end the s3a did not work.
As I realized that Mac has 2 Python version I tried also Python3 s3a and once again did not work.
Maybe is I am not considering something, so I would like and hint from your side.

Thanks!

I tried some command - maybe it helpfull to identify the issue.

$ s3a
-bash: s3a: command not found

$ python3 s3a
/usr/local/bin/python3: can't open file 's3a': [Errno 2] No such file or directory

$ python3 --version
Python 3.8.3

Thanks!

Edson

[MrYsLab]

Hi Edson,
I hope you and the family are well. I do not own a Mac so I can only ask some questions that may help you out. I know s3-extend is being used on Macs, so it should work for those users as well.

Let me make some assumptions, and please verify with the Mac users that these are correct:

1. The users installed Python3 using the Mac instructions and files on python.org.
2. They have pip3 installed.
3. It is the latest version of pip3 - that is 20.1.1
4. When they installed s3-extend, they used the following command:

sudo pip3 install s3-extend

If any of this is incorrect let me know.

Now to a little background on how python manages the installation of packages. Pip uses the "instructions" contained in the setup.py file that is part of the package. The s3-extend package takes advantage of a feature called "entry_points". An entry_point allows a python script to be executed as an executable (a .exe in Windows terms). It places the .exe or executable file on the user's standard executable path. So clearly, at a minimum, the s3a executable was not placed in the executable path.

Now on to some possible diagnostics.

1. Have a Mac user execute the following command:

pip3 show s3-extend

On linux, which should be similar to macOS, this returns:

Name: s3-extend
Version: 1.12
Summary: A Non-Blocking Event Driven Applications Framework
Home-page: https://github.com/MrYsLab/s3-extend
Author: Alan Yorinks
Author-email: MisterYsLab@gmail.com
License: GNU Affero General Public License v3 or later (AGPLv3+)
Location: /usr/local/lib/python3.8/dist-packages
Requires: pymata-cpx, pymata-express, python-banyan
Required-by:

This will prove that the package was actually installed properly and we can move on to the next step.
To see what path the s3-extend executables were installed on, we need to an "uninstall" of the s3-extend package and then observe the output.

To do this, a user would enter:

sudo pip3 uninstall s3-extend

Here is the output on Linux, and should be similar to the Mac:

Found existing installation: s3-extend 1.12
Uninstalling s3-extend-1.12:
  Would remove:
    /usr/local/bin/ardgw
    /usr/local/bin/cpxgw
    /usr/local/bin/espgw
    /usr/local/bin/pbgw
    /usr/local/bin/rpigw
    /usr/local/bin/s3a
    /usr/local/bin/s3c
    /usr/local/bin/s3e
    /usr/local/bin/s3p
    /usr/local/bin/s3r
    /usr/local/bin/wsgw
    /usr/local/lib/python3.8/dist-packages/s3_extend-1.12.dist-info/*
    /usr/local/lib/python3.8/dist-packages/s3_extend/*
Proceed (y/n)? 

The user can type "n" to abort the uninstall. If you look at the output, you will see the path of the executables, and in the case of s3a for the Linux installation, the full path is "/usr/local/bin/s3a".

So if execute the following command in the terminal:

echo $PATH

I should see /usr/local/bin appear within the path and indeed it does:

/home/afy/gems/bin:/home/afy/.cargo/bin:/home/afy/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/go/bin

Most likely the path that s3-extend is using is not part of the user's path and the path environment variable will need to have the path amended to include the path s3-extend is using. Normally this is done by modifying the path variable in the user's .bashrc file. I think this true in macOS also, but I am not sure. If you need some help with this, let me know.
After modifying the path environment variable, they should exit the terminal window, and open a new one, or just reboot the machine, and check the path again to make sure that includes the one we need, and hopefully, s3a will now execute.

Please let me know if any of this helps, or if you need further explanation.

Stay safe and stay well.
Alan

[MrYsLab]

BTW, the user's guide describes the installation procedure. Hopefully, using Google translate, non-English speakers can get a meaningful translation. The documentation talks about Python 3.7 but the latest version of Python 3 should be used. There is no need to go back to Python 3.7.

[HackEduca]

Hi Alan, hope you and your family are healthy and safe as well.
Things are getting worse in Brazil...

I am going to contact them. Actually, they attended a virtual workshop (for Scratchmonth) when my wife and I presented S3onegpio extension.

Thank you very much for these hints.
We are sharing screen during all the steps and they did exactly as described maybe is something related to the path and I check it.

As soon as I have an answer I get back to you.

Thanks and keep safe.

[MrYsLab]

I have been reading about the situation in Brazil over the past several weeks. I wish I had words of wisdom but don't. Just stay safe and well.

I found an article on updating the path for macOS. It explains how to modify the path for a single user or for all users.

[HackEduca]

Hi Alan, hope everything is doing well.
Thanks for all your support so far.

I had a remote check in their computer and I found the following:
In the first day showed up a message "command not found", but probably this was related to be usage of same Terminal window - without close and reopen after installation.

Today after some days of 'rebooting' :o) came up with a different message and by this I can assume that the Path is not the current issue - Is it make sense?

$ s3a
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/bin/s3a", line 10, in
sys.exit(s3ax())
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/s3_extend/s3a.py", line 242, in s3ax
S3A(com_port=com_port, arduino_instance_id=args.arduino_instance_id)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/s3_extend/s3a.py", line 57, in init
self.proc_bp = self.start_backplane()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/s3_extend/s3a.py", line 163, in start_backplane
if 'backplane' in proc.info['name']:
TypeError: argument of type 'NoneType' is not iterable

In the end I just follow your steps and here is the output:

$ pip3 show s3-extend

Name: s3-extend
Version: 1.12
Summary: A Non-Blocking Event Driven Applications Framework
Home-page: https://github.com/MrYsLab/s3-extend
Author: Alan Yorinks
Author-email: MisterYsLab@gmail.com
License: GNU Affero General Public License v3 or later (AGPLv3+)
Location: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages
Requires: pymata-cpx, python-banyan, pymata-express
Required-by:

$ sudo pip3 uninstall s3-extend

Password:
WARNING: The directory '/Users/xxxxxxxxx/Library/Caches/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Found existing installation: s3-extend 1.12
Uninstalling s3-extend-1.12:
Would remove:
/Library/Frameworks/Python.framework/Versions/3.8/bin/ardgw
/Library/Frameworks/Python.framework/Versions/3.8/bin/cpxgw
/Library/Frameworks/Python.framework/Versions/3.8/bin/espgw
/Library/Frameworks/Python.framework/Versions/3.8/bin/pbgw
/Library/Frameworks/Python.framework/Versions/3.8/bin/rpigw
/Library/Frameworks/Python.framework/Versions/3.8/bin/s3a
/Library/Frameworks/Python.framework/Versions/3.8/bin/s3c
/Library/Frameworks/Python.framework/Versions/3.8/bin/s3e
/Library/Frameworks/Python.framework/Versions/3.8/bin/s3p
/Library/Frameworks/Python.framework/Versions/3.8/bin/s3r
/Library/Frameworks/Python.framework/Versions/3.8/bin/wsgw
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/s3_extend-1.12.dist-info/*
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/s3_extend/*
Proceed (y/n)? n

$ echo $PATH

/Library/Frameworks/Python.framework/Versions/3.8/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin

I also asked to run the following command:
$ s3a -c /dev/cu.usbmodemFT121 and the result was the same

Arduino IDE is also OK to upload express.

Thanks!

[MrYsLab]

Thanks. As you mentioned, the path is not the issue. Looking at the error, it appears that the mac is having problems dealing with a library that I use in s3a that allows me to see what processes are currently running, however, that might not be the actual cause of the error.

If you could scroll down to the troubleshooting section of this page of the user's guide and follow the instructions to launch the individual components, that would tell me if the issue is in the s3a script or in one of the components it launches. The purpose of the s3a script is to make starting the application easier for the user. It does all the steps contained in the troubleshooting section. Please let me know if any of the steps failed and if there was an exception stack trace in the console. Please provide the exception trace.

Also, did the users type:

sudo pip3 install s3-extend

or

pip3 install s3-extend

when they installed the package?

Thanks,
Alan

[HackEduca]

Hi Alan,
For installation they used:

sudo pip3 install s3-extend

About the individual steps, described on troubleshooting section, I did it in the first day. This time I forgot.

Everything was installed on May, 26th.
I was in your webpage giving them all the instructions, since the Python installation for Mac.

I will check it and back to you as soon as I have the answer.

Thanks,

Edson

[HackEduca]

Hi Alan,
following what you have requested:

$ backplane

---

Backplane IP address: 192.168.0.10
Subscriber Port = 43125
Publisher Port = 43124
Loop Time = 0.001 seconds

---

$ ardgw

Pymata Express Version 1.15
Copyright (c) 2018-2020 Alan Yorinks All rights reserved.

Opening all potential serial ports...
/dev/cu.usbmodemFD121

Waiting 4 seconds(arduino_wait) for Arduino devices to reset...

Searching for an Arduino configured with an arduino_instance = 1

Arduino found and connected to /dev/cu.usbmodemFD121

Arduino Firmware ID: 2.5 FirmataExpress.ino
Auto-discovery complete. Found 20 Digital Pins and 6 Analog Pins

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/bin/ardgw", line 10, in
sys.exit(arduino_gateway())
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/s3_extend/gateways/arduino_gateway.py", line 511, in arduino_gateway
app = ArduinoGateway(subscriber_list, **kw_options, event_loop=loop)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/s3_extend/gateways/arduino_gateway.py", line 99, in init
super(ArduinoGateway, self).init(subscriber_list=subscriber_list,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/python_banyan/gateway_base_aio/gateway_base_aio.py", line 113, in init
super(GatewayBaseAIO, self).init(back_plane_ip_address=back_plane_ip_address,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/python_banyan/banyan_base_aio/banyan_base_aio.py", line 131, in init
raise RuntimeError('Backplane is not running - please start it.')
RuntimeError: Backplane is not running - please start it.

$ wsgw

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/psutil/_common.py", line 449, in wrapper
ret = self._cache[fun]
AttributeError: _cache

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/psutil/_psosx.py", line 334, in wrapper
return fun(self, *args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/psutil/_common.py", line 452, in wrapper
return fun(self)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/psutil/_psosx.py", line 384, in _get_kinfo_proc
ret = cext.proc_kinfo_oneshot(self.pid)
ProcessLookupError: [Errno 3] No such process (originated from sysctl (len == 0))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/psutil/init.py", line 373, in _init
self.create_time()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/psutil/init.py", line 723, in create_time
self._create_time = self._proc.create_time()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/psutil/_psosx.py", line 334, in wrapper
return fun(self, *args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/psutil/_psosx.py", line 487, in create_time
return self._get_kinfo_proc()[kinfo_proc_map['ctime']]
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/psutil/_psosx.py", line 336, in wrapper
raise NoSuchProcess(self.pid, self._name)
psutil.NoSuchProcess: psutil.NoSuchProcess process no longer exists (pid=28641)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/bin/wsgw", line 10, in
sys.exit(ws_gateway())
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/s3_extend/gateways/ws_gateway.py", line 283, in ws_gateway
WsGateway(subscription_list, **kw_options, event_loop=loop)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/s3_extend/gateways/ws_gateway.py", line 79, in init
super(WsGateway, self).init(subscriber_list=subscription_list,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/python_banyan/banyan_base_aio/banyan_base_aio.py", line 116, in init
p = psutil.Process(pid)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/psutil/init.py", line 346, in init
self._init(pid)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/psutil/init.py", line 386, in _init
raise NoSuchProcess(pid, None, msg)
psutil.NoSuchProcess: psutil.NoSuchProcess no process found with pid 28641

---

Let me know if you need more data.
Thanks!

Edson

[MrYsLab]

Hi Edson,
I am assuming that once you started the backplane, you left it up and running and tried to start ardgw and wsgw in new and separate terminal windows. It that is not the case, please try things in separate windows.
If you did use separate windows, then something is very strange. I don't know if the problem might be a network issue or not. Could you please open a terminal window on the computer and type:

ifconfig

or you might need:

sudo ifconfig

You should the IP addresses assigned to the computer in use. Please check to see that it is the same IP address is the same one being reported when the backplane starts.

Also, the code requires access to 2 IP ports 43124 and 43125. Is it possible that the router they ar using has those ports blocked?

Thanks.

[HackEduca]

Hi Alan,
due to the weekend it was not easy to contact the user about their problem.

I send them a message to created 3 different Terminal, however they did not pay attention at this point and the previous test was made in closing the windows - so need to be discarded.

They send out a new test considering let the previous window opened and we got the following results.

Apparently worked - I have to contact them to see what they are doing as typing s3a is not working.
I keep you posted as soon as I have the answer.

Last test was the ifconfig - this looks greek to me ;o)
I have sent you and email as I don't know I have to hide some data in order to protect the user - Let me know about it if you know - like ip # and so one....

$ backplane

---

Backplane IP address: 192.168.0.10
Subscriber Port = 43125
Publisher Port = 43124
Loop Time = 0.001 seconds

---

$ wsgw

---

WebSocket Gateway using Back Plane IP address: 192.168.0.10
Subscriber Port = 43125
Publisher Port = 43124

---

WebSocket using: 192.168.0.10:9000

$ ardgw
Pymata Express Version 1.15

Copyright (c) 2018-2020 Alan Yorinks All rights reserved.
Opening all potential serial ports...
/dev/cu.usbmodemFD121
Waiting 4 seconds(arduino_wait) for Arduino devices to reset...

Searching for an Arduino configured with an arduino_instance = 1
Arduino found and connected to /dev/cu.usbmodemFD121
Arduino Firmware ID: 2.5 FirmataExpress.ino
Auto-discovery complete. Found 20 Digital Pins and 6 Analog Pins

---

$ sudo ifconfig

send by mail...

Thanks!

Edson

[MrYsLab]

The ifconfig command is similar to the ipconfig command on windows. It shows IP addresses (and other things) that the computer is using for networking.

So if bringing up the 3 windows manually worked, and s3a failed, based on the exception reported, it appears that the Mac is having problems with a package called psutil that is used by s3a.

Looking at the documentation for this package they may need to install a Mac development package called Xcode. Once that is installed, they can then type:

sudo pip3 install psutil --upgrade

Hopefully, that will fix the problem.

[HackEduca]

Hi Alan, unfortunatelly did not work.
We installed the Xcode and also tried to update psutil which is already in the most updated version.

But they are happy so far with the option to individually open 3 terminal window and execute.
I tried to create a shell file but I did not succeed on my task.

---

Curiously this message is the same I have reported in my oher issue when I use the Python 32bit version on 64bit system.
But I realized that there is no 32bit version for Mac.

$ s3a
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/bin/s3a", line 10, in
sys.exit(s3ax())
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/s3_extend/s3a.py", line 242, in s3ax
S3A(com_port=com_port, arduino_instance_id=args.arduino_instance_id)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/s3_extend/s3a.py", line 57, in init
self.proc_bp = self.start_backplane()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/s3_extend/s3a.py", line 163, in start_backplane
if 'backplane' in proc.info['name']:
TypeError: argument of type 'NoneType' is not iterable

Thanks!

Edson

[MrYsLab]

Here is a shell script that might work. It works on Linux, so perhaps will work on the Mac as well. Hitting Control-C should shut everything down.

(trap 'kill 0' SIGINT; backplane & ardgw & wsgw  )

I am not trying to create more work for you, so please ignore the following if you like. The following psutil Python script, may help to understand the problem that macOS is having with psutil. It appears from reading things on the internet that macOS has shut down some user level permissions
in the past, and perhaps that might explain why things are failing for the Mac with s3a. This program should print all running process names to the console. Here is the python program:

import psutil
import sys

# iterate through all the processes
for proc in psutil.process_iter(attrs=['pid', 'name']):
    # check if backplane is running
    if 'backplane' in proc.info['name']:
        print('Backplane is running')
        sys.exit(0)
    else:
        # print out process name other than backplane
        # for all processes found
        print(proc.info['name'])

print('Backplane is not running')

[HackEduca]

Thanks again for the shell script and also for the psutil program.
I will check it with them and let you know.

Thanks!
Edson

[MrYsLab]

Hi Edson. I hope you and the family are well. I am going to close this issue. If we need to reopen it, just place a comment here and we can.

[HackEduca]

Everything is fine with my family, hope same with yours. Sorry for that. I forgot to be back to you, please to close it. Thanks as usual Edson Em dom., 5 de jul. de 2020 às 11:20, Alan Yorinks <notifications@github.com> escreveu:

…

Hi Edson. I hope you and the family are well. I am going to close this issue. If we need to reopen it, just place a comment here and we can. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#20 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKUSNXKDE2UWS2WHVIXBEYDR2CD2XANCNFSM4NLRQ63A> .