Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

100% CPU usage #11

Closed
mirko314 opened this issue Oct 26, 2015 · 18 comments
Closed

100% CPU usage #11

mirko314 opened this issue Oct 26, 2015 · 18 comments

Comments

@mirko314
Copy link

Hello,
I am getting 100% CPU usage when I start the script. It Starts aber the Board was detected and does not end, even though the script is more or less idling and waiting for scratch to start.
Using it on Arch Linux, but should not make a huge difference!
Greetings,
Mirko

@MrYsLab
Copy link
Owner

MrYsLab commented Oct 26, 2015

What version of s2a_fm and pymata are you using? Are you running this on a PC or something like Raspberry Pi. If on a PC, what CPU are you using and how much RAM do you have?

@mirko314
Copy link
Author

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?


Reply to this email directly or view it on GitHub
#11 (comment).

@MrYsLab
Copy link
Owner

MrYsLab commented Oct 27, 2015

Thanks for the info. I will be away from my computer for the next few days. I will investigate and get back to you later this week. Are you running Windows and what version of Python are you using?

On October 27, 2015, at 8:36 AM, Mirko notifications@github.com wrote:

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?


Reply to this email directly or view it on GitHub
#11 (comment);.


Reply to this email directly or view it on GitHub.

@mirko314
Copy link
Author

Alright, I am using it on Linux ( Arch Linux) using it with python 2.7.
I will check it on Windows on the same machine later.

On 27.10.2015 14:41, Alan Yorinks wrote:

Thanks for the info. I will be away from my computer for the next few
days. I will investigate and get back to you later this week. Are you
running Windows and what version of Python are you using?

On October 27, 2015, at 8:36 AM, Mirko notifications@github.com wrote:

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?


Reply to this email directly or view it on GitHub
#11 (comment);.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#11 (comment).

@MrYsLab
Copy link
Owner

MrYsLab commented Oct 27, 2015

No need to check it on Windows since I run linux myself. Windows most likely will not be an improvement.

Is Python 2.7.10 or an earlier version?

On October 27, 2015, at 9:45 AM, Mirko notifications@github.com wrote:

Alright, I am using it on Linux ( Arch Linux) using it with python 2.7.
I will check it on Windows on the same machine later.

On 27.10.2015 14:41, Alan Yorinks wrote:

Thanks for the info. I will be away from my computer for the next few
days. I will investigate and get back to you later this week. Are you
running Windows and what version of Python are you using?

On October 27, 2015, at 8:36 AM, Mirko notifications@github.com wrote:

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?


Reply to this email directly or view it on GitHub
#11 (comment);;.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#11 (comment);.


Reply to this email directly or view it on GitHub.

@mirko314
Copy link
Author

Alright then. Do you have this skyrocketing CPU-Usage?
I am using:
Python 2.7.10 (default, Sep 7 2015, 13:51:49)
[GCC 5.2.0] on linux2

On 27.10.2015 14:49, Alan Yorinks wrote:

No need to check it on Windows since I run linux myself. Windows most
likely will not be an improvement.

Is Python 2.7.10 or an earlier version?

On October 27, 2015, at 9:45 AM, Mirko notifications@github.com wrote:

Alright, I am using it on Linux ( Arch Linux) using it with python 2.7.
I will check it on Windows on the same machine later.

On 27.10.2015 14:41, Alan Yorinks wrote:

Thanks for the info. I will be away from my computer for the next few
days. I will investigate and get back to you later this week. Are you
running Windows and what version of Python are you using?

On October 27, 2015, at 8:36 AM, Mirko notifications@github.com wrote:

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?


Reply to this email directly or view it on GitHub
#11 (comment);;.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#11 (comment);.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#11 (comment).

@MrYsLab
Copy link
Owner

MrYsLab commented Oct 27, 2015

The last I checked it was much less than 100%. I am going to download and install from scratch and will get back to you later in the week.

On October 27, 2015, at 9:52 AM, Mirko notifications@github.com wrote:

Alright then. Do you have this skyrocketing CPU-Usage?
I am using:
Python 2.7.10 (default, Sep 7 2015, 13:51:49)
[GCC 5.2.0] on linux2

On 27.10.2015 14:49, Alan Yorinks wrote:

No need to check it on Windows since I run linux myself. Windows most
likely will not be an improvement.

Is Python 2.7.10 or an earlier version?

On October 27, 2015, at 9:45 AM, Mirko notifications@github.com wrote:

Alright, I am using it on Linux ( Arch Linux) using it with python 2.7.
I will check it on Windows on the same machine later.

On 27.10.2015 14:41, Alan Yorinks wrote:

Thanks for the info. I will be away from my computer for the next few
days. I will investigate and get back to you later this week. Are you
running Windows and what version of Python are you using?

On October 27, 2015, at 8:36 AM, Mirko notifications@github.com wrote:

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?


Reply to this email directly or view it on GitHub
#11 (comment);;;.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#11 (comment);;.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#11 (comment);.


Reply to this email directly or view it on GitHub.

@mirko314
Copy link
Author

Thank you!
BTW, I really like it, we are currently putting together a course for
Highschool with it :)

On 27.10.2015 17:26, Alan Yorinks wrote:

The last I checked it was much less than 100%. I am going to download
and install from scratch and will get back to you later in the week.

On October 27, 2015, at 9:52 AM, Mirko notifications@github.com wrote:

Alright then. Do you have this skyrocketing CPU-Usage?
I am using:
Python 2.7.10 (default, Sep 7 2015, 13:51:49)
[GCC 5.2.0] on linux2

On 27.10.2015 14:49, Alan Yorinks wrote:

No need to check it on Windows since I run linux myself. Windows most
likely will not be an improvement.

Is Python 2.7.10 or an earlier version?

On October 27, 2015, at 9:45 AM, Mirko notifications@github.com wrote:

Alright, I am using it on Linux ( Arch Linux) using it with python 2.7.
I will check it on Windows on the same machine later.

On 27.10.2015 14:41, Alan Yorinks wrote:

Thanks for the info. I will be away from my computer for the next few
days. I will investigate and get back to you later this week. Are you
running Windows and what version of Python are you using?

On October 27, 2015, at 8:36 AM, Mirko notifications@github.com wrote:

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?


Reply to this email directly or view it on GitHub
#11 (comment);;;.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#11 (comment);;.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#11 (comment);.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#11 (comment).

@MrYsLab
Copy link
Owner

MrYsLab commented Oct 27, 2015

I got back earlier than I originally thought I would, so I was able to run the program and get some results. First, let me share what my system is:
Ubuntu 15.10, 64 bit, 8 gb ram. My processor is an i5-2600k.

When I run the system monitor, the average CPU utilization across all cores is reported at between 38-40% (across all cores). If I look at the individual cores, on occasion, I do see a short 100% peak on one of the cores, but this peak appears on random cores while the other 3 are pretty close to idle.

I am able to run other programs while s2a_fm is up and running without issues (the browser, libre office, etc.).

Is the 100% you are seeing on average or peaking on a single core? Also, are you locked out of access to other programs as a result?

What tool are you using to monitor CPU utilization?

BTW, If I run "top", I see a consistent total usage of 145%. If all 4 cores were fully occupied, then I would see 400 %.

@mirko314
Copy link
Author

Well, It is not an overall 100% its just one Core is at 100% even tough
it switches between the cores. I do not get impaired performance Really,
because I still got 7 different Cores. I just hear that my fans getting
loud because of the cpu usage.
I am using the gnome-system-monitor.

On 28.10.2015 00:00, Alan Yorinks wrote:

I got back earlier than I originally thought I would, so I was able to
run the program and get some results. First, let me share what my system is:
Ubuntu 15.10, 64 bit, 8 gb ram. My processor is an i5-2600k.

When I run the system monitor, the average CPU utilization across all
cores is reported at between 38-40% (across all cores). If I look at the
individual cores, on occasion, I do see a short 100% peak on one of the
cores, but this peak appears on random cores while the other 3 are
pretty close to idle.

I am able to run other programs while s2a_fm is up and running without
issues (the browser, libre office, etc.).

Is the 100% you are seeing on average or peaking on a single core? Also,
are you locked out of access to other programs as a result?

What tool are you using to monitor CPU utilization?

BTW, If I run "top", I see a consistent total usage of 145%. If all 4
cores were fully occupied, then I would see 400 %.


Reply to this email directly or view it on GitHub
#11 (comment).

@MrYsLab
Copy link
Owner

MrYsLab commented Oct 28, 2015

Ok, you had me worried. The program is actually running as expected. The reason for the high percentage is that there is a thread running in a tight loop looking for characters to come in from the Arduino (this is in pymata). Because both pymata and s2a_fm are multithreaded, the overall throughput performance (handling data bidirectionally for the Arduino and Scratch program) is pretty good, but CPU utilization is relatively high. Python has a limitation in running threads. The global interpreter lock (GIL) prevents multiple threads running simultaneously, and threads cannot be distributed amongst the cores. Just a fact of life. Dave Beazly does a nice job describing this.

Recently I have used the asyncio library in another project and in the second generation of pymata (it's called pymata-aio). Running similar applications and comparing the 2 libraries, PyMata shows the same performance as described in a previous comment (~35-40%). When I run pymata-io, the average performance drops from 35% to 4% and no core ever runs at 100%. It is not multi-threaded, but uses event driven asyncio, a much more efficient way of handling concurrency.

I am thinking of porting s2a_fm to asyncio (no decision yet). The pymata end is already done, and being used in other Scratch projects rb4s. Stay tuned.

Is it ok for me to close this issue?

@SebCanet
Copy link

Anything to enhance s2a will be welcomed ! :-D
Thanks, regards.

@MrYsLab
Copy link
Owner

MrYsLab commented Oct 28, 2015

@SebCanet Ok, you convinced me. I will take a look at porting it. ;-). I am not sure what is entailed yet, but hopefully it shouldn't be too bad.

@SebCanet
Copy link

Yahou !!!! Thanks a lot !

@MrYsLab
Copy link
Owner

MrYsLab commented Oct 28, 2015

@SebCanet If I document the new program on a github wiki instead of providing a user's manual, would it make things easier for you, since the wiki pages can be translated in the browser?

All of the block translations should remain the same and the current Scratch files should continue to work.

Also, the latest pymata (pymata-aio) auto detects com ports. I will try to have the program auto launch Scratch.

@mirko314
Copy link
Author

Thank you for explaining this behavior! It would be really appriciated if you port it to the async PyMata lib.
BTW: I restructured your s2a_fm.py to enable autmatic startup of scratch(under Linux)
Looking forward for the implementation!
Thank you!
Mirko

@MrYsLab
Copy link
Owner

MrYsLab commented Oct 30, 2015

@mirko314, @SebCanet I just ran a prototype version of s2a_fm using Python asyncio (I am calling the new version s2aio). You can see the results here. The good news is that you can continue to develop Scratch programs using the current s2a_fm and these programs should be able to be used without modification when s2aio becomes available (at least that is the goal). Thanks Mirko for bringing this issue up.

@SebCanet
Copy link

Thansk for everything !!! That' exactly in this way of mind that I created my "s2a control panel", modular way.
And that's why they choose "our" version of software (whereas mBlock, or Scratchbot, or any other) to create documentation for teacher sold here : http://www.xynops.com/scratch-arduino.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants