-
Notifications
You must be signed in to change notification settings - Fork 95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
High CPU load #248
Comments
Can you see whether it's the If so, you could start by tailing the Supervisor logs for them, they are located in |
This is the output of ps aux|grep 'python'
So both backend and logger seem to be high on load. No errors in std-err logs. After a telegram comes it says "Sleeping for 1 second(s)". What is happening in the meantime? |
That's weird. And sleeping should just suspend Python or something. Can you also see the exact load (or CPU usage) of the processes with It's formatted this way:
|
I did some research, and it appears that MySQL is causing the high load. To be more precise, it's causing the iowait to be high. Not sure why this is happening, as this is not happening all the time. Furthermore I did some logging on MySQL slow queries:
Also table I can't imagine the used SD card has anything to do with it, as it is a class 10 SanDisk Ultra Micro SDHC. Maybe it is wise to switch to PostgreSQL? Can this be done easily? |
The I'm not sure whether switching to PostgreSQL will reduce the load. I think you could try playing around with MySQL settings, by having it write more or less frequent to the SD card. Or try some of these suggestions (except for the tempfs). Please be careful and make sure you create a fresh database backup first. |
I agree updating the dsmr_datalogger_meterstatistics table is not the problem, but is makes debugging slow queries in general a bit easier. To migrate to PostreSQL is worth a try, I read that you want stop support MySQL? |
The support will be dropped somewhere in the future, I will only deprecate it the next release. Since a month or two the project is being used a lot and PostgreSQL seems to have a lot less (small) issues. For now the support for MySQL continues and will so for a few releases, until I've also created a script for migrating data easily. And that script isn't finished yet. |
When I run iotop -oPa it shows that the MySQL daemon generates 4 MB of writes every minute. Can you please check what's the number on your Pi? With that amount of writes I can imagine that my Pi slows down, combined with the journaling filesystem. When I stop the supervisor all writes are gone. |
I think that there might be something with your MySQL installation. The application shouldm't generate that much data. Here it's idle most of the time. When it's writing it's only using 25 K/s. Does the Status page of the application indicate any lagging behind regarding data processing? |
I'm using the default MySQL configuration, did not touch the config file. Status page is ok, no lagging behind. Maybe some other users with MySQL server can report their I/O rates? |
I'm using the InnoDB engine now. As this is my default engine, I'm not sure if you explicitly want to use this engine. Is it safe to switch to MyISAM? |
I've checked another MySQL installation for you running this project, and it only consumes slightly more than PostgreSQL. Definitely not the 4 MB/s you're having. It's also on the default InnoDB engine. MyISAM is faster, but only because it lack any data integrity builtin. It's also no longer the default engine for newer versions of MySQL for that reason. Did the performance degrade slowly or was it already like this, since you've installed this project? |
I guess that the performance was bad from the beginning. Can you please share your MySQL configuration? |
What would you like to know about the configuration? It's running the default one that comes with installing mysql-server. No changes. |
Tried on another Pi, with another (brand of) SD-card. Same result. Are you running MySQL or MariaDB? |
That's still weird, and It's MariaDB that I installed there. |
@tpjanssen what revision of the Pi are you using? I asume the 2 or 3? The biggest change between the last two major releases of this project (v1.4 / v1.5) is the upgrade of the framework I'm using (Django), but it should not relate to any performance issues related to the If you have a spare Pi, you could try running a second clean installation of this project, with PostgreSQL (to determine whether it's related to the database backend) simultaneously. It's actually the same setup I'm using here, to have a spare semi-production enviroment to test before releasing stuff. |
Hi Dennis, Sorry for the late response. Been quite busy lately. Before trying your suggestion, I would like to update first. Not sure, but with installing the update, is data migrated to PostgreSQL automatically? |
You should upgrade when possible, but the data will not be migrated automatically to PostgreSQL. I do however have a script to copy the data between two databases. But it is not quite ready yet as it slows down the longer the script runs. I'll have to improve it a bit to make it run smooth. |
Can you please send me the script so I can give it a try. |
Whether you succeed or not, you may switch back to the regular branch by running |
Beste Dennis,
Lange tijd niet meer mee bezig geweest, maar ik tracht nu toch eens te updaten... Hopelijk kan en wil je me helpen.
Ik gebruik nu 1.5.5. Als ik het goed begrijp kan ik enkel het migrate script draaien vanaf 1.6.0 ? Hoe kan ik nu mijn bestaande installatie updaten naar 1.6.0, de data migreren, en daarna updaten naar de latest and the greatest? Heb PostgreSQL en alles al opgezet, alleen loop ik nu vast omdat 1.5.5 het migrate script nog niet kent.
Mvg,
Thijs
…________________________________
From: Dennis Siemensma <notifications@github.com>
Sent: Saturday, April 8, 2017 6:10 PM
To: dennissiemensma/dsmr-reader
Cc: tpjanssen; Mention
Subject: Re: [dennissiemensma/dsmr-reader] High CPU load (#248)
* Make sure you have updated to the latest version (v1.6).
* Then switch to the branch containing the script:
sudo us - dsmr
git checkout -b 223-mysql-to-postgresql-migration origin/223-mysql-to-postgresql-migration
* Follow the docs here<http://dsmr-reader.readthedocs.io/en/223-mysql-to-postgresql-migration/faq.html#how-can-i-migrate-from-mysql-to-postgresql>.
* Please note that the script currently will slow down the longer it runs. So you'll have to abort and restart it some times. It will resume where it left.
* Also note that you should make sure you have a database backup before you do anything at all, limiting any risks.
________________________________
Whether you succeed or not, you may switch back to the regular branch by running git checkout master again, after you're done.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#248 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AYAL5gn9nU3Z6SEkZwQtDyX02z7CNzoTks5rt7FagaJpZM4Lx1KR>.
|
Ik denk dat ik daarvoor eerst het migratiescript weer moet bijwerken. Sowieso zit het migratiescript nog steeds in een (oude) branch. Heb je nog steeds issues met MySQL-performance? Je zou ook nog kunnen kijken naar eventuele optimalisatie: https://www.percona.com/blog/2013/09/20/innodb-performance-optimization-basics-updated/ Bijvoorbeeld |
Volgens mij heb ik dit al geprobeerd, maakte geen verschil. Ik wil gewoon naar de laatste versie, en hopelijk is de hoge load dan ook verdwenen. Hoe kan ik dit het beste aanpakken?
|
Ik denk dat je dan gewoon moet upgraden en kijken of het iets verandert. Volgens mij is dit afdoende:
Als het goed is kom je daarmee op |
Thanks. De reden dat ik de upgrade niet gedaan heb was omdat ik in de veronderstelling was dat MySQL niet meer werkte. Zojuist de upgrade gedraaid, en het draait weer.
Betekent dit dat je nog met het upgrade script bezig bent?
|
Jazeker, maar het heeft geen hoge priotiteit t.o.v. van andere tickets. Mijn plan was om eerst MySQL uit te faseren in de docs, zodat er geen nieuwe installaties meer gedaan worden met MySQL. Vervolgens op een later punt zou ik MySQL daadwerkelijk uitfaseren. |
Duidelijk verhaal. Dit issue kun je wat mij betreft weer sluiten. Ik wacht af of mijn problemen nog terug komen. Als dit zo is, dan lost een update naar PostgreSQL hopelijk alles op.
Bedankt in ieder geval.
…________________________________
From: Dennis Siemensma <notifications@github.com>
Sent: Tuesday, November 21, 2017 8:30 PM
To: dennissiemensma/dsmr-reader
Cc: tpjanssen; Mention
Subject: Re: [dennissiemensma/dsmr-reader] High CPU load (#248)
Jazeker, maar het heeft geen hoge priotiteit t.o.v. van andere tickets. Mijn plan was om eerst MySQL uit te faseren in de docs, zodat er geen nieuwe installaties meer gedaan worden met MySQL. Vervolgens op een later punt zou ik MySQL daadwerkelijk uitfaseren.
Echter, op dat punt ben ik nog lang niet en tot die tijd blijft MySQL werken (alle tests draaien zowel onder MySQL als PostgreSQL). Helaas helpt dit jou niet direct, wanneer mocht blijken dat ook de laatste release niet lekker performt in jouw situatie.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#248 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AYAL5uj0Hx2CaC6fpUjfmAS5sNO0Ddnyks5s4yS6gaJpZM4Lx1KR>.
|
Hi Dennis,
Are there any changes in the last update that can cause high CPU load? My CPU load average is typically around 1.2 . I have 4.2 meter, read out every 5 s. The reader has stored around 135000 telegrams.
Any ideas?
Regards.
The text was updated successfully, but these errors were encountered: