Option to save log files with a click on the button in the admin page #584

joyrider3774 · 2016-10-25T14:54:17Z

As per request of @glynhudson i added a quick option to download the log file. I intentionally did not put text as content type so you always get a download box (depending on the browser) to choose where to save the file and don't view the file inside the browser.

I tested it on my windows and it seems to work fine for me should be tested on linux

add header("Pragma: no-cache"); header("Expires: 0"); remove odbstart & clean

joyrider3774 · 2016-10-25T17:02:37Z

I added 2 extra options to download the emonpi update log and backup log for the emonpi update log i added a button but for backuplog i did not (but did add the route) as i could not find where it is being used.

Please test this on an actual emonpi as i don't have an emonpi i blindly programmed the options dealing with emonpi (the normal log download i did test) !

…st. If not add the file downloaded will contain the message that it did not exist (to prevent blank pages and always downloading a file)

joyrider3774 · 2016-10-25T17:50:16Z

for the emonpi related logfiles i added $session['admin'] verifications as i saw the normal getlog routes did this as well. I also added a fileexists check when going the download route and when the file does not exist it still downloads a file with a message in it saying the logfile did not exist. This is to prevent the browser from navigating away when no file would be downloaded or displaying blank pages. (It makes sure a file will always be downloaded even if the log does not exist) The normal logfile does not have this problem as a file is always created if we have write access and if we don't have write access no button is being shown but the emonpi buttons should always be shown as you trigger the update itselve without reloading the page (ajax call)

glynhudson · 2016-10-25T17:50:40Z

Yah! The logs download great 👍

However, when I open the emonPi log files there seems to be some invalid characters at the start

I don't see these characters when I view the log directly on the Pi:

No issue with emoncms.log, I can view that perfectly in gedit.

JonMurphy · 2016-10-25T17:53:39Z

glyn - those have always been there in the emonpiupdate.log file

glynhudson · 2016-10-25T17:58:17Z

Ah, interesting. So it's no fault of the download.

I wonder where those characters are coming from? I can't see them in the emoncms logview or when I view the log directly

head ~/data/emonpiupdate.log 
Starting emonPi Update >
EUID: 1000

Mon 24 Oct 14:55:32 UTC 2016

git pull /home/pi/emonpi
Updating 10e2b76..c3bee09
Fast-forward
 .gitignore                                      |  27 +++++
 .travis.yml                                     |  42 ++++++++

Mmmm..I'm not sure where to look. Could it be the log file format?

joyrider3774 · 2016-10-25T18:01:32Z

i could try adding ob_clean back in it should clear the buffer before downloading the file... but that would only work if something already had written data to the buffer somehow (like extra whitespaces after ?> etc) it even seems they are 0 characters so not sure if they would come from extra whitespaces in the php files. I'll try the ob_clean thing. I'll change it immedialty and do the pr this way you can test if that's the problem

joyrider3774 · 2016-10-25T18:10:38Z

the zero's might also come from something that "service-runner-update.sh" does but i don't have that file and can't find in the repositories as all the update function does is run that script and pipe the output to a file

…ning of the files (in the downloaded file)

joyrider3774 · 2016-10-25T18:25:47Z

the reason the log on the admin page does not have those zero characters was because trim got called on the result. I'm explicitly doing the same thing now. But i don't know what kind of side effect this might have if you got a logfile of say 500 mb or 1 gigabyte etc as your passing that amount of data to the trim function then.

Calling trim should fix the zero characters when downloading the file as well though

… file and might give problems passing the value to trim.

joyrider3774 · 2016-10-25T18:37:41Z

don't merge / try yet i made a mistake with the trim statements

…the emonpi update & backup load. Emoncms.log doesn't need this.

joyrider3774 · 2016-10-25T18:58:07Z

it should work now. I only trim emonpi update & backup log but not emoncms log. The emonpi logs probably stay very small while the emoncms log can be become large and did not have the problem

joyrider3774 · 2016-10-25T21:35:19Z

i tested with a raspberry pi and emonSD-03May16 image and as far as i can tell it seems to work. Perhaps someone else can do a final test before actually merging

glynhudson

working great for me now, no invalid characters. Just made a few minor comments on path variable referencing. Anyone else able to test, @JonMurphy ?

glynhudson · 2016-10-25T21:59:04Z

Modules/admin/admin_controller.php

+                    header("Pragma: no-cache"); 
+                    header("Expires: 0");
+                    flush();
+                    if (file_exists("/home/pi/data/emonpiupdate.log"))


the path for emonpiupdate.log is defined on line 133

https://github.com/joyrider3774/emoncms/blob/ebe7dc607da65ba2fcb5233bc12d8877f2176656/Modules/admin/admin_controller.php#L133

It would be best to reference this variable $update_logfile here rather than hardcode the abs path in another location

that would not work as far as i can tell the variables are only known on that route (in that if statement). I can move them outside the if statement and reference adapt my code as well as the other code that was there before to use the variable. The only not so clean down side is that the variables would be set each time you use the emonpi route action while perhaps never having need for them although i don't really consider that a bad thing as it would not have an impact and then you can reference the variables

the code actually used harcoded absolute paths before i made my changes ..

glynhudson · 2016-10-25T21:59:20Z

Modules/admin/admin_controller.php

+                    if (file_exists("/home/pi/data/emonpiupdate.log"))
+                    {
+                      ob_start();
+                      readfile("/home/pi/data/emonpiupdate.log");


glynhudson · 2016-10-25T21:59:28Z

Modules/admin/admin_controller.php

+                    }
+                    else
+                    {
+                      echo("/home/pi/data/emonpiupdate.log does not exist!");


glynhudson · 2016-10-25T22:02:28Z

emoncms.log can only be a max of 1Mb (on an emonPi) due to an aggressive log rotate since we are keeping the log on a tmpfs ramdisk. I don't think the log will be too large for trim to handle. The emonpi update log also won't be very large at all few Kb max probably.

https://github.com/openenergymonitor/emonpi/blob/master/logrotate.conf#L12

Working great, just made a couple of minor comments with regard to variable referancing.

…e in other subroutes and grouped all file related references for the emonpi route together

glynhudson · 2016-10-26T09:27:56Z

Perfect, thanks. Merged 👍

joyrider3774 added 2 commits October 25, 2016 16:51

Option to save log files with a click on the button

1901b4e

add download emonpi update log & backuplog subroute

11a1248

add header("Pragma: no-cache"); header("Expires: 0"); remove odbstart & clean

add "&& $session['admin']" and verify if emonpi related log files exi…

323c8ba

…st. If not add the file downloaded will contain the message that it did not exist (to prevent blank pages and always downloading a file)

call trim on the log files to get rid of zero characters on the begin…

fddbbb8

…ning of the files (in the downloaded file)

don't trim emoncms.log file as this could potentially be a really big…

a0f9dbc

… file and might give problems passing the value to trim.

use ob_start and trim(ob_get_clean) to correctly trim the buffer for …

2adc5a5

…the emonpi update & backup load. Emoncms.log doesn't need this.

fix the buttons not appearing near each other (line breaks <BR>)

ebe7dc6

glynhudson requested changes Oct 25, 2016

View reviewed changes

Use variabele references outside update subroute so they are accesibl…

df9e0b0

…e in other subroutes and grouped all file related references for the emonpi route together

glynhudson merged commit df9e0b0 into emoncms:master Oct 26, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Option to save log files with a click on the button in the admin page #584

Option to save log files with a click on the button in the admin page #584

joyrider3774 commented Oct 25, 2016 •

edited

Loading

joyrider3774 commented Oct 25, 2016 •

edited

Loading

joyrider3774 commented Oct 25, 2016

glynhudson commented Oct 25, 2016 •

edited

Loading

JonMurphy commented Oct 25, 2016

glynhudson commented Oct 25, 2016

joyrider3774 commented Oct 25, 2016

joyrider3774 commented Oct 25, 2016

joyrider3774 commented Oct 25, 2016

joyrider3774 commented Oct 25, 2016

joyrider3774 commented Oct 25, 2016

joyrider3774 commented Oct 25, 2016

glynhudson left a comment •

edited

Loading

glynhudson Oct 25, 2016

joyrider3774 Oct 25, 2016

glynhudson Oct 25, 2016

glynhudson Oct 25, 2016

glynhudson commented Oct 25, 2016

glynhudson commented Oct 26, 2016

Option to save log files with a click on the button in the admin page #584

Option to save log files with a click on the button in the admin page #584

Conversation

joyrider3774 commented Oct 25, 2016 • edited Loading

joyrider3774 commented Oct 25, 2016 • edited Loading

joyrider3774 commented Oct 25, 2016

glynhudson commented Oct 25, 2016 • edited Loading

JonMurphy commented Oct 25, 2016

glynhudson commented Oct 25, 2016

joyrider3774 commented Oct 25, 2016

joyrider3774 commented Oct 25, 2016

joyrider3774 commented Oct 25, 2016

joyrider3774 commented Oct 25, 2016

joyrider3774 commented Oct 25, 2016

joyrider3774 commented Oct 25, 2016

glynhudson left a comment • edited Loading

Choose a reason for hiding this comment

glynhudson Oct 25, 2016

Choose a reason for hiding this comment

joyrider3774 Oct 25, 2016

Choose a reason for hiding this comment

glynhudson Oct 25, 2016

Choose a reason for hiding this comment

glynhudson Oct 25, 2016

Choose a reason for hiding this comment

glynhudson commented Oct 25, 2016

glynhudson commented Oct 26, 2016

joyrider3774 commented Oct 25, 2016 •

edited

Loading

joyrider3774 commented Oct 25, 2016 •

edited

Loading

glynhudson commented Oct 25, 2016 •

edited

Loading

glynhudson left a comment •

edited

Loading