Event Video(s) aren't getting attached to email.

[pyrodex]

I've noticed in recent updates from Ubuntu's default version to the master branch that anytime an event is triggered and emailed off the video of the event isn't being attached. This worked in 1.29.0 but with 1.29.0-xenial-2016060101 I noticed it and before that as well. Here is my system details:

root@zoneminder:/var/log/zm# uname -a
Linux zoneminder 4.4.0-28-generic #47-Ubuntu SMP Fri Jun 24 10:09:13 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
root@zoneminder:/var/log/zm# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04 LTS"
root@zoneminder:/var/log/zm# dpkg -l | grep zoneminder
ii zoneminder 1.29.0-xenial-2016060101 amd64 video camera security and surveillance solution
ii zoneminder-doc 1.29.0-xenial-2016060101 all ZoneMinder documentation
root@zoneminder:/var/log/zm#

I can see the events are being generated properly since the mp4 is there and the ffmpeg.log is in the event directory. From zmvideo.log:

07/06/2016 10:53:19.109424 zmvideo[16807].INF [Creating video file Event-33-r1-s1.mp4 for event 33] 07/06/2016 10:53:21.089052 zmvideo[16807].INF [Finished Event-33-r1-s1.mp4]

You can see in zmfilter.log the event is mailed and the image is attached, per my email notification setup, but the event isn't.
07/06/2016 10:53:18.726292 zmfilter[12426].INF [Archiving event 33]
07/06/2016 10:53:21.098528 zmfilter[12426].INF [Creating notification email]
07/06/2016 10:53:21.438639 zmfilter[12426].INF [Sending notification email 'ZoneMinder: Alarm - Backdoor-33 (2016-07-06 10:50:35)']
07/06/2016 10:53:21.439869 zmfilter[12426].INF [Attaching '/usr/share/zoneminder/www/events/3/16/07/06/10/50/35/00077-capture.jpg]
07/06/2016 10:53:21.440641 zmfilter[12426].INF [Attaching '/usr/share/zoneminder/www/events/3/16/07/06/10/50/35/]
07/06/2016 10:53:21.500236 zmfilter[12426].INF [Notification email sent]

Here is a successful event before the upgrade for reference from zmfilter.log:

07/06/2016 08:49:23.232178 zmfilter[2843].INF [Archiving event 22]
07/06/2016 08:49:26.948009 zmfilter[2843].INF [Creating notification email]
07/06/2016 08:49:27.112325 zmfilter[2843].INF [Sending notification email 'ZoneMinder: Alarm - Driveway-22 (2016-07-06 08:44:37)']
07/06/2016 08:49:27.113986 zmfilter[2843].INF [Attaching '/usr/share/zoneminder/www/events/1/16/07/06/08/44/37/00207-capture.jpg]
07/06/2016 08:49:27.114838 zmfilter[2843].INF [Attaching '/usr/share/zoneminder/www/events/1/16/07/06/08/44/37/Event-22-r1-s1.mp4]
07/06/2016 08:49:28.996618 zmfilter[2843].INF [Notification email sent]

Here is the directory of the failed events after the upgrade showing the file is there and generated:

root@zoneminder:/var/log/zm# ls -la /usr/share/zoneminder/www/events/3/16/07/06/10/50/35/ | grep -v jpg
total 13240
drwxr-xr-x 2 www-data www-data 12288 Jul 6 10:53 .
drwxr-xr-x 3 www-data www-data 4096 Jul 6 10:50 ..
-rw-r--r-- 1 www-data www-data 0 Jul 6 10:50 .33
-rw-r--r-- 1 www-data www-data 620497 Jul 6 10:53 Event-33-r1-s1.mp4
-rw-r--r-- 1 www-data www-data 325171 Jul 6 10:53 ffmpeg.log
root@zoneminder:/var/log/zm#

If you need anything else let me know and thanks for the investigation.

[knight-of-ni]

Can you show us how you configured your email tab?
I'm particularity interested to know if you have enabled ZM_NEW_MAIL_MODULES, since that feature is the newest change to the part of zmfilter that handles attachments.

[pyrodex]

Sure. All the settings are default out of the box minus the email addresses, EMAIL_BODY, and URL.

[knight-of-ni]

This has been caused by commit b32ef77
The short description to the problem is that zmfilter.pl calls sub generateVideo which in turn calls zmvideo.pl. generateVideo expect zmvideo to return the filename of the video it generated, only it no longer does.

@connortechnology appears to have made extensive changes to zmvideo.pl recently, and consequently the solution is not as simple as uncommenting this line:
https://github.com/ZoneMinder/ZoneMinder/blob/master/scripts/zmvideo.pl.in#L260

In any case, I am out of time for today, and will explore what the solution should be later, unless someone else beats me to it.

[connortechnology]

I don't think the problem is too much harder though. The old standard case hasn't changed much. My efforts were to add a mode in which multiple events would be concatenated into one file, in which case we should print that out.

I'll have a PR in a bit.

[knight-of-ni]

Yup, I just ran out of time last night to make any further progress. Thanks for fixing this.

[knight-of-ni]

@pyrodex I'm not able to test #1557 from my office. If you've got time, see if you can apply that PR's changes to your system and let us know if everything works. Otherwise, I'll test it sometime later when I've got an opportunity.

[pyrodex]

@knnniggett sure, since I use the PPA I've modified the file by hand in the same places and about to test now. Anything else I need to test besides my normal use case with this change?

[pyrodex]

@knnniggett

No go, here is what I see:

07/11/2016 09:26:52.072907 zmfilter[9468].INF [Archiving event 165]
07/11/2016 09:26:52.076766 zmfilter[9468].INF [Creating notification email]
07/11/2016 09:26:52.521986 zmfilter[9468].INF [Sending notification email 'ZoneMinder: Alarm - Driveway-165 (2016-07-11 09:21:29)']
07/11/2016 09:26:52.523468 zmfilter[9468].INF [Attaching '/usr/share/zoneminder/www/events/1/16/07/11/09/21/29/00076-capture.jpg]
07/11/2016 09:26:52.524259 zmfilter[9468].INF [Attaching '/usr/share/zoneminder/www/events/1/16/07/11/09/21/29//usr/share/zoneminder/www/events/1/16/07/11/09/21/29/Event-165-r1-s1.mp4]
07/11/2016 09:26:52.525042 zmfilter[9468].ERR [Can't send email: /usr/share/zoneminder/www/events/1/16/07/11/09/21/29//usr/share/zoneminder/www/events/1/16/07/11/09/21/29/Event-165-r1-s1.mp4: not readable]

I created a forced alarm situation and ran my filter manually to make it go as seen here:

[knight-of-ni]

I know you said the video file was getting created in your initial post, but can you check again the file exists and is not zero bytes?

/usr/share/zoneminder/www/events/1/16/07/11/09/21/29//usr/share/zoneminder/www/events/1/16/07/11/09/21/29/Event-165-r1-s1.mp4

[pyrodex]

The print from zmvideo.pl seems to dump the entire path hence the double directory structure but when I shorten I see the file there.

root@zoneminder:/usr/bin# 
root@zoneminder:/usr/bin# ls -la /usr/share/zoneminder/www/events/1/16/07/11/09/21/29//usr/share/zoneminder/www/events/1/16/07/11/09/21/29/Event-165-r1-s1.mp4
ls: cannot access '/usr/share/zoneminder/www/events/1/16/07/11/09/21/29//usr/share/zoneminder/www/events/1/16/07/11/09/21/29/Event-165-r1-s1.mp4': No such file or directory
root@zoneminder:/usr/bin# ls -la /usr/share/zoneminder/www/events/1/16/07/11/09/21/29/Event-165-r1-s1.mp4
-rw-r--r-- 1 www-data www-data 929815 Jul 11 09:25 /usr/share/zoneminder/www/events/1/16/07/11/09/21/29/Event-165-r1-s1.mp4
root@zoneminder:/usr/bin# 

[knight-of-ni]

Oh yeah, that'll do it. I didn't notice that until you mentioned it.

[knight-of-ni]

@connortechnology Previous behavior of video.pl is to return the filename only, without the path.

The safest change seems to be to change the GenerateVideo sub in Event.pm:
https://github.com/ZoneMinder/ZoneMinder/blob/master/scripts/ZoneMinder/lib/ZoneMinder/Event.pm#L271
https://github.com/ZoneMinder/ZoneMinder/blob/master/scripts/ZoneMinder/lib/ZoneMinder/Event.pm#L274

Alternatively, we could remove modify the generateVideo sub in zmfilter to not sprintf the path:
https://github.com/ZoneMinder/ZoneMinder/blob/master/scripts/zmfilter.pl.in#L417

Unless you know the former will cause a problem with your recent changes, that is the solution I would rather take. We don't always know what third party tools are out there, which expect zmvideo.pl to return the filename only. No sense in causing breakage if we can avoid it.

[connortechnology]

I expect to change code to handle full paths. Just giving a file name is not useful because we have no idea where it is.

[connortechnology]

Yeah I went with your second suggestion. PR updated.

My reasoning is that not giving the full path is really not useful to a human,and fixing any external tools that rely on the behaviour is not difficult, nor is it our problem. Plus I would like to know about these external tools.

[pyrodex]

@connortechnology @knnniggett just made the modifications by hand and it worked! Anything else you want me to test?

07/12/16 14:42:45.564456 zmfilter[27966].INF [Scanning for events using filter '_TempFilter1468348965']
07/12/16 14:42:45.635001 zmfilter[27966].INF [Archiving event 182]
07/12/16 14:42:48.627256 zmfilter[27966].INF [Creating notification email]
07/12/16 14:42:49.002692 zmfilter[27966].INF [Sending notification email 'ZoneMinder: Alarm - Walkway-182 (2016-07-12 14:42:12)']
07/12/16 14:42:49.040433 zmfilter[27966].INF [Attaching '/usr/share/zoneminder/www/events/4/16/07/12/14/42/12/00076-capture.jpg]
07/12/16 14:42:49.042444 zmfilter[27966].INF [Attaching '/usr/share/zoneminder/www/events/4/16/07/12/14/42/12/Event-182-r1-s1.mp4]
07/12/16 14:42:50.284828 zmfilter[27966].INF [Notification email sent]

[knight-of-ni]

I did?

[knight-of-ni]

In any case, glad it worked. If you've got the time and haven't done so already, go through your entire system and see if all the buttons and widgets work. The more testers we have the less likely something will sneak its way into the release.