Error Logs as Mail Attachments

[tanvi09]

Included the functionality of adding " Inline message Errors" in the mail as attachments from the _job files. For fast tracking of errors, this saves time to go and check the logs.

Properties are configurable via the config properties files and are defined as

mail.attach.name=.*job.*.log
mail.inline.error=true

_mail.attach.name_ supports regex like ..log or ..job.* . This would attach files matching this regex as attachments from the job folder.
_mail.inline.error_ would also have the error statements as inline text in the mail body.

Hope this helps!

[erwa]

Hi Tanvi,
Why do you want to attach log files to the emails? The logs are already viewable in the Azkaban web UI. There are also AJAX APIs you can use to fetch job logs: http://azkaban.github.io/azkaban/docs/2.5/#api-fetch-execution-job-logs

[kingster]

Hi Anthony,

We use azkaban as our monitoring and alerting system. In this scenario, log files as attachments, increases our response time to any issues, since the error files are readily available, specially on mobile devices ( though the same can be accessed via web UI as you mentioned but logging in and seeing the response becomes tedious time when there are multiple alerts, and action is to be taken based on the cause which threw the alert ) , and assessing the severity of the issue straight away and appropriate action if any taken immediately.

[erwa]

Hi Kinshuk,

I have concerns about this change. Log files can be very big and trying to attach them may cause the email to exceed the max email size, causing the email send to fail. Thus, I do not think we should merge this change.

[kingster]

Hi Anthony

Thanks for pointing out this case. This skipped my mind, since for most of our use cases the log files are normally small. This change set doesn't by default start by attaching files. This has to be specifically enabled via the config.

We can have another config, which says the max size of the attachment, and if that is crossed, it would skip it from attaching to the email. How do you think about it?

[kaneda]

@kingster wouldn't just the log lines leading up to and include the failure be what you're looking for? @erwa

[suvodeep-pyne]

Cleaning up unattended / old PRs