New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated logrotate frequency to allow for an override #1814

Merged
merged 26 commits into from Aug 1, 2018

Conversation

Projects
None yet
4 participants
@kdorosh

kdorosh commented Jun 27, 2018

To use this feature add additional file/file groups (via wildcard) and times to AdditionalLogrotateFile.

Kevin Dorosh
Updated logrotate frequency to allow for an override by listing addit…
…ional file/file groups (via wildcard).

@kdorosh kdorosh changed the base branch from master to hs_staging Jun 27, 2018

@kdorosh kdorosh changed the base branch from hs_staging to master Jun 27, 2018

@kdorosh kdorosh added the hs_staging label Jun 28, 2018

Show outdated Hide outdated ...gularity/executor/config/SingularityExecutorLogrotateAdditionalFile.java Outdated
@@ -36,6 +36,7 @@ notifempty
{{#if extrasFiles}}
{{#each extrasFiles}}{{{filename}}} {
{{#if logrotateFrequencyOverride }}{{{logrotateFrequencyOverride}}}{{/if}}

This comment has been minimized.

@ssalinas

ssalinas Jul 2, 2018

Member

We should double check if this will 'just work'. Not all versions of logrotate support hourly by default (Many are normally run on a daily cron, not hourly). If you look in SingularityExecutorTaskLogManager.writeLogrotateFile you'll notice that we additionally write an hourly cron for the specific logrotate conf file that needs it. We will likely need an additional check in that method to see if an additional file has an hourly rotate and possibly a separate cron file if so (the current cron uses the -f option to support hourly and force a logrotate run for those versions that do not support it)

@ssalinas

ssalinas Jul 2, 2018

Member

We should double check if this will 'just work'. Not all versions of logrotate support hourly by default (Many are normally run on a daily cron, not hourly). If you look in SingularityExecutorTaskLogManager.writeLogrotateFile you'll notice that we additionally write an hourly cron for the specific logrotate conf file that needs it. We will likely need an additional check in that method to see if an additional file has an hourly rotate and possibly a separate cron file if so (the current cron uses the -f option to support hourly and force a logrotate run for those versions that do not support it)

This comment has been minimized.

@ssalinas

ssalinas Jul 2, 2018

Member

Additional note, enough OSs likely support hourly logrotate now that we could do away with this extra check if we verify that it still works as intended

@ssalinas

ssalinas Jul 2, 2018

Member

Additional note, enough OSs likely support hourly logrotate now that we could do away with this extra check if we verify that it still works as intended

This comment has been minimized.

@kdorosh

kdorosh Jul 2, 2018

We have discussed this at length and determined that it's ok to put the logrotate config files at /etc/logrotate.d/hourly. The Mesos slave running our SingularityExecutor needs to support hourly cron jobs at this directory. We have already internally updated our puppet deploy to enforce this config on mesos slaves.

@kdorosh

kdorosh Jul 2, 2018

We have discussed this at length and determined that it's ok to put the logrotate config files at /etc/logrotate.d/hourly. The Mesos slave running our SingularityExecutor needs to support hourly cron jobs at this directory. We have already internally updated our puppet deploy to enforce this config on mesos slaves.

This comment has been minimized.

@kdorosh

kdorosh Jul 3, 2018

We have decided to support older logrotate versions and added code to Singularity to force a rotation hourly using the -f flag and a cron schedule. This implementation writes config files to two directories:

  • /etc/logrotate.d
  • /etc/logrotate.d/hourly
@kdorosh

kdorosh Jul 3, 2018

We have decided to support older logrotate versions and added code to Singularity to force a rotation hourly using the -f flag and a cron schedule. This implementation writes config files to two directories:

  • /etc/logrotate.d
  • /etc/logrotate.d/hourly

@HubSpot HubSpot deleted a comment from kdorosh Jul 2, 2018

Kevin Dorosh added some commits Jul 2, 2018

@ssalinas ssalinas added the hs_qa label Jul 5, 2018

@ssalinas ssalinas added this to the 0.21.0 milestone Jul 9, 2018

@ssalinas

Two last nit-picky things before we get this on hs_stable

@ssalinas

This comment has been minimized.

Show comment
Hide comment
@ssalinas

ssalinas Jul 12, 2018

Member

🚢

Member

ssalinas commented Jul 12, 2018

🚢

1 similar comment
@kdorosh

This comment has been minimized.

Show comment
Hide comment
@kdorosh

kdorosh commented Jul 12, 2018

🚢

@ssalinas ssalinas added the hs_stable label Jul 16, 2018

@pschoenfelder

This comment has been minimized.

Show comment
Hide comment
@pschoenfelder

pschoenfelder Jul 17, 2018

Contributor

🚢

Contributor

pschoenfelder commented Jul 17, 2018

🚢

@baconmania

This comment has been minimized.

Show comment
Hide comment
@baconmania

baconmania Jul 17, 2018

Contributor

🚢

Contributor

baconmania commented Jul 17, 2018

🚢

@ssalinas ssalinas merged commit 9910a1e into master Aug 1, 2018

0 of 2 checks passed

continuous-integration/travis-ci/pr The Travis CI build could not complete due to an error
Details
continuous-integration/travis-ci/push The Travis CI build could not complete due to an error
Details

@ssalinas ssalinas deleted the logrotate_freq_override branch Aug 1, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment