Logrotate and S3 upload tweaks #881

Merged
merged 5 commits into from Feb 9, 2016

Conversation

Projects
None yet
2 participants
@tpetr
Member

tpetr commented Feb 5, 2016

  • support file extensions for extra logrotated files
  • support uploading files to a separate S3 bucket

@tpetr tpetr added the hs_staging label Feb 5, 2016

@tpetr tpetr changed the title from WIP: Logrotate and S3 upload tweaks to Logrotate and S3 upload tweaks Feb 5, 2016

+ private final Optional<String> extension;
+ private final Optional<String> dateformat;
+
+ public static SingularityExecutorLogrotateAdditionalFile fromString(String value) {

This comment has been minimized.

@ssalinas

ssalinas Feb 5, 2016

Member

Need an @JsonCreator for this one?

@ssalinas

ssalinas Feb 5, 2016

Member

Need an @JsonCreator for this one?

This comment has been minimized.

@tpetr

tpetr Feb 5, 2016

Member

Dang -- I forgot the annotation but Jackson is magically making it work (based on the method name & signature i assume). I'll add the annotation.

@tpetr

tpetr Feb 5, 2016

Member

Dang -- I forgot the annotation but Jackson is magically making it work (based on the method name & signature i assume). I'll add the annotation.

@tpetr tpetr added this to the 0.4.10 milestone Feb 5, 2016

+
+ int index = 1;
+ for (SingularityExecutorS3UploaderAdditionalFile additionalFile : configuration.getS3UploaderAdditionalFiles()) {
+ result = result && writeS3MetadataFile(additionalFile.getS3UploaderFilenameHint().or(String.format("extra%d", index)), logrotateDirectory, String.format("%s*.gz*", additionalFile.getFilename()), additionalFile.getS3UploaderBucket(), additionalFile.getS3UploaderKeyPattern(), finished);

This comment has been minimized.

@ssalinas

ssalinas Feb 5, 2016

Member

I think this is forcing that all files to upload are gzipped, do we want to enforce that if it's possible to use this an a folder for uploading arbitrary files?

@ssalinas

ssalinas Feb 5, 2016

Member

I think this is forcing that all files to upload are gzipped, do we want to enforce that if it's possible to use this an a folder for uploading arbitrary files?

This comment has been minimized.

@tpetr

tpetr Feb 5, 2016

Member

Yep (and that's how it was operating before this PR: https://github.com/HubSpot/Singularity/pull/881/files#diff-e8314245f66e3ac2fc51f17320d6f1f6L197). Arbitrary file upload via a "drop" folder will come in a separate PR, if we attempt it

@tpetr

tpetr Feb 5, 2016

Member

Yep (and that's how it was operating before this PR: https://github.com/HubSpot/Singularity/pull/881/files#diff-e8314245f66e3ac2fc51f17320d6f1f6L197). Arbitrary file upload via a "drop" folder will come in a separate PR, if we attempt it

tpetr added a commit that referenced this pull request Feb 9, 2016

@tpetr tpetr merged commit a598d5f into master Feb 9, 2016

1 of 2 checks passed

continuous-integration/travis-ci/push The Travis CI build failed
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@ssalinas ssalinas deleted the logrotate-tweaks branch Apr 5, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment