Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logrotate and S3 upload tweaks #881

Merged
merged 5 commits into from Feb 9, 2016
Merged

Logrotate and S3 upload tweaks #881

merged 5 commits into from Feb 9, 2016

Conversation

@tpetr
Copy link
Member

tpetr commented Feb 5, 2016

  • support file extensions for extra logrotated files
  • support uploading files to a separate S3 bucket
@tpetr tpetr added the hs_staging label Feb 5, 2016
@tpetr tpetr changed the title WIP: Logrotate and S3 upload tweaks Logrotate and S3 upload tweaks Feb 5, 2016
private final Optional<String> extension;
private final Optional<String> dateformat;

public static SingularityExecutorLogrotateAdditionalFile fromString(String value) {

This comment has been minimized.

Copy link
@ssalinas

ssalinas Feb 5, 2016

Member

Need an @JsonCreator for this one?

This comment has been minimized.

Copy link
@tpetr

tpetr Feb 5, 2016

Author Member

Dang -- I forgot the annotation but Jackson is magically making it work (based on the method name & signature i assume). I'll add the annotation.

@tpetr tpetr added this to the 0.4.10 milestone Feb 5, 2016

int index = 1;
for (SingularityExecutorS3UploaderAdditionalFile additionalFile : configuration.getS3UploaderAdditionalFiles()) {
result = result && writeS3MetadataFile(additionalFile.getS3UploaderFilenameHint().or(String.format("extra%d", index)), logrotateDirectory, String.format("%s*.gz*", additionalFile.getFilename()), additionalFile.getS3UploaderBucket(), additionalFile.getS3UploaderKeyPattern(), finished);

This comment has been minimized.

Copy link
@ssalinas

ssalinas Feb 5, 2016

Member

I think this is forcing that all files to upload are gzipped, do we want to enforce that if it's possible to use this an a folder for uploading arbitrary files?

This comment has been minimized.

Copy link
@tpetr

tpetr Feb 5, 2016

Author Member

Yep (and that's how it was operating before this PR: https://github.com/HubSpot/Singularity/pull/881/files#diff-e8314245f66e3ac2fc51f17320d6f1f6L197). Arbitrary file upload via a "drop" folder will come in a separate PR, if we attempt it

tpetr added a commit that referenced this pull request Feb 9, 2016
Logrotate and S3 upload tweaks
@tpetr tpetr merged commit a598d5f into master Feb 9, 2016
1 of 2 checks passed
1 of 2 checks passed
continuous-integration/travis-ci/push The Travis CI build failed
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@ssalinas ssalinas deleted the logrotate-tweaks branch Apr 5, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.