Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logrotate and S3 upload tweaks #881

Merged
merged 5 commits into from Feb 9, 2016
Merged

Logrotate and S3 upload tweaks #881

merged 5 commits into from Feb 9, 2016

Conversation

tpetr
Copy link
Contributor

@tpetr tpetr commented Feb 5, 2016

  • support file extensions for extra logrotated files
  • support uploading files to a separate S3 bucket

@tpetr tpetr changed the title WIP: Logrotate and S3 upload tweaks Logrotate and S3 upload tweaks Feb 5, 2016
private final Optional<String> extension;
private final Optional<String> dateformat;

public static SingularityExecutorLogrotateAdditionalFile fromString(String value) {
Copy link
Member

@ssalinas ssalinas Feb 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need an @JsonCreator for this one?

Copy link
Contributor Author

@tpetr tpetr Feb 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dang -- I forgot the annotation but Jackson is magically making it work (based on the method name & signature i assume). I'll add the annotation.

@tpetr tpetr added this to the 0.4.10 milestone Feb 5, 2016

int index = 1;
for (SingularityExecutorS3UploaderAdditionalFile additionalFile : configuration.getS3UploaderAdditionalFiles()) {
result = result && writeS3MetadataFile(additionalFile.getS3UploaderFilenameHint().or(String.format("extra%d", index)), logrotateDirectory, String.format("%s*.gz*", additionalFile.getFilename()), additionalFile.getS3UploaderBucket(), additionalFile.getS3UploaderKeyPattern(), finished);
Copy link
Member

@ssalinas ssalinas Feb 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is forcing that all files to upload are gzipped, do we want to enforce that if it's possible to use this an a folder for uploading arbitrary files?

Copy link
Contributor Author

@tpetr tpetr Feb 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep (and that's how it was operating before this PR: https://github.com/HubSpot/Singularity/pull/881/files#diff-e8314245f66e3ac2fc51f17320d6f1f6L197). Arbitrary file upload via a "drop" folder will come in a separate PR, if we attempt it

tpetr pushed a commit that referenced this issue Feb 9, 2016
Logrotate and S3 upload tweaks
@tpetr tpetr merged commit a598d5f into master Feb 9, 2016
1 of 2 checks passed
@tpetr tpetr removed hs_qa labels Feb 9, 2016
@ssalinas ssalinas deleted the logrotate-tweaks branch Apr 5, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants