Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
S3 Search Improvements #1391
Opening this early for any feedback.
Currently, all data about which files to upload to s3, and any possible bucket or key pattern overrides is stored in the executor configuration. This unfortunately means that SingularityService has no idea if it needs to search a separate bucket/pattern to find logs. It can only search what it knows about from it's defaults and the request group overrides.
Notable changes so far:
Note - SingularityService needs to be deployed before the executor for this particular change
need @Consumes on POST endpoints more s3 pagination tweaks more attempts at better pagination need >= here fix maxPerPage more robust search options add missing file fix for findbugs continuation token format needs group too use isTruncated for ending re-request to respect page size revert re-request for page size missing tokens gives false positive end of content typo