[ADD] Order groups when deploying to S3 #37
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reason
I wanted to use
s3deploy
to deploy a Single Page Application to S3. On SPAs, the order in which things are deployed is important because you can have inconsistencies. On these casesindex.html
must be the latest file on being uploaded.Imagine an scenario in which you have a CloudFront distribution pointing to an S3 bucket. As SPA,
index.html
must havecache=0
to avoid problems with new versions. Ifindex.html
is not uploaded the latest file, you can have these problematic scenarios:index.html
is the first file on being uploaded and other files (dependencies such as CSS or JS) take a while on being updated (maybe because you are also uploading big files), CloudFront can cache resources not present on a request performed during the deploy.index.html
first and then an error happens (OS error reading a file or network connectivity), all requests will be pointing to this new file which points to resources that may be are not present.What is the goal?
Adding support to order file uploads.
Approach
A task of obtaining the order groups for all local files is executed (see
groupLocalFiles
) before plan process. This method stores on memory the local files (only indispensable information) that must be processed later. Then, each group is processed sequentially, uploading files present on each group in parallel (in the same way than before).Refactoring
In order to make more easy the testing and allowing new local stores on the future (maybe a
tgz
file or similar), a new abstraction of local store has been added to the deploy process.However, as I don't know if this approach fits with the project's needs, I left previous tests intact. Maybe in the future could be changed to adapt this new approach.
Testing
Testing has been added for new processes.