Skip to content

Commit

Permalink
Fix S3 path caching double-prefix
Browse files Browse the repository at this point in the history
Original PR: #621
Fixes: #619

I've added unit-test to Martyn's PR.

Without this fix, if `prefix` is set on S3 publish endpoint,
aptly would incorrectly build path cache and re-upload every object
on publish.
  • Loading branch information
smira committed Aug 16, 2017
1 parent 35e2253 commit 790d858
Show file tree
Hide file tree
Showing 3 changed files with 17 additions and 1 deletion.
1 change: 1 addition & 0 deletions AUTHORS
Expand Up @@ -28,3 +28,4 @@ List of contributors, in chronological order:
* Charles Hsu (https://github.com/charz)
* Clemens Rabe (https://github.com/seeraven)
* TJ Merritt (https://github.com/tjmerritt)
* Matt Martyn (https://github.com/MMartyn)
2 changes: 1 addition & 1 deletion s3/public.go
Expand Up @@ -267,7 +267,7 @@ func (storage *PublishedStorage) LinkFromPool(publishedDirectory, baseName strin
poolPath := filepath.Join(storage.prefix, relPath)

if storage.pathCache == nil {
paths, md5s, err := storage.internalFilelist(storage.prefix, true)
paths, md5s, err := storage.internalFilelist("", true)
if err != nil {
return errors.Wrap(err, "error caching paths under prefix")
}
Expand Down
15 changes: 15 additions & 0 deletions s3/public_test.go
Expand Up @@ -257,4 +257,19 @@ func (s *PublishedStorageSuite) TestLinkFromPool(c *C) {
c.Check(err, IsNil)

c.Check(s.GetFile(c, "pool/main/m/mars-invaders/mars-invaders_1.03.deb"), DeepEquals, []byte("Spam"))

// for prefixed storage:
// first link from pool
err = s.prefixedStorage.LinkFromPool(filepath.Join("", "pool", "main", "m/mars-invaders"), "mars-invaders_1.03.deb", pool, src1, cksum1, false)
c.Check(err, IsNil)

// 2nd link from pool, providing wrong path for source file
//
// this test should check that file already exists in S3 and skip upload (which would fail if not skipped)
s.prefixedStorage.pathCache = nil
err = s.prefixedStorage.LinkFromPool(filepath.Join("", "pool", "main", "m/mars-invaders"), "mars-invaders_1.03.deb", pool, "wrong-looks-like-pathcache-doesnt-work", cksum1, false)
c.Check(err, IsNil)

c.Check(s.GetFile(c, "lala/pool/main/m/mars-invaders/mars-invaders_1.03.deb"), DeepEquals, []byte("Contents"))

}

0 comments on commit 790d858

Please sign in to comment.