New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backwards compatible archive processing #485

Merged
merged 3 commits into from Jun 24, 2014

Conversation

Projects
None yet
2 participants
@bcobb
Contributor

bcobb commented Jun 19, 2014

馃嵈

In order to migrate all data from the community site (and, technically, to remain backwards compatible with it), Supermarket needs to understand how to process more than just GZipped tarballs. This PR uses libarchive so that Supermarket can process all existing cookbook archives. It also modifies the process by which a cookbook's metadata is found to support some backwards-compatible quirks, such as cookbooks which have metadata in file named "metadata.json.txt"

Once Supermarket is officially launched, this code can likely be modified to be more restrictive, or just reverted outright. But right now it'd be nice to just get everything migrated over.

The libarchive gem requires the libarchive-dev package to be installed on Ubuntu, so this PR is blocked by an update to the Supermarket cookbook. However, I think this code is ready to be reviewed.

@bcobb bcobb added the Signed CLA label Jun 19, 2014

@bcobb bcobb added Signed CLA and removed Signed CLA labels Jun 23, 2014

@bcobb bcobb referenced this pull request Jun 23, 2014

Merged

Package libarchive-dev #20

@tristanoneil

This comment has been minimized.

Contributor

tristanoneil commented Jun 23, 2014

馃憤 Looks good to me. I'm wondering if we want want to squash a few of these or all but one (with a really detailed commit message) since we intend to revert most of this, might make it easier on us in the future, but I'll leave that up to you.

@bcobb bcobb added Signed CLA and removed Signed CLA labels Jun 23, 2014

@bcobb

This comment has been minimized.

Contributor

bcobb commented Jun 23, 2014

@tristanoneil rearranged code and commits so that the last one (21114b2) ought to be revertible when we no longer need to be backwards-compatible.

@bcobb bcobb added Signed CLA and removed Signed CLA labels Jun 24, 2014

@tristanoneil

This comment has been minimized.

Contributor

tristanoneil commented Jun 24, 2014

@bcobb Nice! :shipit:

bcobb added some commits Jun 23, 2014

Factor out CookbookUpload::Archive
This sets Supermarket up to alter its archive processing logic
independently of its file regex logic, which will make a future
(temporary) change to using libarchive transparent and revertible.
Make metadata.json pattern more precise
Namely: use \A instead of ^, include \Z in the pattern so that matches
are exact, and escape the period preceeding the extension.
Process archives in a backwards-compatible way
For the time being, we'll use libarchive, along with a relaxed and less
efficient metadata seeking process to process cookbook archives. Doing
so allows the data migration to complete. This commit should revert
cleanly once the backwards-compatible period is over.

@bcobb bcobb added Signed CLA and removed Signed CLA labels Jun 24, 2014

bcobb added a commit that referenced this pull request Jun 24, 2014

Merge pull request #485 from opscode/backwards-compatible-archive-pro鈥
鈥essing

Backwards compatible archive processing

@bcobb bcobb merged commit d8b6080 into master Jun 24, 2014

1 check passed

continuous-integration/travis-ci The Travis CI build passed
Details

@bcobb bcobb deleted the backwards-compatible-archive-processing branch Jun 24, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment