Skip to content
This repository has been archived by the owner on Feb 24, 2020. It is now read-only.

Add an ACI Renderer library. #297

Closed
wants to merge 9 commits into from
Closed

Add an ACI Renderer library. #297

wants to merge 9 commits into from

Conversation

sgotti
Copy link
Contributor

@sgotti sgotti commented Dec 19, 2014

Hi,

this big PR tries to add a new ACI Renderer library (as proposed in appc/spec#16).
I tried to split it in multiple self consistent patches as I'm touching various subsystems. Other then adding new tar functions (some already proposed in #276, if its merged I'll rebase this PR) (the first two patches are very verbose as they adds a new godep) the new features are:

  • cas: on aci import, additionally save some ACI infos: the image manifest, its ImageID, if the images is a "latest" image and import time.
  • cas: on aci import, uses a diskv store colled appIndex used as an index to retrieve all the ImageID with a specific App Name.
    For doing this is uses as the key name the hash of the app name concatatenated with the ImageID.
  • cas: A GetACI function that, given an app name and optional labels, returns the imageID of the best matching image (see Clarifications on dependency matching and split between fetching and getting. appc/spec#73 for what I tried to do to find the best matching image)
  • cas: A GetImageManifest function that given an imageID, returns the image manifest of that image.
  • acirenderer: a library that, given an app name and optional labels, or an ImageID, creates the dependencies tree and renders it in the requested directory.

I'm using and testing all these changes with https://github.com/sgotti/acido in the "acirenderer" branch.

Note: As already noted in cas/cas.go by @philips and #71 quite probably appIndex should be stored in a database.

I have some doubt that I'd like to clarify related the separation between discovery/fetching and rendering. I opened appc/spec#73.

Thanks!

@sgotti
Copy link
Contributor Author

sgotti commented Dec 20, 2014

I updated the PR removing the verbose patches (already provided by #276) as restoring files' time isn't strictly needed for the acirenderer.

@sgotti sgotti force-pushed the acirenderer branch 3 times, most recently from a56fd35 to 630176c Compare December 21, 2014 17:37
@sgotti
Copy link
Contributor Author

sgotti commented Dec 21, 2014

I updated the PR with various fixes and made it to work without any change in appc/spec (I'm going to close appc/spec#57 and propose future single PR).

@sgotti
Copy link
Contributor Author

sgotti commented Dec 24, 2014

I don't know if someone started reviewing this, but I updated the PR with various changes and, instead of adding another comment with all the changes, I directly changed the first one.

@vcaputo
Copy link
Contributor

vcaputo commented Jan 9, 2015

@sgotti Thanks! Could you break this up into multiple PRs for the unrelated commits though? this guys is kinda all over the place. /cc @jonboulle

@sgotti sgotti force-pushed the acirenderer branch 3 times, most recently from 2f1ca06 to abd1c6e Compare January 15, 2015 09:51
@sgotti sgotti force-pushed the acirenderer branch 4 times, most recently from f19c3c8 to f6bb983 Compare January 19, 2015 09:42
@sgotti
Copy link
Contributor Author

sgotti commented Jan 19, 2015

I rebased it on top of the new getaci provided in #395.

…eACI

Add a new diskv store to save the ImageManifest of an ACI. As future functions
will need the image manifest this avoids the need (and slowness) of extracting
it from the ACI every time.

As an ACI and its image manifest are tied together, on WriteACI the image
manifest is saved using the same key of the blob store.
GetImageManifest retrieves the ImageManifest with the specified key.
This db handler wraps a database/sql and handles (but it can be changed) a
github.com/cznic/ql db adding external locking functions and various utility
functions.

As multiple rkt processes can concurrently try to access the db there the need
to lock it. This is done locking the dir containing the db file.

To avoid long wait times of concurrent rkt processes trying to access a DB,
only short living transactions should be done.

The DoTx function executes the provided txfuncs inside a unique transaction.
If one of the functions returns an error the whole transaction is rolled back.

The Do functions calls DoTx but before it locks and opens the db, and after it
closes and unlocks the db.

For the reasons explained above the Do function should always be used.
The ACIInfo table save informations on the ACIs. The saved informations are:

* BlobKey is the key in the blob and imageManifest store of the related ACI file.
* AppName is the app name provided by the ACI.
* ImportTime is the time this ACI was imported in the store
* Latest defines if the ACI was imported using the latest pattern (no version
label provided on ACI discovery)
On fetching, if downloaded with the "latest" pattern (no version label
specified), the ACI is saved in the store with the "latest" flag.
GetACI returns the best ACI that matches the required app name and the provided
labels.
If there are multiple matching ACIs choose the latest one (defined as the
last one imported in the store).
If no version label is requested, ACIs marked as latest in the ACIInfo are
preferred.
@sgotti
Copy link
Contributor Author

sgotti commented Feb 4, 2015

I'm going to close this as I split the acirenderer patch in two parts (#464 and #465). The first one adding the real acirenderer that doesn't needs any previous PR and the other that adds dependency resolving logic and needs all the previous PRs.

@sgotti sgotti closed this Feb 4, 2015
@sgotti sgotti deleted the acirenderer branch May 24, 2015 16:30
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants