Charm storage metadata #77

Merged
merged 6 commits into from Jan 9, 2015

Conversation

Projects
None yet
3 participants
Member

axw commented Nov 17, 2014

We introduce a new "storage" attribute to charm metadata, which describes required and optional storage for a charm. Each may be raw block devices, or file systems. The charm declares the name, type and some restrictions on the storage, and the administrator specifies how to fulfil that storage when deploying the charm.

The metadata format is based on the storage phase 1 scope document:
https://docs.google.com/a/canonical.com/document/d/1-9ZPfdgpkj2R9mBG_tlSclGGyK3tRpMf2L4C37mzYD8/edit#bookmark=id.6abw2ee0ebgm

which is mostly the same as described in the spec: https://docs.google.com/a/canonical.com/document/d/1OhaLiHMoGNFEmDJTiNGMluIlkFtzcYjezC8Yq4nAX3Q/edit

Note that the spec was never signed off, and the scope document is yet to be reviewed. This metadata format is subject to change, and should not be documented until it is finalised. In the short term, this unblocks further work on storage in Juju (which should be uncontroversial).

There are also two new hook kinds defined for notifying units of storage attachment and detachment.

meta.go
+ // Shared indicates that the storage is shared between all units of
+ // a service deployed from the charm. It is an error to attempt to
+ // assign non-shareable storage to a "shared" storage requirement.
+ Shared bool
@wallyworld

wallyworld Nov 18, 2014

Owner

I'd personally like to see, for each of these attributes and where applicable, a comment indicating what the default value is if not explicitly specified in the metadata. The defaults are specified in the schema definition elsewhere, but this is where the main doc for the meaning of each attribute is, so someone reading the code may find it useful. YMMV.

@axw

axw Nov 18, 2014

Member

Done.

Owner

wallyworld commented Nov 18, 2014

provisional LGTM pending william's approval

+)
+
+// Storage represents a charm's storage requirement.
+type Storage struct {
@fwereade

fwereade Dec 2, 2014

Contributor

can we be explicit about serialization names throughout please?

@axw

axw Dec 2, 2014

Member

Can do, though I'm reluctant to increase the use of bson tags here. I'll whip up a patch tomorrow to duplicate the Meta structure in juju/state.

@axw

axw Dec 5, 2014

Member

I asked Rog about this, and he said that Meta is being used directly in the charm store code. So I went ahead and added bson tags with names to everything.

@fwereade

fwereade Jan 9, 2015

Contributor

yuck, but, ok :/

meta.go
+ // create, in order of most to least preferred.
+ //
+ // Filesystem has no default, and is option.
+ Filesystem []Filesystem `bson:",omitempty"`
@fwereade

fwereade Dec 2, 2014

Contributor

hmm, not sure we want bson serialization here, do we? Yaml, yes, because that's the native format for charms -- but we don't want our DB "schema" defined outside state, lest we change it accidentally.

@axw

axw Dec 2, 2014

Member

hmm, not sure we want bson serialization here, do we? Yaml, yes, because that's the native format for charms -- but we don't want our DB "schema" defined outside state, lest we change it accidentally.

I took my cue from the existing Meta structure. state/charm.go already directly uses charm.Meta as a document struct field. This can be changed of course.

meta.go
+
+ // MkfsOptions is any options to use when creating the filesystem.
+ // MkfsOptions will be passed directly to "mkfs".
+ MkfsOptions []string `bson:",omitempty"`
@fwereade

fwereade Dec 2, 2014

Contributor

Not quite convinced about putting "mkfs" or "mount" explicitly into the names here -- windows is a thing :-/. Do these definitely have to be in phase 1?

@axw

axw Dec 2, 2014

Member

I know, I struggled with this a bit. Specifying preferred filesystems is part of the spec, and Mark was quite adamant about Juju not being opinionated in this case. We could do without the options in phase 1 I think, but I don't really see a way to avoid them. Do you have some ideas?

@axw

axw Dec 5, 2014

Member

I've taken them out. This was something suggested by dpb in Mark's spec, but there was never any formal agreement. I think it's a good idea to have it, but we can do it later on.

BTW: I'm not really convinced that having names without mkfs/mount in them would be very useful. The values are not abstract, they're directly tied to those commands. Windows charms are unlikely even to specify a file system type, let alone options.

@@ -141,8 +227,8 @@ func parseStringList(list interface{}) []string {
}
slice := list.([]interface{})
result := make([]string, 0, len(slice))
- for _, cat := range slice {
- result = append(result, cat.(string))
+ for _, elem := range slice {
@fwereade

fwereade Dec 2, 2014

Contributor

Thanks. I wonder what "cat" was short for... "category" maybe?

@axw

axw Dec 2, 2014

Member

Yep, parseStringList used to be called parseCategories I think.

meta_test.go
+ "store0": charm.Storage{
+ Name: "store0",
+ Type: charm.StorageBlock,
+ CountMax: 1,
@fwereade

fwereade Dec 2, 2014

Contributor

Shouldn't we have a CountMin of 1 here too?

@axw

axw Dec 2, 2014

Member

CountMin is 0 because the storage is not marked "required". CountMax should be -1 (unbounded); this test is wrong.

meta_test.go
+ "store1": charm.Storage{
+ Name: "store1",
+ Type: charm.StorageFilesystem,
+ CountMax: 1,
@fwereade

fwereade Dec 2, 2014

Contributor

ditto

@axw

axw Dec 2, 2014

Member

ditto

@@ -424,6 +534,138 @@ var ifaceSchema = schema.FieldMap(
},
)
+func parseStorage(stores interface{}) map[string]Storage {
@fwereade

fwereade Dec 2, 2014

Contributor

I'm wondering whether we can usefully pack some of this code into a schema.Checker? Having something that returns a validated Storage would be really nice -- is there some drawback to that approach that I'm not thinking of?

@axw

axw Dec 3, 2014

Member

I think the main reason for meta.Check() is that we want to check metadata de/serialised to bson in state (see state.State.Charm).

Member

axw commented Dec 5, 2014

@fwereade PTAL.

axw added some commits Oct 29, 2014

Add "storage" to charm metadata
Charms will be able to specify what
their storage requirements/wants are.
Address review comments
- consistent bson tags on Meta, Relation, Storage
- fix CountMax default value, and test
Update storage metadata to match spec
- No more filesystem
- No more persistent
- Change count to multiple.range
- Add description field

I have not yet added minimum size or
properties: these can be added later.
Member

axw commented Jan 6, 2015

PTAL. I've updated again to match recent changes to the spec.

  • No more filesystem (i.e. filesystem type preference)
  • No more persistent (this is configured by the user, on the storage pool)
  • Change count to multiple.range
  • Add description field

I have not yet added minimum size or properties: these can be added later.

Contributor

fwereade commented Jan 9, 2015

LGTM

axw added a commit that referenced this pull request Jan 9, 2015

@axw axw merged commit c4e615f into juju:v4 Jan 9, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment