remove references to hub.cnlabs.io from duffle?

[bacongobbler]

With #420 in place, one must log into https://hub.cnlabs.io before searching/pushing/pulling bundles from "the hub". This makes it a great opportunity to consider whether or not we want to launch duffle/CNAB with a "CNAB Hub" at all.

I'm wondering if we should remove the concept of a "default" repository entirely from duffle. The only real change to duffle would be here, when duffle signs bundles under hub.cnlabs.io (cc @radu-matei).

The main value add here would be Duffle removing its tight dependency on a default registry. This doesn't mean that we need to stop development on https://hub.cnlabs.io, but it opens up the ecosystem to innovate. e.g. Docker Enterprise users may opt for DockerHub, Azure customers would adopt ACR instances, etc.

[radu-matei]

Yes, I completely agree with that we should remove the default reference, as it is also causing a bunch of other small inconveniences when referencing bundles.

[technosophos]

Do we need the concept of a remote URL inside of the local storage? Is this mainly for namespacing?

I'm not 100% clear on what the local store is supposed to provide, but we have to figure out the following:

• If I create foo:1.2.3, is it local/foo:1.2.3 or just foo:1.2.3
• If I then push foo:1.2.3 to my.hub, does my local storage have foo:1.2.3, local/foo:1.2.3, or my.hub/foo:1.2.3 (or is inclusive, here -- it could be more than one)
• If I pull your.hub/foo:1.2.3, which is different than my.hub/foo:1.2.3, does that get pulled into local storage as your.hub/foo:1.2.3 or local/foo:1.2.3?
• If I edit your.hub/foo:1.2.3, does it remain your.hub:1.2.3 or does it become my.hub/foo:1.2.3 (thus destroying my own my.hub/foo:1.2.3), foo:1.2.3, or local/foo:1.2.3? ANd what does this operation destroy?
• If I push your.hub/foo:1.2.3 to my own hub, does it become my.hub/foo:1.2.3? What happens to the thing that used to be my.hub/foo:1.2.3? (Essentially, here we have performed a namespace swap simply in order to change the location of something.)

I think Docker made a mistake by doing the nginx:latest vs foo/nginx:latest vs my.repo.com/foo/nginx:latest naming scheme because it is really confusing. But it is also frankly a pain to conflate "fully qualified name" with "location where this thing lives (or may live) on the internet" -- and this becomes especially true when you have a push/pull model that may "change the name" of a thing merely in virtue of storing it somewhere else. That last part is actually only becomes problematic when we have something like local storage, where we may just end up in a giant aliasing mess.

[bacongobbler]

Hey @technosophos! I'll try and answer your questions as best as I can.

Do we need the concept of a remote URL inside of the local storage?

This (and the follow-up discussion/arguments you point out) are good design questions. I hadn't considered the issue where if we drop the concept of a "default registry", how do we resolve foo:1.2.3 to a remote endpoint. I'll start by breaking down the user stories and see where we get.

* If I create `foo:1.2.3`, is it `local/foo:1.2.3` or just `foo:1.2.3`

If you create foo:1.2.3, the name will be foo:1.2.3, same as how docker build -t would work.

* If I then push `foo:1.2.3` to `my.hub`, does my local storage have `foo:1.2.3`, `local/foo:1.2.3`, or `my.hub/foo:1.2.3` (or is inclusive, here -- it could be more than one)

This is the part where it gets confusing. With this proposal, foo:1.2.3 does not reference a remote repository in the name. Therefore duffle push foo:1.2.3 would fail with an error. A user would have to re-build (or rewrite) their bundle using the name my.hub/foo:1.2.3 for it to be pushed.

A potential option is to allow for a "default registry" for duffle, where if a user sets their global duffle configuration to set their default registry to foo.example.com, then duffle push foo:1.2.3 could mean "push me to foo.example.com/foo:1.2.3". Any thoughts/opinions on that approach?

To answer the second question, if you build a bundle and call it foo:1.2.3, your local storage would have foo:1.2.3. If you are pushing to my.hub/foo:1.2.3, your local storage would have my.hub/foo:1.2.3.

* If I pull `your.hub/foo:1.2.3`, which is different than `my.hub/foo:1.2.3`, does that get pulled into local storage as `your.hub/foo:1.2.3` or `local/foo:1.2.3`?

your.hub/foo:1.2.3. If the content is identical, then your.hub/foo:1.2.3 and my.hub/foo:1.2.3 would reference the same bundle in $DUFFLE_HOME/repositories.json.

* If I edit `your.hub/foo:1.2.3`, does it remain `your.hub:1.2.3` or does it become `my.hub/foo:1.2.3` (thus destroying my own `my.hub/foo:1.2.3`), `foo:1.2.3`, or `local/foo:1.2.3`? And what does this operation destroy?

Ideally if you change just the destination repository (the name), only the name in repositories.json would change, but the content would stay the same. The operation would not change the underlying bundle's content; only the reference in $DUFFLE_HOME/repositories.json.

Basically, repositories.json would change from

{
    "your.hub/foo": {
        "1.2.3": "ab154c3d8a2f95025f7866e17cbc918e79964731"
    }
}

to

{
    "my.hub/foo": {
        "1.2.3": "ab154c3d8a2f95025f7866e17cbc918e79964731"
    }
}

This may require a change in bundle.cnab: we might need to remove the name field from bundle.cnab, or else a rewrite would require a re-sign of the bundle... Unless we're okay with the fact that moving a bundle from one repository to the next requires a re-sign, in which case then the name/registry should stay as part of the bundle and requires a re-sign on any name change. That being said, removing the name field from a bundle.cnab may affect design assumptions made for duffle import/export... Still pondering on the design.

* If I push `your.hub/foo:1.2.3` to my own hub, does it become `my.hub/foo:1.2.3`? What happens to the thing that used to be `my.hub/foo:1.2.3`? (Essentially, here we have performed a namespace swap simply in order to change the location of something.)

I think I answered that in the last question but please let me know if I missed something.

But it is also frankly a pain to conflate "fully qualified name" with "location where this thing lives (or may live) on the internet" -- and this becomes especially true when you have a push/pull model that may "change the name" of a thing merely in virtue of storing it somewhere else. That last part is actually only becomes problematic when we have something like local storage, where we may just end up in a giant aliasing mess.

I'd love to hear more on this! Would you be free some time to pair on this topic? I'd love to hear your thoughts/opinions on the docker image name conflation problem and possibly whiteboard this out.