Should we be able to specify multiple tags for broker to list bundles from?

[maleck13]

This form is for bug reports and feature requests. Major features will go through a spec process.

Feature (managing multiple services and bundle tags):

Currently the broker can only look at a single tag when searching a registry for bundles. This seems like it could cause some difficulties.
For example is I have two bundles and want to update one of those bundles to a new tag for a bug fix, I would currently have to update both tags to ensure that both bundles continued to show up.
So I would like to propose that we come up with a way to allow the broker to look at multiple different tags to find the various service bundles.
Possibly going one step further and defining a resource that specifies a service the spec of which defines the registry, image and tag to use along with a secret to use for the credentials

A question I have which will inform this discussion, is just how the broker is intended to be used in a cluster.
Do we expect that there will be many automation brokers in a cluster (one per service effectively) or just one per cluster. Are there any limitations in the current design that mean there can be only one per cluster?
I ask this as it would be an option to solve the above problem to deploy two brokers and configure them to manage just one service each.

ping @jmrodri @rthallisey @shawn-hurley

[shawn-hurley]

I think that right now, you could specify a white_list to ignore the broken service and a new registry pointing at a different tag that you could also filter out only the bundles that you wanted.

I see if this is not ideal but I do think that it is currently achievable with the current config options.

A question I have which will inform this discussion, is just how the broker is intended to be used in a cluster.

I think that up until now we had assumed that a single broker per namespace. You could have multiple brokers in a cluster but they would need to be namespaced. A limitation would be you would need multiple service catalog's "ClusterServiceBroker" for each broker.

[maleck13]

@shawn-hurley Right it seems reasonable to me to expect to use a single ASB per cluster and make it configurable enough to be used by many groups to deliver there services.

[mhrivnak]

Could you elaborate on this pattern of using tags, perhaps with a couple of examples?

I think the usage is intended to be with a floating tag. You might version a service bundle on its own using one-off tags for each version. But you'd also expect a floating tag to be used that always references the image that should be available in the service catalog. For example, the version-specific tags below would never change, but the "staging" and "production" tags would be updated to reference the specific images you want to be available in those environments. Then you'd configure the broker's adapter to look for one or the other.

1.2.0
1.2.1
1.3.0-beta1
staging
production

[maleck13]

I believe I understand what you are saying . So in your approach, you would tag and test an APB as 1.2.0 and once happy then re tag as production

In the past when delivering images to a registry we have always had a specific tag for each release.

4.2
4.2.1
4.3
4.4

So my thoughts here are likely coloured by how we have done this in the past. But using a floating tag raises a lot of questions in my mind. Not that I am saying the approach is wrong, it may be that I just need to adjust my thinking 🤔 .

If you have a floating tag it means you could not use IfNotPresent right? You would always need a pull policy of Always. Are there any considerations required if this is the case and is that something that we would want to promote (IE things could change from one provision / bind to the next) without the end consumer realizing.

Specific versions, I believe, would help with traceability: using specific tags would help with supporting and debugging issues with that version of the Bundle. Having a floating tag means you would need some way to track the floating tag back to a specific version I would think?

The cluster owner / customer seems to loose control of choosing when to upgrade if you just use a floating tag (sometimes this could be an advantage but is it something we should prescribe across the board?)

If there is a floating tag and the service the bundle provisions changes a major version how do you communicate that to the end consumer who may have provisioned the same service a week ago and got v 1.0 of that service and now provisions a new service for a new project suddenly gets v 2.0. How would they discover this and update their development SDKs etc appropriately? Perhaps it could be communicated via the service class? Or would you see this be handled by having two different bundles one named: service-bundle-1.0:production and a second one named service-2.0:production which would end up with two different service classes in the catalog. This is different that the usual way of managing image versions.

There may be other questions that are not occurring to me currently.

On the feature
Having thought a little more about the multiple tags I am not sure it is the best approach either as it could be difficult to manage (assuming that many services delivered by potentially many different groups in an organization is a valid use case) as there is no correlation between the tag and the service in the config. I.E If you have a tag 4.2, 5.0, 1.0 you could accidentally pickup another maybe much older bundle also tagged like that unless you manage the whitelist carefully. The same problem would occur with using multiple registry definitions each with a different tag.

If we felt a feature like this was needed, it leads me to think the CRD approach gives the most control and allows for both the floating tag approach and the individual version approach

[maleck13]

Interested in some others view points working on Bundles ping @david-martin @wei-lee

[maleck13]

Any more thoughts on this from the wider group?

[mhrivnak]

If you have a floating tag it means you could not use IfNotPresent right? You would always need a pull policy of Always. Are there any considerations required if this is the case and is that something that we would want to promote (IE things could change from one provision / bind to the next) without the end consumer realizing.

That's correct.

Specific versions, I believe, would help with traceability: using specific tags would help with supporting and debugging issues with that version of the Bundle. Having a floating tag means you would need some way to track the floating tag back to a specific version I would think?

"Content addressibility" for container images was designed (and re-designed...) to help with this. A tag is always a mutable reference, even if your convention is that 4.2.1 will always reference the same image. If you want to know with certainty which image was present somewhere, the hash is the best way to identify it. It would be a good idea for us to make sure that a service bundle's image hash gets logged and is otherwise available for traceability.

The cluster owner / customer seems to loose control of choosing when to upgrade if you just use a floating tag (sometimes this could be an advantage but is it something we should prescribe across the board?)

If there is a floating tag and the service the bundle provisions changes a major version how do you communicate that to the end consumer who may have provisioned the same service a week ago and got v 1.0 of that service and now provisions a new service for a new project suddenly gets v 2.0. How would they discover this and update their development SDKs etc appropriately? Perhaps it could be communicated via the service class? Or would you see this be handled by having two different bundles one named: service-bundle-1.0:production and a second one named service-2.0:production which would end up with two different service classes in the catalog. This is different that the usual way of managing image versions.

Agreed. And this is a point that applies to many other kinds of content. Running the postgresql-apb today vs next week has the same problem as running yum install postgresql today vs. next week. Each might behave differently a week from now, hopefully in compatible ways.

The difference of course is that for third-party RPMs, we have a notion of a Linux distribution that is a curated set of packages at specific versions with compatibility guarantees. We're ok with slight changes because we trust that the third party provider won't break anything within a major release. If you're developing your own RPMs, it's up to you to curate them into repositories that align with whatever compatibility guarantees you need.

For service bundles, we don't quite have a concept like a linux distribution. In theory we could start making releases of the entire set of bundles, and make major vs. minor releases that come with compatibility guarantees. I'm not sure how much actual demand there would be for that though.

As is the case with most content types, the best way to guarantee that you consistently and reliably get the same image is to curate your own repositories. You can use something like Pulp or just run an on-premises registry of your choosing.

Otherwise, the next-best option is to reference images by their hash instead of by tag. In at least one registry adapter, we allow you to enumerate a list of specific images. It would be reasonable for that list to support hashes in addition to tags (maybe it already supports this?), to let you be very specific about which image you want to use.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[jmrodri]

/close

[openshift-ci-robot]

@jmrodri: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.