Clean Up App Catalogs (Only Giant Swarm and Playground catalogs remaining)

[cokiengchiara]

Outcome of this is just delete incubator and move stuff to right catalog. No quality bars.

User story:

1. As a GS person giving an external demo, I want one catalog I could demo, so that I don’t waste time asking someone every time which of the many catalogs I can demo.
2. As an end user developer, I want to easily see which apps I could actually use for my work.

Todos

• Decision: how many catalogs for clients from Managed Apps https://github.com/giantswarm/giantswarm/issues/7443
• Decision: Definition of Done for apps and catalogs https://github.com/giantswarm/giantswarm/issues/7445
• Review apps to see if they meet the definitions of done
• Move k8s-initiator and fluentd-shipping-logs to playground

Incubator status Mon, Mar 16, '20

• kubernetes-elastic-stack-elastic-logging -- to be deleted in favor of efk-stack
• redis-app — decided to delete
• kong-app — @yasn77 this cycle, to GS catalog
• prometheus-operator-app-chart -- to GS catalog, @glitchcrab talk to customer
• Aqua, to GS catalog -- @glitchcrab
• Ff up w @pipo02mix : k8s-initiator-app, to playground catalog. Decided to delete. When Fer has time he will add to Playground.

Playground status Mon, Mar 16, '20

• app-mesh (to playground)
• efk-logging - delete (removed from playground catalogs as well)
• efk-stack - move to Giant Swarm @webwurst will ping @glitchcrab for steps to do
• eventrouter (to playground)
• fluent-logshipping (to playground)
• giantswarm-todo (to playground)
• linkerd2 (to playground)
• loki-stack (to playground)

Next story: https://github.com/giantswarm/giantswarm/issues/8444

[piontec]

Please check the last column of https://github.com/orgs/giantswarm/projects/107 for a detailed list of apps that need to be moved.

[cokiengchiara]

Hi! Why did we close this again? In order to finish this, we:

Need to move all Incubator apps either to GS catalog or Playground catalog right?

We've made the decisions and set up the board, but haven't actually moved them yet :D

@puja108 before @piontec went on vacation we were talking about making sure that we do the announcement for Playground (cc and ty @Oshratn ) but prerequisite for that is that the apps are moved.

I suggest we put a date on this to get this done. Originally, Lukasz and I talked about asking SEs if moving from Incubator to Playground by Feb 7 is doable. Then I realized we didn't talk about moving the promised apps to GS catalog, which I understand has harder to reach quality bars.

1. Can we set a date for all apps to be moved?
2. For implementation, do you think Feb 7 is doable?

[piontec]

I closed as I thought this is only the "spec" part and actual actions are in other tickets. But I'm ok to keep it. As for the date - I think we need to ping all the responsible engineers again.

[cokiengchiara]

Thu, Mar 12, '20 update the status of incubator apps inside the issue. @piontec , just to confirm " efk-logging" will be deleted in favor of efk-stack, right? So nothing to do there?

[piontec]

Yeap, that's correct.

[puja108]

Is there anything blocking this? I feel this has somehow stalled and people are still getting confused by apps not being clearly in the catalogs they belong to. And in turn this is blocking the app catalog communication.
If it is taking too much time/work please consider a two step approach:

1. bring playground apps (redis and k8s/initiator) to playground
2. bring remaining incubator apps to GS

I would consider 1 as urgent to not confuse people, like e.g. Simon looking into incubator catalog and thinking we will support redis.

Not sure why the move is taking as technically changing the catalog of an app should be super quick. I can only imagine there will be issues with current customers using it. If that is the case I would propose:

1. move the apps to new catalog, but do NOT remove from incubator
2. hide incubator catalog (installed apps still work)
3. move customers to app in new catalog (even manually changing app CR is ok)
4. once all app CRs are migrated, delete incubator

you might need issues for each app

[cokiengchiara]

Is there anything blocking this?

@puja108 these are the only things remaining. Yasser and Cedric are doing that this cycle, it's not highest priority on their list. For Aqua, need to get someone from Halo to do it (I think) because Julien is on parental leave

kong-app — @yasn77 this cycle, to GS catalog
prometheus-operator-app-chart -- @ced0ps this cycle, to GS catalog
Ff up w @jgsqware : Aqua, to GS catalog

I'm not sure how technically challenging it is.

[cokiengchiara]

@cokiengchiara clarify w Puja and Lukasz this Playground <> GS Catalog thing

@yasn77 impression: If move to GS Catalog, additional work that is not necessary yet right now.

[puja108]

To be clear: this is only about changing the catalog not anything else, not quality bars, most additional work. And it is not technically challenging to change the catalog of an app.

The only challenge could be moving a customer that already has sth running from incubator, but as I explained above therte's a workaround for the migration phase.

The move itself without migration could be done within a single day (most probably not even a few hours) by a single person, if there's something I'm not seeing there please tell me. If not, I would not split it between multiple people but have someone just do it and create issues for people to followup with their customers.

[cokiengchiara]

To be clear: this is only about changing the catalog not anything else, not quality bars, most additional work. And it is not technically challenging to change the catalog of an app.

@puja108 @piontec with this change, https://github.com/giantswarm/giantswarm/pull/9651/files we're saying w alpha or beta, you need only go through basic maturity level. But still, that is some work. From what I hear Cedric and Yasser will still have zero time to work on that, for example, to create super basic documentation for Aqua and prometheus-operator if only because it is their first time being exposed to that app.

Just for the sake of deleting incubator, can we make it acceptable to just change the catalog, and not do any quality bar thingies?

[puja108]

2 things:

1. IMHO for sake of less confusion let's just change catalog
2. My thoughts behind having app catalog and playground were to have minimal hurdles between app dev and publishing as app. The goal was to make it the go to alternative to putting yaml/chart/something in a random repo and linking to it from docs/blog/content or sending to customers, but basic is already beyond that minimal goal.

as for 2, it was actually an area goal/kpi to "make it as easy and quick as possible to publish an app", this goal is now being prioritized lower by "ensuring (minimal) quality" here.

and again it feels like quality bars and catalogs do not really fit as they work towards different and sometimes opposing goals.

I can understand @piontec and halo team members pushing for quality and it's not wrong, but I'm also not sure how to solve this conundrum that the quality bars seem to be blocking publishing of ready to use things.

[cokiengchiara]

• IMHO for sake of less confusion let's just change catalog

I agree with this. @piontec

Step 1: Clean up catalog / delete incubator catalog. So just move what was in incubator to giantswarm without any quality improvements. This is a one time thing and we won't have to do anymore (unless we change catalog structure again haha)
Step 2: Separate tickets per app to achieve basic maturity

[piontec]

I'm OK with that. Let's unblock that cleanup finally.

[glitchcrab]

A note here for @webwurst - we need to clean this up. I'll coordinate with you.

$ kg charts -n giantswarm -o yaml elk | grep URL
  tarballURL: https://giantswarm.github.io/giantswarm-incubator-catalog/kubernetes-elastic-stack-elastic-logging-0.2.0.tgz

They can continue using this as it won't break anything, but obviously that app is now gone.

[cokiengchiara]

Remaining tasks

• Remove duplicate prometheus-operator and aqua in playground @glitchcrab
• move efk from playground to giantswarm - @webwurst or @glitchcrab

[glitchcrab]

I'm going to call this issue complete under the revised scope. Quality checklists will be completed under separate issues.