How code libraries are being handled with Release Manager?

[felipecruz91]

Is your feature request related to a problem? Please describe.
We have a use case where two components need to use a Python library.

• Component A - Dockerplain quickstarter.
  The Jenkinsfile will build, test and deploy the component to OpenShift.

• Component B - Dockerplain quickstarter.
  The Jenkinsfile will build, test and deploy the component to OpenShift.

• Library - Python quickstarter.
  The Jenkinsfile will package and upload the library to Nexus (it will not be deployed to OpenShift using odsComponentStageRolloutOpenShiftDeployment() because of the own nature of the component).

It would look like this:

Component A --- uses ---> Library
Component B --- uses ---------^

And the Release Manager is configured as follows:

repositories: 
  - id: ComponentA
    type: ods
  - id: ComponentB
    type: ods
  - id: Library
    type: ods-service

When deploying to D using Release Manager, we get the following error:

java.lang.RuntimeException: DeploymentConfigs (component: 'Library') found that are not ODS managed: '[Library]'!
Please fix by rolling them out through 'odsComponentStageRolloutOpenShiftDeployment()'!

And that's expected - the Library component is a Python library (not an app that needs to run in OpenShift as a container).
So we have removed the DeploymentConfig from the Tailor template file, leaving it practically empty (no objects defined):

apiVersion: template.openshift.io/v1
kind: Template
objects:
parameters:
- name: TAILOR_NAMESPACE
  required: true
- name: ODS_OPENSHIFT_APP_DOMAIN
  required: true

Describe the solution you'd like
I'd like to learn more about your opinion on how ODS could easily cover the use case aforementioned. Would it make sense to create a "Library quickstarter" whose Jenkinsfile only takes care of uploading the lib to Nexus, and won't include the odsComponentStageRolloutOpenShiftDeployment() stage?

Describe alternatives you've considered
So far, the workaround I know to deploy a Python Library in ODS is neither easy nor straightforward to the end-user:

1. Create a Python quickstarter.
2. Edit the Jenkinsfile to compress the package, upload it to Nexus and comment out the deployment stage:

    stageCompressPythonLibrary(context)
    odsComponentStageUploadToNexus(context, [repository: 'candidates'])
    // odsComponentStageRolloutOpenShiftDeployment()

3. Make a first deployment with Release Manager (it will fail)
4. Rename the openshift-exported folder to openshift and empty the openshift/template.yml file:

apiVersion: template.openshift.io/v1
kind: Template
objects:
parameters:
- name: TAILOR_NAMESPACE
  required: true
- name: ODS_OPENSHIFT_APP_DOMAIN
  required: true

5. Remove the OpenShift resources (DeploymentConfig, Service, Routes, etc) that were created by the first deployment. If you don't do this step, you will get the following exception when deploying to D:

java.lang.RuntimeException: DeploymentConfigs (component: 'interfacepackage') found that are not ODS managed: '[interfacepackage]'!
Please fix by rolling them out through 'odsComponentStageRolloutOpenShiftDeployment()'!

If you've reached this point, thank you for your time reading this😀

ping @michaelsauter @metmajer

[michaelsauter]

Thanks for the clear write-up!

The issue with a "library quickstarter" is that you would need to have one per tech stack ... which would result in lots of duplication because the difference to a deployed component is not that big. I would think a recipe in the documentation how to do it would be enough. Another alternative would be to have "quickstarter options" so that during provisioning, one can choose whether deployment should be included or not.

That said, the probably bigger question is how library repos are treated for documentation purposes .... there I don't know how it works.

[felipecruz91]

I understand your concern about the duplication and it is totally valid. Yet, from my experience, I can tell that sometimes even writing a recipe, would not be enough for the end-users to successfully complete this (sadly).

My take is that the fewer manual steps the end-user needs to do, the better (as this will reduce the likelihood of making a mistake somewhere). So, the alternative you propose about the "quickstarter options" could be a nice solution: end-users could check a "library" checkbox in the Prov App form so that the Jenkins template applied during provisioning will then include the upload to Nexus and exclude the deployment to OpenShift.

Then, in the Release Manager part, set a new ods type such as ods-library that will be used to treat documentation differently for this kind of component if necessary.

[metmajer]

@felipecruz91 let's please not focus on library quickstarters or checkboxes yet. Although convenient, they add additional other complexities and changes in places where we don't want to make them.

@michaelsauter do you see quick manual workaround or are we running into fundamental design decisions of ODS here?

[michaelsauter]

From a technical perspective, no workaround is required. Simply:

• remove the rollout stage
• delete the DeploymentConfig and Service
• add the upload to nexus stage to make your lib available to other components

The only question I see is how documentation of that lib should be handled in the release manager ... if you don't list the lib in the RM repo, you are fine technically, but maybe some documentation should be generated for it, and I'm not sure how all that works exactly right now (e.g. you might not want a TIP/R but you might want content in the SSDS).

[metmajer]

Cheers Michael, great to know that technical blockers can be removed so easily. I am with you... I am also not sure if having a library modelled as a "Technical Component" is the correct way. I'll figure this out.

FYI @clemensutschig

[clemensutschig]

I am taking this one up - and have a proposal :)

[clemensutschig]

It's not that easy btw - because the RM will try to deploy something that is not there :) (or create useless artifacts... )

I will introduce a new component type (ods-library) - that will only be triggered in assemble mode, and here you can do sort of what you want (e.g. a custom agent build for example, or upload to nexus, or...) - I have a usecase for this, which I know a couple product teams have as well .. and that is a custom builder. Today you have to deploy this outside of any lifecycle to make this fly

We will also NOT create a TIR (and mark the component as NOT installed by ODS - because it will not be installed on QA or PROD)

@michaelsauter @serverhorror @metmajer - thoughts...?

[michaelsauter]

I guess what I referred to above was that I am not sure whether this library should be handled at all by the release manager: one way to handle it would be to treat a library like an external dependency. Meaning it is developed and documented "somewhere else" and then released. The application is then simply consuming the new version.

If viewed like this, the release manager doesn't need to list the repo in the YAML file and therefore doesn't even know about it.

The downside with this is that many libraries will likely be used by just one team, in which case it may be more convenient to have also the documentation and issue tracking etc. in one project. If you then do list it as a repo in the RM, I think something like you suggested @clemensutschig is necessary because then "the RM will try to deploy something that is not there" ;)

Not sure if that helps, but I don't have further thoughts on this right now ...

[clemensutschig]

@michaelsauter @metmajer @s2oBCN take a look at #805 - comments more than welcome

[clemensutschig]

I tested it with the RM - and declared a dependency from the compoent running with the custom agent to the custom agent - worked super sweet. and the docs look good now as well

[clemensutschig]

@felipecruz91 - it's fixed in master :)