Operator creates too many repositories

[kyal2000]

Hello,
while developing some routes the operator creates very much Integration-kits and new repositories in the registry.

So my questions are:

When does the operator create a new kit/repository?
How can i avoid this circumstance and get more control over my repos?
Thanks in advance

[squakez]

I think you can find answers in #2736 for the question related to IntegrationKits. We use incremental builds, so, there may be several layers.

[catshout]

Hi @squakez thanks for pointing us this out. I have some questions so far

• Can we re-user an existing Integration Kit as it contains no sources itself?
• Is it ok to cleanup a created Integration Kit at the k8s cluster and the dependent repository in the the registry after the build process?
• Is it possible to perform a local build (or on a build server) upfront and deploy the built image afterwards?

Background of these question is that we need to have a production grade platform. Remaining Integration Kits and Repositories with no function won't even work as we're running ouf of ressources.

Best
Gerald

[squakez]

Hi @squakez thanks for pointing us this out. I have some questions so far

No problem, I'll try answering inline.

* Can we re-user an existing Integration Kit as it contains no sources itself?

Just copying the description of documentation related to the IntegrationKit:
"The IntegrationKit is a fundamental side resource which describe a container image created by the camel-k operator as well as the configurations that need to be applied to every integration that is executed on top of it. An IntegrationKit does not include any source code or resource file defined by the Integration from which it has been generated."

* Is it ok to cleanup a created Integration Kit at the k8s cluster and the dependent repository in the the registry after the build process?

In theory you should not delete IntegrationKit generated by the platform. If you run kamel kit delete, infact, you won't be allowed to do that. In practice, you can perform a deletion via kubectl integrationkit delete. While we are reasoning about the best way to garbage collect/compact interim IntegrationKit, I think you can use this strategy with the due caution. If you come out with some idea on how to perform the clean, you know, contributions are welcome. We have open a few issues around this theme: the aforementioned #2736, then #2732 and #593

* Is it possible to perform a local build (or on a build server) upfront and deploy the built image afterwards?

I think this is something similar of what was described in #2737.

Background of these question is that we need to have a production grade platform. Remaining Integration Kits and Repositories with no function won't even work as we're running ouf of ressources.

Yeah, it's a complex topic which need to be revisited. Thanks for popping it up.

Best Gerald

[squakez]

By the way, you may have also been affected by #3007 which may have caused the incremental build not to work at all in the latest Camel K versions. Your question made me dig and discover this bug, thanks for it!

This wrong behavior could be an additional reason to the high amount of containers you have experienced lately.

[catshout]

Thanks @squakez for the reply. Let me summarize my understanding ..

1. So an Integration Kit is 1:1 related to an Integration and is NOT re-usable for another Integration. Is my understanding right?
2. We will proceed with the workaround you suggested kubectl integrationkit delete to cleanup the space after the integration is running. Same with the deletion of the corresponding repository. Is the ready state of the Integration Kit an indicator that this can be done with no side effects?
3. What does the platform Integration Kit perform when installing the Camel-K Operator?

Best
Gerald

[squakez]

No problem.

About point 1, that is not correct. The Kit should be reused as a base layer for Integrations that are similar (for instance, a similar set of dependencies). Let's say you develop this integration:

kamel run Sample.java

And then, another integration such as

kamel run Sample2.java -d camel-zipfile

We expect, the second integration to have an IntegrationKit built on top of the first IntegrationKit as there is only 1 dependency difference between both. Unfortunately, you may have experienced #3007, which lately made the incremental build to be broken.

About point 3, I am not sure to understand the question. Do you refers to the [IntegrationPlatform](https://camel.apache.org/camel-k/1.8.x/architecture/cr/integration-platform.html), maybe?

[catshout]

@squakez I was wrong with point 3, the Integration Kit was the one for my test Integration.

Are there standards in which namespaces the Integration Kits are coming up?

What's about point 2? And here, when I'm going to delete the repository, will this be re-created?

Best
Gerald

[squakez]

The IntegrationKits are created in the same namespace where the Integration is executed. About point 2, there could be side effects if you have some other integration that is using such IntegrationKit. Make sure to delete only those Kits that have no Integration attached (ie, the interim ones that are coming from incremental tests you may have done on an Integration). The IntegrationKit won't be recreated, unless some new Integration with the same requirements will need to do so.

[catshout]

Hi @squakez I did just perform the following commands

kamel uninstall --all --global
k delete ns camel-k
k delete ns integration
k create ns camel-k
k create ns integration
kcn integration
k create secret generic kaniko-secret --from-file=/home/cas-dev-gke/kaniko-secret.json -n camel-k
kamel install --build-publish-strategy=Kaniko --registry gcr.io --organization cas-dev-gke --registry-secret kaniko-secret --global --force -n camel-k
kamel run hello.groovy

The current namespace is integration. See

k config get-contexts
*         gke_cas-dev-gke_europe-west3-a_cas-dev-cluster-1                                         gke_cas-dev-gke_europe-west3-a_cas-dev-cluster-1   gke_cas-dev-gke_europe-west3-a_cas-dev-cluster-1                        integration

Further I checked the Integration Kit and the Integration

k get integration -A
NAMESPACE     NAME    PHASE     KIT                        REPLICAS
integration   hello   Running   kit-c83ac3s8es090jfq3rig   1

k get integrationkit -A
NAMESPACE   NAME                       PHASE   TYPE       IMAGE
camel-k     kit-c83ac3s8es090jfq3rig   Ready   platform   gcr.io/cas-dev-gke/camel-k-kit-c83ac3s8es090jfq3rig:209947

So the Integration Kit was being created in the namespace camel-k, the integration in the namespace integration.

Do you have any suggestions what might be wrong here?

2 further questions ..

1. What does the Integration Kit exactly do once an Integration is up and running?
2. What is the function of the Kit Builder, e.g. camel-k-kit-c83ac3s8es090jfq3rig-builder`? Could this be removed after the build has done?

[squakez]

All seems good. As you've installed the operator globally (--global), then certain resources (IntegrationKit) will be created in the operator namespace.

What does the Integration Kit exactly do once an Integration is up and running?

The kit will be used once again if that Integration Pod needs to be rescheduled or as an incremental layer for any further kit which has a similar set of dependencies.

What is the function of the Kit Builder, e.g. camel-k-kit-c83ac3s8es090jfq3rig-builder`? Could this be removed after the build has done?

A Builder is a configuration required by the build task to perform its operation. I think it can be removed once it has finished its duties. However I think it would be safer to let the operator perform those kind of cleaning (when this consider necessary) in order to avoid tampering the consistency of the whole system.

[catshout]

Tx @squakez for the replies. The overall picture becomes more and more clear now.

Could we influence the repository name and/or Integration Kit name when being created? This would help in terms of clarity which one is related to which configuration.

Is there a way to figure out all running integrations to a given Integration Kit?

So is there actually any kind of cleaning the operator is providing (e.g. garbage collection)? Or will be provided within next version?

We want to use all this stuff into production and so far we need to be clear what's necessary for a stable CI/CD with less ressources as possible.

P.S. Many questions, I can offer that we're going to contribute by extending the documentation. May be others would have the same questions.

[catshout]

@squakez One further question from an architecture point of view.

If an Integration Kit is being used by multiple Integrations, the Integration itself isn't self contained; as it has dependencies to an Integration Kit. The relation between Integration Kits and Integrations is 1:n, what I got from your answers.

How does this fit into a micro service pattern? As I understand the micro service approach such a micro service should be autonomous. The first autonomous layer with Camel-K are multiple integrations dependent to one Integration Kit.

What are the reasons behind this? How could be reached an autonomy of single Integrations?

[squakez]

Tx @squakez for the replies. The overall picture becomes more and more clear now.

Glad to help and clarify.

Could we influence the repository name and/or Integration Kit name when being created? This would help in terms of clarity which one is related to which configuration.

No, this is not possible. The name is autogenerated and it calculate hashes in order to simplify the comparison between kits.

Is there a way to figure out all running integrations to a given Integration Kit?

Not automatically. You can get the Integration spec and figure it out the Kit used there.

So is there actually any kind of cleaning the operator is providing (e.g. garbage collection)? Or will be provided within next version?

Not yet, but it's definetely under the radar. See #2736

We want to use all this stuff into production and so far we need to be clear what's necessary for a stable CI/CD with less ressources as possible.

P.S. Many questions, I can offer that we're going to contribute by extending the documentation. May be others would have the same questions.

Contributions are welcome as usual!

[squakez]

@squakez One further question from an architecture point of view.

If an Integration Kit is being used by multiple Integrations, the Integration itself isn't self contained; as it has dependencies to an Integration Kit. The relation between Integration Kits and Integrations is 1:n, what I got from your answers.

How does this fit into a micro service pattern? As I understand the micro service approach such a micro service should be autonomous. The first autonomous layer with Camel-K are multiple integrations dependent to one Integration Kit.

What are the reasons behind this? How could be reached an autonomy of single Integrations?

Sharing a base layer is a tradeoff required to improve the speed for build/execution and to reduce the resource needed to store the images. I don't think it affect the autonomy of the microservice, given that the Kit is immutable. Also, when working with containers you always use some layer built in top of another. This is exactly the same concept.

[catshout]

Tx @squakez that makes the concept behind more clear as well.

One of the main concepts behind Camel-K is to run the builds of an integration on the cluster, maintained by the Camel-K Operator. Nevertheless, is there a way when everything has been tested in test environment and an appropriate Integration Kit (built sometime before) is existing in the production cluster to perform the Integration image build somewhere else as at the production cluster?

Background is that we want to separate the image build from the production runtime? This would make even more sense as we want to use the native build option introduced in Camel-K 1.7 that is resource consuming.

[github-actions]

This issue has been automatically marked as stale due to 90 days of inactivity.
It will be closed if no further activity occurs within 15 days.
If you think that’s incorrect or the issue should never stale, please simply write any comment.
Thanks for your contributions!