Error during unshare(...): Operation not permitted

[madmesi]

Description :
my Sample.java camel-k integration on kubernetes failed:

I installed camel -k with command line:
kamel install --registry=myregistry.example.com --force

    - kubectl : Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:56:40Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

    - kubeadm version: kubeadm version: &version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:54:15Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
    
    - Docker: ce-19.03.4
    - os: centos 7 
    - kernel: 3.10.0-1127.8.2.el7.x86_64
    - I have 3 masters and 5 worker nodes
    My integration file :
    Sample.java

import org.apache.camel.builder.RouteBuilder;

public class Sample extends RouteBuilder {
  @Override
  public void configure() throws Exception {
          from("timer:tick")
        .log("Hello Camel K!");
  }
}

I used to have this error in the (error state) pod:
Error:

Error during unshare(CLONE_NEWUSER): Invalid argument
User namespaces are not enabled in /proc/sys/user/max_user_namespaces.
level=error msg="error parsing PID \"\": strconv.Atoi: parsing \"\": invalid syntax"
level=error msg="(unable to determine exit status)"

after doing echo 2147483647 > /proc/sys/user/max_user_namespaces on all nodes error changed to:

Error:

Error during unshare(...): Operation not permitted
level=error msg="exit status 1"

Is there something that I've missed?
Thanks in advance for helping.

[spenceclark]

I'm getting the same problem also.

Where would I run "echo 2147483647 > /proc/sys/user/max_user_namespaces"? It sounds like this needs to be run on the nodes
but I'm using a managed kubernetes from DigitalOcean, so I don't have that kind of access to the underlying nodes.

But even doing that doesn't seem to fix the problem.

[astefanutti]

I suspect this is caused by Buildah running into a container runtime that's too much constrained.

Could you provide the output of:

$ kubectl get integrationplatforms.camel.apache.org -o yaml

and:

$ kubectl get nodes -o jsonpath="{.items[*].status.nodeInfo}"

Otherwise, what container runtime is used in you cluster.

Some context can be found in containers/buildah#1901.

A possible work-around would be to use Kaniko instead of Buildah.

[spenceclark]

This first command return this:

apiVersion: v1
items:
- apiVersion: camel.apache.org/v1
  kind: IntegrationPlatform
  metadata:
    creationTimestamp: "2020-07-08T11:57:51Z"
    generation: 1
    labels:
      app: camel-k
    name: camel-k
    namespace: camel
    resourceVersion: "47746528"
    selfLink: /apis/camel.apache.org/v1/namespaces/camel/integrationplatforms/camel-k
    uid: f157b75a-a042-49b9-a375-dd4dba4e6e3e
  spec:
    build:
      kanikoBuildCache: false
      maven:
        settings: {}
      registry:
        address: registry.gitlab.com/portemis/integration
        organization: camel
        secret: gitlab-reg
    cluster: Kubernetes
    profile: Kubernetes
    resources: {}
  status:
    build:
      baseImage: adoptopenjdk/openjdk11:slim
      buildStrategy: pod
      kanikoBuildCache: false
      maven:
        localRepository: /tmp/artifacts/m2
        settings:
          configMapKeyRef:
            key: settings.xml
            name: camel-k-maven-settings
        timeout: 3m45s
      persistentVolumeClaim: camel-k
      publishStrategy: Buildah
      registry:
        address: registry.gitlab.com/portemis/integration
        organization: camel
        secret: gitlab-reg
      runtimeVersion: 1.3.0
      timeout: 5m0s
    cluster: Kubernetes
    phase: Ready
    profile: Kubernetes
    version: 1.0.1
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

and the second one returns this:

map[architecture:amd64 bootID:d4ba7862-4330-4d3e-9b65-9ae6b401402c containerRuntimeVersion:docker://18.9.2 kernelVersion:4.19.0-0.bpo.6-amd64 kubeProxyVersion:v1.17.6 kubeletVersion:v1.17.6 machineID:6600638a8bf3461d8c19889aeba510de operatingSystem:linux osImage:Debian GNU/Linux 9 (stretch) systemUUID:6600638a-8bf3-461d-8c19-889aeba510de] map[architecture:amd64 bootID:4f7e95b0-8bb4-47b7-92a7-9d72e344e3ba containerRuntimeVersion:docker://18.9.2 kernelVersion:4.19.0-0.bpo.6-amd64 kubeProxyVersion:v1.17.6 kubeletVersion:v1.17.6 machineID:1dcab06f655a43b2b9451bce9b84379d operatingSystem:linux osImage:Debian GNU/Linux 9 (stretch) systemUUID:1dcab06f-655a-43b2-b945-1bce9b84379d]

[astefanutti]

Thanks, that confirms Buildah with the Docker container runtime is the problem.

From containers/buildah#1901, it seems a system call, that's forbidden by default with the Docker container runtime, is still necessary when the user has no CAP_SYS_ADMIN in the container.

The only option seems to change the Docker container runtime to use a different seccomp profile, e.g. docker run --security-opt seccomp=/usr/share/containers/seccomp.json, but that requires being able to configure your cluster container runtime.

A work-around is to use other builder strategy, like Kaniko or Spectrum, with kamel install --build-publish-strategy=kaniko or by editing your IntegrationPlatform directly.

[lburgazzoli]

@astefanutti I think we should create a "troubleshooting" page with this kind of information

[astefanutti]

@lburgazzoli right, good idea. I've just created #1595 for it.

[madmesi]

Thanks Guys for responding. I'm a bit confused here. I tried to install camel-k following the operatorhub and this. and I still don't know which one helps me to create the integration.

After your response I tried removing the "olm" namespace followed by the kamel uninstall command.
And then I went through the procedure with
kamel install --registry https://myregistry.example.com/v2 --registry-auth-username YOUR_USERNAME --registry-auth-password SECRET_PASSWORD --build-publish-strategy=Kaniko --cluster-setup

But this is what I got after looking for the integrationplatform:

kubectl get integrationplatforms.camel.apache.org -o yaml
apiVersion: v1
items: []
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

[astefanutti]

@madmesi the cluster-setup option only performs CRDs and cluster roles installation. So you may remove that option to have the operator set up.

[astefanutti]

Documentation has been provided with #1627. Let me close this. Feel free to re-open if needed.