Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot deploy catalog apps which use multiple namespaces? #13057

Open
kenfdev opened this issue Apr 25, 2018 · 8 comments

Comments

@kenfdev
Copy link

commented Apr 25, 2018

Rancher versions:
rancher/server: v2.0.0-beta4-rc2
rancher/agent: v2.0.0-beta4-rc2

Infrastructure Stack versions:
kubernetes (if applicable): 1.10.1-rancher1

Docker version: (docker version,docker info preferred)
Both Rancher Server/Node

Client:
 Version:      17.03.2-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   f5ec1e2
 Built:        Tue Jun 27 03:35:14 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.03.2-ce
 API version:  1.27 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   f5ec1e2
 Built:        Tue Jun 27 03:35:14 2017
 OS/Arch:      linux/amd64
 Experimental: false

Operating system and kernel: (cat /etc/os-release, uname -r preferred)
Server

root@rancher-server:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
root@rancher-server:~# uname -r
4.4.0-53-generic

Node

root@ubuntu1:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.2 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.2 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

Setup details: (single node rancher vs. HA rancher, internal DB vs. external DB)
single node rancher and single worker node.

Steps to Reproduce:

  1. Go to catalogs and add custom url of OpenFaaS(https://github.com/openfaas/faas-netes.git)
  2. Create openfaas, openfaas-fn namespace and assign to project
  3. Deploy OpenFaaS via the catalog ( set functionNamespace=openfaas-fn for the answers ). Remember to use the existing namespace ( openfaas ) when you deploy it.

Results:

Rancher Server logs

configmap "prometheus-config" created
configmap "alertmanager-config" created
serviceaccount "faas-controller" created
service "gateway" created
deployment "gateway" created
clusterrole "faas-controller" created
clusterrolebinding "faas-controller" created
clusterrolebinding "faas-controller-fn" created
service "alertmanager" created
deployment "alertmanager" created
service "faas-netesd-external" created
service "gateway-external" created
2018/04/25 05:29:23 [INFO] Updating service [gateway-external] with public endpoints [[{"port":31112,"protocol":"TCP","serviceName":"openfaas:gateway-external","allNodes":true}]]
2018/04/25 05:29:23 [INFO] Updating pod [openfaas/gateway-64cb969d44-krdm2] with public endpoints [[{"port":31112,"protocol":"TCP","serviceName":"openfaas:gateway-external","allNodes":true}]]
service "prometheus-external" created
2018/04/25 05:29:23 [INFO] Updating service [prometheus-external] with public endpoints [[{"port":31119,"protocol":"TCP","serviceName":"openfaas:prometheus-external","allNodes":true}]]
service "alertmanager-external" created
service "nats-external" created
service "faas-netesd" created
deployment "faas-netesd" created
2018/04/25 05:29:27 [INFO] Updating node [ubuntu1] with public endpoints [[{"nodeName":"c-mmf7x:m-7653519801fb","addresses":["192.168.0.78"],"port":31119,"protocol":"TCP","serviceName":"openfaas:prometheus-external","allNodes":true},{"nodeName":"c-mmf7x:m-7653519801fb","addresses":["192.168.0.78"],"port":31112,"protocol":"TCP","serviceName":"openfaas:gateway-external","allNodes":true},{"nodeName":"c-mmf7x:m-7653519801fb","addresses":["192.168.0.78"],"port":80,"protocol":"TCP","podName":"ingress-nginx:nginx-ingress-controller-2clkn","allNodes":false},{"nodeName":"c-mmf7x:m-7653519801fb","addresses":["192.168.0.78"],"port":443,"protocol":"TCP","podName":"ingress-nginx:nginx-ingress-controller-2clkn","allNodes":false}]]
service "nats" created
deployment "nats" created
service "prometheus" created
deployment "prometheus" created
2018/04/25 05:29:29 [INFO] Updating pod [openfaas/prometheus-59c5db458f-m4zwq] with public endpoints [[{"port":31119,"protocol":"TCP","serviceName":"openfaas:prometheus-external","allNodes":true}]]
2018/04/25 05:29:29 [INFO] Updating pod [openfaas/prometheus-59c5db458f-m4zwq] with public endpoints [[{"port":31119,"protocol":"TCP","serviceName":"openfaas:prometheus-external","allNodes":true}]]
deployment "queue-worker" created
E0425 05:29:30.266321       1 generic_controller.go:204] AppController project-stmsz/openfaas [helm-controller] failed with : Kubectl apply failed. Error: error: the namespace from the provided object "openfaas-fn" does not match the namespace "openfaas". You must pass '--namespace=openfaas-fn' to perform this operation.
: exit status 1

When I last tried to deploy with Rancher2.0 beta3, it failed with a message saying the release name openfaas doesn't match openfaas-fn or something like that (sorry, I don't have the log right now). I know this is still an early version of beta4 but it seems to be able to deploy most of the resources.

image

I couldn't narrow down the issue to see who was causing this error hence opened this issue. The catalog deployment succeeds if you don't set the functionNamespace=openfaas-fn answer probably because the deployment only affects a single namespace openfaas.

I'm assuming the ServiceAccont is failing to be deployed. The template is here.

{{- $functionNs := default .Release.Namespace .Values.functionNamespace }}
{{- if (ne .Release.Namespace $functionNs) }}
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: {{ template "openfaas.name" . }}
    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
    component: faas-controller
    heritage: {{ .Release.Service }}
    release: {{ .Release.Name }}
  name: faas-controller
  namespace: {{ $functionNs | quote }}
{{- end }}

Thank you in advance. I'm really liking this workflow in 2.0!

@StrongMonkey

This comment has been minimized.

Copy link
Member

commented Apr 26, 2018

@kenfdev the original design was to deploy apps in a single namespace and prevent resources being created in another namespace. I was thinking that we could add an option to allow user ignore the constraint.

@deniseschannon deniseschannon added this to the v2.1 milestone Apr 26, 2018
@kenfdev

This comment has been minimized.

Copy link
Author

commented Apr 26, 2018

@StrongMonkey Thanks for the response! Yes, I was thinking that was the case.

I was thinking that we could add an option to allow user ignore the constraint.

Looking forward to this change. I assume OpenFaaS isn't the only app that deploys to multiple namespaces. We're isolating the functions from the core resources by separating the namespaces.

@cjellick cjellick modified the milestones: v2.1, v2.2 Aug 22, 2018
@cjellick cjellick modified the milestones: v2.2, Backlog Sep 18, 2018
@alexellis

This comment has been minimized.

Copy link

commented Jan 12, 2019

@kenfdev you don't have to deploy OpenFaaS to multiple namespaces, but I think that it's preferable.

Alex

@pietervogelaar

This comment has been minimized.

Copy link

commented Oct 2, 2019

The same for linkerd2 (https://helm.linkerd.io/edge):

I get the following error:

Failed to install app linkerd. Error: release linkerd failed: resource's namespace kube-system doesn't match the current namespace linkerd
@pietervogelaar

This comment has been minimized.

Copy link

commented Oct 2, 2019

@kenfdev the original design was to deploy apps in a single namespace and prevent resources being created in another namespace. I was thinking that we could add an option to allow user ignore the constraint.

Is this option already available?

@wmorgan

This comment has been minimized.

Copy link

commented Oct 2, 2019

Yeah this is a bit unfortunate for Linkerd because we are trying to do the Right Thing with scoping our RBAC to the minimal required permissions, instead of using a blanket ClusterRole. cc @grampelberg

@rbq

This comment has been minimized.

Copy link

commented Oct 17, 2019

Same with the Stash backup operator. :(

Failed to install app stash. Error: release stash failed: resource's namespace kube-system doesn't match the current namespace stash

@StrongMonkey

This comment has been minimized.

Copy link
Member

commented Oct 17, 2019

@cjellick I think we should probably re-think the design. Assign this issue to you so it can be put into our backlog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
10 participants
You can’t perform that action at this time.