Support creating stacks with the same name across different endpoints

[zhaoyao91]

Description

I use portainer 1.15.0 to manage two endpoints. I cannot create two stacks with same name even in different endpoints.

Steps to reproduce the issue:

1. create a stack named 'test' in endpoint 1
2. try to create a stack named 'test' in endpoint 2
3. it errors: stack already exists with this name

[deviantony]

Yes, this is by design in order to ensure uniqueness for each stacks.

I.e, currently Portainer uses the stack name to create UAC on related resources (services, containers), so the name of the stack needs to be unique within the whole Portainer instance.

[zhaoyao91]

For our usage, each endpoint is a different cluster and environment. So you suggest I start dedicated portainer for each cluster?

[ncresswell]

Anthony, cant we just prefix the stack name with the endpoint name for internal functions (uac etc)? Rgds, Neil Cresswell On 2/11/2017, at 9:49 PM, Yao Zhao <notifications@github.com<mailto:notifications@github.com>> wrote: For our usage, each endpoint is a different cluster and environment. So you suggest I start dedicated portainer for each cluster? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_portainer_portainer_issues_1347-23issuecomment-2D341354674&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=V2DTvH_hRTycJgkaGCOuQ2_uSN79EQ8Kia1F40W3ipU&s=fRrTa5z5LLZi7EUdmmIUXk24xHEHzZMQB6bn1Y1aDC4&e=>, or mute the thread<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWGrlWUMRwiV-2DxzXwKIoRrNS4flZZGCyks5syYIEgaJpZM4QPUTj&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=V2DTvH_hRTycJgkaGCOuQ2_uSN79EQ8Kia1F40W3ipU&s=2ur_6iicbbJQ5Eoe5gmjJllitLB3PvLFPK1LrSEI9Gc&e=>.

[deviantony]

@zhaoyao91 you could prefix/suffix the stack name with the environment name, e.g. '-dev', '-qa'...

@ncresswell I'll re-open and investigate if we can do something here.

[zhaoyao91]

Prefix stack name is not a good workaround since each component should not know that it's in some environment. I will use different portainer instances to hold on for some time. Look forward to good news :)

[deviantony]

This was just an example, you just to need to create unique stack names. I understand the point tho and will investigate.

[WTFKr0]

Don't you add the swarmID for that purpose ?
If the two endpoints are not in same cluster, the SwarmID should be different
So the unicity should include that swarmID

I think the unicity check here can be done on stack ID instead of stack name :

portainer/api/http/handler/stack.go

Lines 158 to 163 in 693f131

	for _, stack := range stacks {
	if strings.EqualFold(stack.Name, stackName) {
	httperror.WriteErrorResponse(w, portainer.ErrStackAlreadyExists, http.StatusConflict, handler.Logger)
	return
	}
	}

[deviantony]

Yes, this was added in order to ensure uniqueness within Portainer. But when you retrieve services/containers associated to a stack, the only reference for that stack is the name of the stack in the com.docker.stack.namespace label.

When we store the UAC of a stack, we also use the stack name (tho we could use a combo of swarmID / stack name here) as the identifier associated to the resource control object.

At the moment, stack name + swarm ID is used to create a unique identifier. It was choose over stack name + endpoint ID because we should be able to retrieve the stacks of a cluster when connected to any node of the cluster (manager, worker...). If we used stack name + endpoint ID, then we would have only be able to lists the stacks available in the endpoint where they were created.

It'd be nice if containers/services would include the ID of the associated Swarm in the labels, but this is not avaible at the moment. I wonder if we could add this into Docker?

Edit: I believe this would need to be added in the swarmkit project as well as the moby project:

• https://github.com/moby/moby/blob/master/daemon/cluster/executor/container/container.go#L226
• https://github.com/docker/swarmkit/blob/master/api/objects.proto#L165

[deviantony]

Opened an issue in moby to be able to retrieve Swarm identifier from labels: moby/moby#36231

[fopina]

just bumped into this issue. is it still not possible to allow stacks with same names in different endpoints? isn't the uniqueness given by other attributes yet?

[arturslogins]

For me this is actual too :/

[yunfandev]

We also encountered this problem, when will it be fixed?

[ncresswell]

Not for at least 3 months. Rgds, Neil Cresswell On 23/03/2020, at 11:42 PM, yunfan <notifications@github.com> wrote:  We also encountered this problem, when will it be fixed? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_portainer_portainer_issues_1347-23issuecomment-2D602515431&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=LYdng6gT1l37i9iJgIpnVKWEA8LvMItgtv0leWqA9HU&s=3ZWgVdHbcaqZpyOhGQal9cD21YCP_EIatiWjwj9zssI&e=>, or unsubscribe<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AFQ2XFNKYM2BIAMRWZOYKT3RI44H7ANCNFSM4EB5ITRQ&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=LYdng6gT1l37i9iJgIpnVKWEA8LvMItgtv0leWqA9HU&s=oGivOhcVxG_oDHHF1GNd8paAjDgkbqS0Diflg64qaAg&e=>.

[RFBomb]

Just ran into this issue.
I originally had 3 stack set up, but I was migrating to a new system.

A RasPi3 running portainer has endpoints for a RasPi4 and a NAS server. I was migrating my stacks from the RasPi4 to the NAS, so for a short period of a couple hours I had 3 'end points' on my portainer instance. Once I migrated all the stacks over, I removed the endpoint for the Raspi4. So now I only have the Pi-3 and the NAS server as endpoints.

Trouble is, even though I removed the end point, I cannot use any stack named that were originally in that end point. To compound the issue, and since there doesn't seem to be a way to restore an end point, I have no way to delete the memory of those stack names, meaning they are forever forbidden it seems.

This really should be fixed, or atleast open up a way to manually clear them out for situations like mine.

[soletan]

Sorry for adding another same-here-post. But when in swarm context, a single portainer GUI is meant to be installed in that swarm, isn't it? In that case I only interact with that single portainer GUI which is meant to collect information from configured endpoints.

I am not familiar with other setups, for sure, but when using one swarm's portainer for accessing a remote swarm's resources using an edge agent I don't see the problem in relying on stack name + endpoint ID for uniquely identifying services of a stack. Even listing those in UI ain't no issue for I can't list stacks or services of multiple endpoints simultaneously, can I? Without selecting an endpoint there is no opportunity for showing resources bound to particular endpoints.

In my case this issue is limiting portainer's usability to manage and expose clusters of different customers in a single GUI for I can't stick to a similar pattern for setting up either customer's cluster.

[joonas-fi]

This has been very painful for me. Portainer supports multiple endpoints, i.e. many different clusters. IMO each separate cluster should be its freestanding entity - other clusters internal state should not impose limits on other clusters. Now it does.

Background: I use immutable infrastructure pattern to manage my servers, i.e. I set up a new cluster and migrate my services into the new one, then remove that old one. I've to increase stack name numbers just to migrate my services. It's a royal pain, and that effectively makes my stack names dynamic, i.e. if I need to, I can't rely on hardcoding stack name anywhere.

[deviantony]

Just an update on that topic, we have an implementation ready for this that is currently under testing: #4520