Admin UI seems to be protected by a new ACL.

[darron]

Compiled from master - we're testing a new sink for go-metrics:

[staging]root@i-c972d41e:~# consul version
Consul v0.5.2-110-g4933843
Consul Protocol: 3 (Understands back to: 1)

I'm getting this error trying to view the web ui:

2015/06/30 20:28:47 [ERR] http: Request /v1/internal/ui/nodes?dc=us-east-1-staging&token=<hidden>, error: ACL not found

Based on what I read here: #1024 - I've updated some of the ACLs:

[staging]root@i-c972d41e:~# curl -s http://127.0.0.1:8500/v1/acl/list?token=$TOKEN | jq .[]
{
  "ID": "default-agent-token-removed",
  "Name": "",
  "Type": "client",
  "Rules": "key \"\" {\n  policy = \"read\"\n}\nkey \"_rexec/\" {\n  policy = \"read\"\n}\nkey \"config/\" {\n  policy = \"read\"\n}\nservice \"\" {\n policy = \"write\"\n}"
}
{
  "ID": "anonymous",
  "Name": "",
  "Type": "client",
  "Rules": "key \"config/\" {\n  policy = \"read\"\n}\nservice \"\" {\n  policy = \"read\"\n}"
}

Any idea what I'm missing?

[darron]

If it matters, our ACL default policy is also allow:

https://gist.github.com/darron/820e2718331c056eb5b3

[darron]

We ended up going back to 0.5.2 proper and all of the problems with the webui went away.

[ryanuber]

@darron hmm, this is interesting, the service discovery ACL's shouldn't affect this, and I couldn't reproduce this locally. Did you ever figure out what was going on? If you have a config I can reproduce with, then I'll definitely get this fixed before a new release.

[darron]

It was very odd - we had additional ACL issues that we couldn't work around - which was why we reverted and backed down.

Will wait to test with the next release.

[ryanuber]

I think I've actually seen this before now that I look at it closer. If you aren't using a token while browsing around the UI, the javascript code still tries to send the ?token parameter, and javascript's undefined value gets passed. Thus, when trying to look up the undefined token, Consul responds with an error that it can't find the token. Not sure if what you were seeing is the same thing.

@darron can you share any of the other ACL issues? Consul 0.6 is going to have a number of ACL enhancements, so if we can catch any bugs early on that would be great!

[darron]

Hmm - not sure if that was it - I definitely didn't have a query string on my urls and I could always see the services and nodes pages.

The other ACL issue was just that we were getting an rpc error: rpc error: Permission denied - when we really shouldn't have been.

Once your team gets closer to an RC we'll take another look.

I'm not sure it wasn't where we built our binary from - I wasn't around for that - maybe we missed an important patch or something.

[slackpad]

Figured this out (pasted from this post):

Because they're both on localhost - it was passing our production token - the last UI I used - which was why it was giving a 403. I found it in Local Storage:

http://shared.froese.org/2015/8njjz-12-52.jpg

Once I changed it to the proper token it worked everywhere.

Before - I could always load the homepage - it would error on the KV store and I would update the token in settings.

Looks like I can get to the settings page no matter what token:

http://shared.froese.org/2015/rydfv-12-58.jpg

If you pass a non-existent token - essentially what it was doing - should it error or use the anonymous one? With my settings - the anonymous token does work to at least view the page.

There's an interesting last question on there. If passed a token that doesn't exist should we log an error and attempt using the anonymous token as a default, instead of returning an error? Or maybe if the token is not found we should return a better message and surface it in the UI that the token is invalid.

[slackpad]

Also now that the default experience in the web UI can 403, we might want to make that better if it happens to better explain what's going on.

[darron]

Yeah - it's a bit of a tough call. I know you scan scope cookies by directory - that way the key for:

http://localhost:8500/ui/#/us-east-1-staging/
http://localhost:8500/ui/#/us-east-1-prod/

Could be different - but I think the # makes it not a real directory and am not sure if cookies can use that.

Regardless - you're using LocalStorage Maybe you could query local storage by dc name? That way you could have multiple keys that wouldn't collide.

May not be worth it though - just some thoughts.

[pauzed]

I believe that we're experiencing this issue currently after upgrading our Consul cluster from 0.5.2 to 0.6.0. All other features of 0.6.0 appear to be working, but our UI fails to load due to the "undefined" token Javascript is passing. Here are our relevant snippets of configuration:

"datacenter": "myDC
"acl_datacenter": "myDC",
"acl_default_policy": "deny",
"acl_token": "myToken"

And our relevant ACL:
{
"ModifyIndex": 58221546,
"CreateIndex": 15271,
"Rules": "# Default all keys to read-only\nkey "" {\n policy = "read"\n}\n# Default all services to allowing registration\nservice "" {\n policy = "write"\n}\nkeyring = "read"\nevent "" {\n policy = "write"\n}",
"Type": "client",
"Name": "RO",
"ID": "myToken"
}

When visiting the UI, one of the requests throws a 403 forbidden here: :8500/v1/internal/ui/nodes?dc=myDC&token=undefined I believe DC is getting properly picked up from the default.json config, but the token is not. Viewing :8500/v1/internal/ui/nodes in the browser properly returns an expected json payload, visiting ::8500/v1/internal/ui/nodes?dc=myDC returns a proper json payload, as does :8500/v1/internal/ui/nodes?dc=myDC&token=myToken. Curling these endpoints also all result in the expected json payloads. I see under Chrome Developer Tools that there is a token key in my local storage with no value. Changing this value to myToken has not helped. In the browser, I've tried passing dc and token like such with no success either:

:8500/ui/?dc=myDC&token=myToken
:8500/ui?dc=myDC&token=myToken
:8500/ui/#/?dc=myDC&token=myToken
:8500/ui/#?dc=myDC&token=myToken

Please let me know what other information I can pass along to assist in reproducing the issue.

[darron]

Yeah - I'm having this problem again on my upgraded 0.6 cluster as well -
still wanting to track it down but have been on other things lately and
haven't been able to get to it.

On Wed, Dec 16, 2015 at 12:25 PM pauzed notifications@github.com wrote:

I believe that we're experiencing this issue currently after upgrading our
Consul cluster from 0.5.2 to 0.6.0. All other features of 0.6.0 appear to
be working, but our UI fails to load due to the "undefined" token
Javascript is passing. Here are our relevant snippets of configuration:

"datacenter": "myDC
"acl_datacenter": "myDC",
"acl_default_policy": "deny",
"acl_token": "myToken"

And our relevant ACL:
{
"ModifyIndex": 58221546,
"CreateIndex": 15271,
"Rules": "# Default all keys to read-only\nkey "" {\n policy =
"read"\n}\n# Default all services to allowing registration\nservice ""
{\n policy = "write"\n}\nkeyring = "read"\nevent "" {\n policy =
"write"\n}",
"Type": "client",
"Name": "RO",
"ID": "myToken"
}

When visiting the UI, one of the requests throws a 403 forbidden here:
:8500/v1/internal/ui/nodes?dc=myDC&token=undefined I believe DC is getting
properly picked up from the default.json config, but the token is not.
Viewing :8500/v1/internal/ui/nodes in the browser properly returns an
expected json payload, visiting ::8500/v1/internal/ui/nodes?dc=myDC returns
a proper json payload, as does
:8500/v1/internal/ui/nodes?dc=myDC&token=myToken. Curling these endpoints
also all result in the expected json payloads. I see under Chrome Developer
Tools that there is a token key in my local storage with no value. Changing
this value to myToken has not helped. In the browser, I've tried passing dc
and token like such with no success either:

:8500/ui/?dc=myDC&token=myToken
:8500/ui?dc=myDC&token=myToken
:8500/ui/#/?dc=myDC&token=myToken
:8500/ui/#?dc=myDC&token=myToken

Please let me know what other information I can pass along to assist in
reproducing the issue.

—
Reply to this email directly or view it on GitHub
#1071 (comment).

[oswell]

I manually hacked in the change above on my systems and the UI started loading again. As stated in the PR, there may be further issues beyond this, but I haven't run across anything else yet.

[slackpad]

@oswell thanks for finding that - I merged it and I think you got them all (the service discovery ACL was added to that internal endpoint in 0.6).

[pauzed]

@oswell @darron @slackpad I can confirm that this worked for me. I updated my application.min.js locally on a single node (both under /var/lib/consul/ui/application.min.js and /var/lib/consul/ui/static/application.min.js), restarted consul on that node, and able to see the UI as expected. Updating my token key in my local session data from empty, to an invalid token properly gives me a 403, and setting a correct token under this key provides me with the access I expect from my acl.

[slackpad]

Thanks @pauzed!

[hopperd]

Is the only workaround for this to manual edit the application.min.js file? When will a UI release be published to contain this fix?

[slackpad]

Hi @Split3 we are working on a release that should go out this week and contain the fix. You can get the latest Consul master and run the build process to generate the UI files without any manual edits.

[hopperd]

Since this issue is still open I'm assuming this didn't make it out into the 0.6.3 or the 0.6.2 release is that correct?

[slackpad]

@Split3 this made it into the released web_ui files for 0.6.1+ (https://releases.hashicorp.com/consul/0.6.3/consul_0.6.3_web_ui.zip). If you use -ui-dir and point to that release all should be good.

Unfortunately, right after the 0.6.3 release we discovered that the internal -ui assets were missing this fix (#1467), so you won't be able to use that until the next release. This comment should provide awareness of that issue, but since it's already fixed in master I'll close this.

[hopperd]

I just upgraded one of our clusters to 0.6.3 and we use the -ui-dir attribute and the downloaded resources for that release didn't contain this fix. I had to manually patch it to get it to work properly.

[slackpad]

@Split3 can you share the patch you are applying?

[hopperd]

The patch I used was the one mentioned in this thread above.

[hopperd]

Let me verify that our configuration infrastructure is pulling down the correct artifacts. As I just verified in the zip that indeed the patch is in there, so it must be something with our configuration system.

[hopperd]

Ok, I verified it is a bug in our configuration system. Fix looks good, thanks!

[slackpad]

@Split3 awesome - thanks for the update!