Ldap integration; Logout throws "ERR_EMPTY_RESPONSE" error; "runtime error: invalid memory address or nil pointer dereference"

[kodyabbott]

What happened?

I have added LDAP to a newly provisioned Grafana server and when trying to sign out with a user I logged in using LDAP I cannot signout and get the "ERR_EMPTY_RESPONSE" in the browser

From the /var/log/grafana/grafana.log there are multiple blocks of this same error:

logger=context userId=2 orgId=1 uname=email@domain.com t=2023-12-22T16:42:56.936147254Z level=error msg="Request error" error="runtime error: invalid memory address or nil pointer dereference" stack="runtime/panic.go:261 (0x5035ef7)\nruntime/signal_unix.go:861 (0x5035ec5)\ngithub.com/grafana/grafana/pkg/api/login.go:263 (0x87af8a2)\ngithub.com/grafana/grafana/pkg/api/response/web_hack.go:40 (0x615608f)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:131 (0x6148b2e)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6148bd2)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6148bd2)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6148bd2)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/services/contexthandler/contexthandler.go:145 (0x87cf79b)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6148bd2)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6148bd2)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6148bd2)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6148bd2)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6148bd2)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/render.go:44 (0x61496bd)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6148bd2)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6148bd2)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6148bd2)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6148bd2)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/middleware/csrf/csrf.go:66 (0x7e1eb7a)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/middleware/recovery.go:179 (0x87a9f75)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/middleware/loggermw/logger.go:72 (0x7e1fb96)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/middleware/request_metrics.go:83 (0x672d735)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/middleware/request_tracing.go:88 (0x87aabd5)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/middleware/requestmeta/request_metadata.go:66 (0x87ce835)\nnet/http/server.go:2136 (0x538ee28)\ngithub.com/grafana/grafana/pkg/web/context.go:52 (0x6147616)\ngithub.com/grafana/grafana/pkg/web/router.go:155 (0x614ad6d)\ngithub.com/grafana/grafana/pkg/web/router.go:211 (0x614b8d4)\ngithub.com/grafana/grafana/pkg/web/macaron.go:163 (0x6148ffc)\nnet/http/server.go:2938 (0x5391b0d)\nnet/http/server.go:2009 (0x538d9f3)\nruntime/asm_amd64.s:1650 (0x50553e0)\n"

my ldap.toml is pretty simple too:

[[servers]]
host = "dc.domain.int"
port = 389
use_ssl = false
start_tls = false
ssl_skip_verify = true
bind_dn = "DOMAIN\\%s"
search_filter = "(sAMAccountName=%s)"
search_base_dns = ["dc=domain,dc=int"]

[servers.attributes]
member_of = "memberOf"
email =  "mail"

# Map ldap groups to grafana org roles
[[servers.group_mappings]]
group_dn = "cn=cloudteam,ou=dept,dc=domain,dc=int"
org_role = "Admin"
grafana_admin = true

[[servers.group_mappings]]
group_dn = "*"
org_role = "Viewer"

and lastly the ldap section from grafana.ini:

[auth.ldap]
enabled = true
config_file = /etc/grafana/ldap.toml
allow_sign_up = true
skip_org_role_sync = false

I noticed someone else is having a similar issue that was posted 2 days ago but they are on Windows OS but everything else is the same as me: #79753

Lastly I have a HAR file which I can share but would rather send it directly to someone.

What did you expect to happen?

Successfully signout and load http://hostname:3000/logout without issue

Did this work before?

Yes, when I log in and out with local admin I can successfully still with no issues, even with ldap turned on

How do we reproduce it?

1. Enable ldap using Microsoft AD as the host
2. Sign in as a new ldap user
3. Try to sign out

Is the bug inside a dashboard panel?

No response

Environment (with versions)?

Grafana: Version 10.2.3 (commit: 1e84fed, branch: HEAD)
OS: amzn2023.x86_64
Browser: Firefox 121.0 (64-bit)

Grafana platform?

A package manager (APT, YUM, BREW, etc.)

Datasource(s)?

No response

[viharm]

I've had LDAP integration for a long time. I recently upgraded to 10.2.3 (deployed as a Docker container), and when I logout, I encounter a 502 error. I simply reverted to 10.2.2 and it works fine.

[yeshodhan]

facing the same issue with latest version: 10.2.3

logger=context userId=4 orgId=1 uname=user1 t=2023-12-27T15:39:44.533166097Z level=error msg="Request error" error="runtime error: invalid memory address or nil pointer dereference" stack="runtime/panic.go:261 (0x546e5d7)\nruntime/signal_unix.go:861 (0x546e5a5)\ngithub.com/grafana/grafana/pkg/api/login.go:263 (0x8c1d2a2)\ngithub.com/grafana/grafana/pkg/api/response/web_hack.go:40 (0x65919ef)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:131 (0x658448e)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/extensions/ratelimiting/ratelimiting.go:54 (0x96e882c)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6584532)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6584532)\nnet/http/server.go:2136(0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6584532)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/services/contexthandler/contexthandler.go:145 (0x8c3d19b)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6584532)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6584532)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6584532)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6584532)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6584532)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/render.go:44 (0x658501d)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/middleware/subpath_redirect.go:23 (0x8c174e5)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6584532)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6584532)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6584532)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6584532)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/middleware/csrf/csrf.go:66 (0x828b0da)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/middleware/recovery.go:179 (0x8c17975)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/middleware/loggermw/logger.go:72 (0x828c0f6)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/middleware/request_metrics.go:83 (0x6b96495)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/middleware/request_tracing.go:88 (0x8c185d5)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/middleware/requestmeta/request_metadata.go:66 (0x8c3c235)\nnet/http/server.go:2136 (0x57c77e8)\ngithub.com/grafana/grafana/pkg/web/context.go:52 (0x6582f76)\ngithub.com/grafana/grafana/pkg/web/router.go:155 (0x65866cd)\ngithub.com/grafana/grafana/pkg/web/router.go:211 (0x6587234)\ngithub.com/grafana/grafana/pkg/web/macaron.go:163 (0x658495c)\nnet/http/server.go:2938 (0x57ca4cd)\nnet/http/server.go:2009 (0x57c63b3)\nruntime/asm_amd64.s:1650 (0x548dac0)\n"

[kodyabbott]

Seems like a lot was done on /pkg/api/login.go recently.

I am not familiar with Go but seems there were a few changes in the sIgn out redirect URL in these commits:
#75269
#79271
#79635

[DavideyLee]

I have encountered the same issue. help

[luketanti]

Same issue here. 10.2.3 LDAP Logout gives 502 error

[mgyongyosi]

Hey, it's Misi from the IAM team! Sorry for the inconvenience, but unfortunately, there is a bug introduced with v10.2.3 which has already been fixed on the 10.3 branch (will be released in the second half of January). Short term, we suggest to roll back Grafana to v10.2.2.

[IvanZenger]

Hey, it's Misi from the IAM team! Sorry for the inconvenience, but unfortunately, there is a bug introduced with v10.2.3 which has already been fixed on the 10.3 branch (will be released in the second half of January). Short term, we suggest to roll back Grafana to v10.2.2.

In version 10.2.2, the logout function works (no nil pointer error, no 502 Status, UI-works), but certain error messages still occur. Does the fix in version 10.3.0 address these issues?

logger=http.server t=2024-01-18T07:09:17.120230165Z level=info msg="Successful Logout" User=ivanzenger@hallo.ch

logger=context userId=862 orgId=1 uname=zengeriv t=2024-01-18T07:09:17.120307557Z level=info msg="Request Completed" method=GET path=/logout status=302 remote_addr=172.16.48.39 time_ms=12 duration=12.755702ms size=29 referer=https://grafana/dashboards handler=/logout

logger=http.server t=2024-01-18T07:09:17.157047409Z level=error msg="Failed to parse user ID" error="identifier is not initialized"

logger=context userId=0 orgId=0 uname= t=2024-01-18T07:09:17.762102425Z level=info msg= error="[auth.unauthorized] cannot authenticate request" remote_addr=172.16.48.36 traceID=

logger=context userId=0 orgId=0 uname= t=2024-01-18T07:09:17.762173722Z level=info msg="Request Completed" method=GET path=/api/live/ws status=401 remote_addr=172.16.48.36 time_ms=0 duration=134.254µs size=102 referer= handler=/api/live/ws

logger=context userId=0 orgId=0 uname= t=2024-01-18T07:09:18.477685228Z level=info msg= error="[auth.unauthorized] cannot authenticate request" remote_addr=172.16.48.39 traceID=

Im Asking because we have a CVE and need to upgrade to 10.2.3.

[dfaltum]

Same phenomenon with version 10.4.0 and LDAP + OAUTH2 auth provider:

logger=http.server t=2024-04-09T15:09:07.199171357Z level=error msg="Failed perform proper logout" error="user token not found"
logger=http.server t=2024-04-09T15:09:07.199280627Z level=info msg="Successful Logout" userID=2
logger=context userId=2 orgId=1 uname=user t=2024-04-09T15:09:07.200406739Z level=error msg="Request error" error="runtime error: invalid memory address or nil pointer dereference" stack="runtime/panic.go:261 (0x56ac0f7)\nruntime/signal_unix.go:861 (0x56ac0c5)\ngithub.com/grafana/grafana/pkg/api/login.go:266 (0x9316912)\ngithub.com/grafana/grafana/pkg/api/response/web_hack.go:40 (0x6c202af)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:131 (0x6c12b6e)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6c12c12)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6c12c12)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6c12c12)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/services/contexthandler/contexthandler.go:137 (0x9338881)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6c12c12)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6c12c12)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6c12c12)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6c12c12)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6c12c12)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/render.go:44 (0x6c136fd)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6c12c12)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6c12c12)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6c12c12)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x6c12c12)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/middleware/csrf/csrf.go:66 (0x8929bda)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/middleware/recovery.go:180 (0x931097d)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/middleware/loggermw/logger.go:72 (0x892abf6)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/middleware/request_metrics.go:75 (0x70b7db5)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/middleware/request_tracing.go:88 (0x9311655)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/middleware/requestmeta/request_metadata.go:66 (0x93379d5)\nnet/http/server.go:2136 (0x5a07608)\ngithub.com/grafana/grafana/pkg/web/context.go:52 (0x6c11656)\ngithub.com/grafana/grafana/pkg/web/router.go:155 (0x6c14dcd)\ngithub.com/grafana/grafana/pkg/web/router.go:211 (0x6c15934)\ngithub.com/grafana/grafana/pkg/web/macaron.go:163 (0x6c1303c)\nnet/http/server.go:2938 (0x5a0a2ed)\nnet/http/server.go:2009 (0x5a061d3)\nruntime/asm_amd64.s:1650 (0x56cb600)\n"

[steverweber]

after taking a quick look at /api/login.go:268 for grafanav11.0.

i assume "redirect" is set as nil.. perhaps related to "msg="Failed perform proper logout" error="user token not found"
	redirect, err := hs.authnService.Logout(c.Req.Context(), c.SignedInUser, c.UserToken)
failing on.
	c.Redirect(redirect.URL)

Note i have

enable_login_token = true