External authentication

[awlx]

Environment

• Python version: 3.5.4
• NetBox version: Example: v2.3.2

Ability to use external authentication via a proxy (eg. Keycloak-proxy, Nginx, Apache) by setting the appropriate headers. It was already described as a possibility here but I find no documentation or anything about this topic.

Proposed Functionality

It should be possible to pass authentication headers to netbox and either add a user automatically to the local database and make it possible to give the correct permissions in netbox or use the permissions passed via a header.

Use Case

This would be very useful because it wouldn't be necessary to implement and maintain dozens of different authentication mechanisms directly to Netbox but just pass the headers.

There is already an example for this in the wild but it requires some code changes and patching Netbox on every update is in my opinion not a good idea. So it would be really cool if something like this could be implemented in the codebase.

Example:
https://groups.google.com/d/msg/netbox-discuss/BTB8q8CzmrA/2BcnbectAQAJ

[jeremystretch]

It sounds like this should be folded into #1989. The idea also needs to be fleshed out a lot more:

• What are the proposed headers?
• How do you handle user creation and deletion?
• Illustrate how an existing tool would employ this functionality to authenticate a user
• Any conflicts between this and the existing UI/API authentication

[awlx]

Yep, sounds like those two could be solved together.

• What are the proposed headers?
  Most common seem to be REMOTE_USER and Authorization. Maybe it would be useful to have some other headers to pass E-Mail address or something.
  Or the way Graylog does it with configurable Headers:
  http://docs.graylog.org/en/2.4/pages/users_and_roles/external_auth.html#single-sign-on

• How do you handle user creation and deletion?
  I would propose to just add any existing user and no automatic deletion process in Netbox itself. Since nobody without external could log into Netbox anyway and a few users shouldn't slow down Netbox.
  Other solution is to not create the user at all and just work with the forwarded permissions but I think this needs a bigger code change?

• Illustrate how an existing tool would employ this functionality to authenticate a user
  Icinga2 and Graylog already have the ability to do so. See here:
  https://www.icinga.com/docs/icingaweb2/latest/doc/05-Authentication/#external-authentication
  http://docs.graylog.org/en/2.4/pages/users_and_roles/external_auth.html#single-sign-on

• Any conflicts between this and the existing UI/API authentication.
  That is a good question I cannot answer at the moment. Basically the code needs to skip the login page of the UI when it detects the header and the option to do external auth is enabled in the config. Maybe we should also add a variable to only allow certain hosts to pass these strings otherwise it could add some security issues if we just accept the REMOTE_USER header from everywhere ;).

[rsepulvedacl]

Grafana has a very useful way to authenticate users. I'm writing a proxy based in PHP and cURL and it's working very well.

I'm using SimpleSAMLphp to authenticate users when they try to connect to Grafana through this proxy. Once authenticated, the proxy passes some user's data to Grafana through the headers.

You can see some documentation from Grafana here: http://docs.grafana.org/tutorials/authproxy/.

I'd like to see something similar into Netbox. 😃

[cimnine]

What are the proposed headers?

I think the exact names should be configurable.

Any conflicts between this and the existing UI/API authentication.

Additionally to checking the authentication cookie, Netbox would also need to check for the header.
But I could see problems with the API, as it relies on headers for authentication.

[jeremystretch]

I would propose to just add any existing user and no automatic deletion process in Netbox itself.

The user also needs to be assigned permissions and/or groups in order to have any write access.

Other solution is to not create the user at all and just work with the forwarded permissions but I think this needs a bigger code change?

A valid user must be created in the database. This is a requirement of the framework and schema.

[awlx]

The user also needs to be assigned permissions and/or groups in order to have any write access.

Maybe if the group which is forwarded matches an existing group in Netbox, we could work with those permissions? Otherwise just create the user with a default group (configurable).

[globoudou]

I'm on similar project, trying to use MS Kerberos Auth with Netbox. Globally it works with the following configuration :

• reverse proxy apache 2.4.6 perform the kerb auth on /login url as follow :
  AuthType GSSAPI
GssapiCredStore /etc/keytab
Require valid-user
RewriteEngine On
RewriteCond %{LA-U:REMOTE_USER} (.+)
RewriteRule . - [E=RU:%1]
RewriteRule "^(.*)$" "/"
RequestHeader set KAUTHUSER "%{RU}e" env=RU
RequestHeader unset Authorization

• add django RemoteUser in netbox/settings.py :
  AUTHENTICATION_BACKENDS = [ 'django.contrib.auth.backends.RemoteUserBackend', ]
MIDDLEWARE = ( ... , 'django.contrib.auth.middleware.PersistentRemoteUserMiddleware', ... )

• change REMOTE_USER with KAUTHUSER in django/contrib/auth/middleware.py :
  header = "HTTP_KAUTHUSER"

In reverse-proxy configuration we can't deal with REMOTE_USER, because django is looking for a system variable, not an http header. This is the reason why apache rewrite it to KAUTHUSER http header.

Forcing the authentication in Netbox (LOGIN_REQUIRED = True), it works as expected. The user's samaccountname@REALM must be provisioned in Netbox, and the SSO is working.

The problem is when the user try to use the "logout" button. The redirect scheme is :
/logout/ ==302==> / ==302==> /login/?next=/ ==500
It failed with error 500 :
_thread._localobject as no attribute changed_objects in netbox/extras/middleware.py line 28
After that I must restart the Netbox instance to log again.

I don't understand what is the problem...

[cimnine]

PerssitentRemoteUserMiddleware

Not sure if typo on Github or in your config, but this should probably be PersistentRemoteUserMiddleware, as per Documentation (Using Remote User on Login Pages Only).

[globoudou]

PerssitentRemoteUserMiddleware

Not sure if typo on Github or in your config, but this should probably be PersistentRemoteUserMiddleware, as per Documentation (Using Remote User on Login Pages Only).

Yes... error on keyboard... it was correct in my conf.

[DanSheps]

Just to add on here, I would love if Duo was supported.

[cimnine]

You would only be able to do this using a OIDC, OAuth or SAML reverse proxy, for example the Keycloak Security Proxy. That's as far as the proposed implementation would go.

[rsepulvedacl]

I could use Shibboleth on Apache, doing the following changes in the NetBox code:

I modified the file /opt/netbox/netbox/netbox/settings.py, adding the following lines:

MIDDLEWARE = (
[...]
    'utilities.middleware.CustomRemoteUser', # I added this line at the end of the list
)
[...]
AUTHENTICATION_BACKENDS = [ 'django.contrib.auth.backends.RemoteUserBackend', ] # I added this line at the end of the file

I also added the following lines at the end of /opt/netbox/netbox/utilities/middleware.py:

from django.contrib.auth.middleware import RemoteUserMiddleware

class CustomRemoteUser(RemoteUserMiddleware):
    header = 'HTTP_REMOTE_USER'

In the file /opt/netbox/netbox/netbox/configuration.py, I changed to False the following line:

LOGIN_REQUIRED = False

You can also use another external authentication method, changing the header if necessary, and you can use redirections in Apache for /login and /logout URL's.

These changes are based on @globoudou's comment.

[awlx]

@rsepulvedacl I tried to reproduce your setup on v2.5.2 but for some reason it is not working at all.

Am I missing a step from your description? I changed everything you mentioned and added the following to my nginx config (for now it's static for tests).

proxy_hide_header Authorization; proxy_set_header HTTP_REMOTE_USER 'netbox';

The user netbox exists and is able to login if I change everything back to the previous state.

[rsepulvedacl]

@awlx, try it again without the HTTP_ prefix:

proxy_hide_header Authorization;
proxy_set_header REMOTE_USER 'netbox';

You could be really passing HTTP_HTTP_REMOTE_USER to NetBox with your settings. If for some reason, this doesn't work, change the header settings in both: nginx and /opt/netbox/netbox/utilities/middleware.py, but you should preserve the HTTP_ prefix in this last file.

I only tested this with Apache, using Shibboleth. The setting I used on Apache is the following:

RequestHeader set REMOTE_USER expr=%{REMOTE_USER}

You can also try with or without quotes around the user name. Good luck!

[rsepulvedacl]

@awlx, I don't know nginx very well, but I found some settings that could be useful here:
https://developer.okta.com/blog/2018/08/28/nginx-auth-request#bonus-who-logged-in

[aficustree]

@awlx , what did you end up getting to work? I need to do something similar but instead use OAUTH2. I was going to use the nginx oauth2 proxy and then make the changes to middleware/settings/configuration.py as above. It looks like that plugin pulls the UPN of the auth'd user to $upstream_http_x_auth_request_user which I'd assume we set as the REMOTE_USER header. Waiting on the app to be enabled by our directory team but curious if you ended up getting your use-case working.

[leoluk]

We ended up implementing a custom OAuth provider via social_django by using a custom settings.py file.

If you want to try something similar, maybe this helps:

https://gist.github.com/leoluk/16d91ec22d833945c7ac7ed2b3b05a27

Hoping for a proper upstream hook.

[aficustree]

very clean set of monkey patches. well done. almost see how this could indeed be generalized into a proper extensible authentication framework.

[aficustree]

@leoluk , curious, followed your gist using the AzureAD social Django backend but how do I actually trigger the backend to fire? I see the custom url as ^oauth/ but the login page doesn't seem to hit it and if I just goto localhost/oauth/ I get a 404 with not matching the pattern

^oauth/ ^login/(?P<backend>[^/]+)/$ [name='begin']
^oauth/ ^complete/(?P<backend>[^/]+)/$ [name='complete']
^oauth/ ^disconnect/(?P<backend>[^/]+)/$ [name='disconnect']
^oauth/ ^disconnect/(?P<backend>[^/]+)/(?P<association_id>\d+)/$ [name='disconnect_individual']

feel like I'm close but just missing something obvious.

my backends are set as follows:

AUTHENTICATION_BACKENDS = (
    'social_core.backends.azuread_tenant.AzureADTenantOAuth2',
    'django.contrib.auth.backends.ModelBackend',
)

[leoluk]

Something like LOGIN_URL = '/oauth/login/azuread/' should take care of redirecting you to the proper login page (you would have to check the Django Social docs on how exactly the backend is called).

[aficustree]

ah thanks, that was the tip I needed to look at the social Django code and discovered I should have used oauth/login/azuread-tenant-oauth2/

[aficustree]

for some reason the login button doesn't seem to pick up the LOGIN_URL changes. No idea why but it just keeps with the regular login page. It works if I go direct to the URL but not if I click the login button.

[bluikko]

Should the modifications in #2328 (comment) work at 2.5.13?

I have authentication at Apache working but Netbox does not seem to do anything with those changes - the user is not with admin rights and the "login" feature works as normal.

[rsepulvedacl]

@bluikko, the modifications should work. Please consider that this method won't make any changes to login and logout buttons. You should consider to create a redirection and/or use rewrite on Apache.

It's been a long time since I implemented this, so I don't remember whether users are created automatically or not, but I remember that users won't be administrators automatically, unless you make some more improvements to this monkey patch or decide to make them administrators by hand.

[bluikko]

@rsepulvedacl I see. Thanks. Unfortunately it is a problem for me if the user permissions would need to be manually managed.

I hope the developers would consider this enhancement request and integrate it properly with LDAP.

[davidc]

I don't think bloating Netbox away from its core functionality is helpful or maintainable or a good use of development time, nor a wise decision as messing up authentication/authorisation is potentially very dangerous.

Leave authentication to the web server front-ends which already have plenty of options available, in a mature state of development.

For example, why on earth would someone devote the huge amount of effort to implementing SAML authentication in Netbox (#1677) instead of simply using e.g. Apache2 with mod_auth_mellon in front (as I do). Same for #118.

The options are countless and, no matter how many are implemented in Netbox, there will always be calls for more, so why even go down this route?

But, Netbox does needs to have better support for already-externally-authorised users:

1. Respect the headers (that the administrator has explicitly defined in Netbox configuration) for already-authorised users.

2. A way to automatically create the Netbox internal user when a new already-authorised user is detected. The admin should be able to specify in the config file what Netbox groups such a user will be added to.

3. If external auth is in use, disable internal login/logout pages and allow customisation of the target URL of login/logout links (or remove the login/logout links entirely at the administrator's option).

4. Documentation on how to configure this, which should be plainly obvious for anyone looking for "another way to auth to Netbox". And documentation on how to bootstrap this process - i.e. how to create or rename an existing admin user to their SSO username.

I have implemented #2328 (comment) and it works fine for 1, but it should be a configuration option rather than an edit to core files that will be lost on update. Then the "loose ends" 2-4 should be implemented.

[leoluk]

The nice thing about Django is that is has a large ecosystem of high quality authentication backends which can easily be used with Netbox, like in the snippet above or my openshift-netbox role: https://github.com/leoluk/openshift-netbox/blob/master/openshift/netbox/openshift_auth.py

These don't belong in the core, but a well-defined set of hooks for users to configure their own authentication would solve most of these use cases. In my particular case, these are the only modifications made:

• Overwriting the logout URL such that, in addition to terminating the user session, the user gets logged out of an external IdP (https://github.com/leoluk/openshift-netbox/blob/master/openshift/netbox/openshift_auth.py)

• For any login provider which uses callbacks, the callback URL needs to be accessible before the user is logged in. Right now, this requires a modification to LoginRequiredMiddleware and the root URLs (https://github.com/leoluk/openshift-netbox/blob/master/openshift/netbox/openshift_middleware.py and https://github.com/leoluk/openshift-netbox/blob/master/openshift/netbox/openshift_urls.py).

• Various extra settings.py configurations to load the provider, enable and configure it (https://github.com/leoluk/openshift-netbox/blob/master/openshift/netbox/settings.py).

I agree with @davidc that HTTP headers are the best approach for "generic" authentication.

[CrackerJackMack]

I have a working docker, kubernetes, nginx+oauth2_proxy and netbox related patches working so it is optionally enabled. How would you like these pull requests? One per feature? All in one? My works are based on v2.6.9 but I'm sure I could port them to develop.

[sdktr]

I have a working docker, kubernetes, nginx+oauth2_proxy and netbox related patches working so it is optionally enabled. How would you like these pull requests? One per feature? All in one? My works are based on v2.6.9 but I'm sure I could port them to develop.

I think only the netbox -part is desired in this repo. A netbox PR including documentation on how to enable/configure it. Maybe a seperate issue in the 'netbox-docker' repo outlining how you did it with your k8s based setup?

[bootc]

@CrackerJackMack I'd like to have your K8s components please to integrate into bootc/netbox-chart. This is a setup I would very much like to be able to use and a scenario I'd like to have in the chart.

[jeremystretch]

I've been going over the discussion here and in other issues, trying to determine the scope for this change. I think it boils down to two different approaches:

1. Extend NetBox's internal authentication to recognize a certain HTTP header (e.g. REMOTE-USER) set by a reverse proxy (nginx or Apache) and assign the current user based on its value.
2. Install a Django app to provide external authentication (similar to how django-auth-ldap works currently).

As the second case should be handled by #3351 (plugin support), this issue should focus on the first case: HTTP header-based authentication. Of course, a complete implementation of this feature entails not only user assignment, but user creation and group assignment (for granting of permissions) as well.

There are some good references linked in the chat above, such as Grafana's implementation, but we need to thoroughly define the login/logout workflow before any serious progress can be made.

[jeremystretch]

It's surprising this hasn't gotten any more feedback in the past month. We really need to work on scoping this out fully and defining a working model for this to be implemented. Would anyone like to volunteer to take the lead? We need at least a few solid use cases for reference I think.

[kobayashi]

If no objection here, I handle this to implement a header for proxy authentication.

[kobayashi]

1. Extend NetBox's internal authentication to recognize a certain HTTP header (e.g. REMOTE-USER) set by a reverse proxy (nginx or Apache) and assign the current user based on its value.
2. Install a Django app to provide external authentication (similar to how django-auth-ldap works currently).

For this issue, agree to focus on the first one to authenticate with a proxy like apache or nginx in front of netbox.

[CrackerJackMack]

I don't have time for an official PR (sorry). But I'm are using https://github.com/pusher/oauth2_proxy with https://github.com/divio/django-simple-sso with some very minor alterations to netbox.

There is also https://github.com/agoragames/nginx-google-oauth as well as https://github.com/cloudflare/nginx-google-oauth which have different headers.

For hosted solutions there is https://teams.cloudflare.com/access/index.html and https://cloud.google.com/iap/

I'll see if I can post a gist of the patch this weekend. Maybe a good starting point for a more generic solution to support more auth proxies.

edit: mispoke about django-simple-sso, this was a previous attempt

[bluikko]

@CrackerJackMack maybe I am wrong but at least some of your links seem to do authentication with external sources. I believe that this issue is about using the "preauthentication" headers set by the web server?

For example I am using GSSAPI where the web server sets REMOTE_USER HTTP header and all NetBox would need to do for basic support is to check for presence of that header.

It would be good if NetBox could be told to automatically give administrator rights to the users. Several web apps that I administer can split authentication and authorization so that authentication is done based on the HTTP headers and then authorization is done in the web app, for example by using LDAP to check group memberships of the user in REMOTE_USER.
This would be great but I'd even be happy if just the basic case would be supported.

[CrackerJackMack]

@bluikko You are only partially wrong I believe. They specifically exist to auth, and set HTTP headers. All the links I provided act as a trusted reverse proxy which you can trust that authentication and authorization was handled prior to setting and sending the HTTP headers. In the simplest, most crude examples:

Google/Cloudflare ---> nginx ---> gunicorn(django)

nginx ---> oauth2_proxy ---> gunicorn(django)

nginx --> ngx_http_auth_request_module[---> oauth2_proxy] --> gunicorn(django)

Depending on the solution and if it's configurable with that solution, all of them set HTTP headers with we inherently trust. This choice is based entirely on this snippet from https://docs.djangoproject.com/en/3.0/howto/auth-remote-user/

Warning

Be very careful if using a RemoteUserMiddleware subclass with a custom HTTP header. You must be sure that your front-end web server always sets or strips that header based on the appropriate authentication checks, never permitting an end-user to submit a fake (or “spoofed”) header value. Since the HTTP headers X-Auth-User and X-Auth_User (for example) both normalize to the HTTP_X_AUTH_USER key in request.META, you must also check that your web server doesn’t allow a spoofed header using underscores in place of dashes.

This warning doesn’t apply to RemoteUserMiddleware in its default configuration with header = 'REMOTE_USER', since a key that doesn’t start with HTTP_ in request.META can only be set by your WSGI server, not directly from an HTTP request header.

If you need more control, you can create your own authentication backend that inherits from RemoteUserBackend and override one or more of its attributes and methods.

The not-pr-worthy gist I promised. It also includes a more complicated nginx auth_request module setup to defer to oauth2_proxy using a side request.
https://gist.github.com/CrackerJackMack/8f0e84b2d6ca981f4262b5fa5136375c

[sdktr]

Hi @CrackerJackMack , shouldn’t the ‘user normalization’ be part of the frontend proxy’s job? I think it opens up a whole can of possible rules that end up beging maintained in Netbox on how to interpret the header?

[CrackerJackMack]

@sdktr Funny enough, the proxies would argue that it's the application's problem to clean up things if desired 😆

This is one of those annoying external authentication issues everyone likes to gloss over in their "Have them login with facebook!" blog posts. The issue isn't a issue with external authentication on a technical level, but more so that you have no idea how the login system works externally or what it will return in the headers and/or cookie data.

There are some agreed upon standards to help adoption, but they aren't a hard rule.

First, we have to choose an upstream auth service and see what header is does, or can send back to us. The auth service could do oauth2, SAML, openid connect, LDAP, ... We don't really know how users actually login to the service. Depending on the external authentication the headers could be a combination or just a single set of the following headers

• X-Goog-Authenticated-User-Email & X-Goog-Authenticated-User-Id
• X-Forwarded-User & X-Forwarded-Email
• X-Auth-Request-User and X-Auth-Request-Email response
• Cf-Access-Authenticated-User-Email & Cf-Access-Jwt-Assertion 2
• Whatever header you want to foward
• skipping JWT validation
• skipping referrer userinfo only headers
• skipping azure or SAML examples for now

Next problem. Are the username and email forwarded to us usable as is? Well that's really dependent on the downstream application (netbox & administrator). If usernames are intended to be emails then you just use X-Forwarded-Email and ignore -User for example. are the username and email prefixed as to help identify the upstream: accounts.google.com:username@gsuite.domain.com?

I had a 3rd point but I can't recall what it is right now.

[jeremystretch]

I think it makes sense to implement this feature by introducing two new abilities.

First, I propose implementing a custom subclass of Django's RemoteUserBackend. We can provide hooks into this class in the form of configuration parameters to control attributes such as:

• The HTTP header name
• Whether to automatically create new user accounts on login
• Groups/permissions to assign to the user
• Username "cleaning" (e.g. adding/removing prefixes)

The configuration parameter for this might look something like:

REMOTE_AUTH_CONFIG = {
    'HTTP_HEADER': 'REMOTE_USER',
    'AUTO_CREATE': True,
    'DEFAULT_GROUPS': [...],
    'DEFAULT_PERMISSIONS': [...],
}

The remote authentication backend will be enabled only if REMOTE_AUTH_CONFIG is defined. (The current ViewExemptModelBackend will still be in place as a fallback for local authentication.) This will likely address the majority of use cases for this feature.

The second component is to allow the injection of a custom backend class, to be inserted at the beginning of AUTHENTICATION_BACKENDS. This would take the form of a separate configuration parameter, e.g.

REMOTE_AUTH_BACKEND = 'path.to.my.custom.backend'

REMOTE_AUTH_CONFIG and REMOTE_AUTH_BACKEND would be mutually exclusive to avoid confusion.

I believe this strikes a nice balance between built-in support and custom extensibility, and should be fairly low-effort to maintain long-term.

[jeremystretch]

In the interest of moving the conversation along, I've introduced PR #4299. This largely implements what I described, however I opted to use individual configuration parameters rather than a dictionary for easier validation. The following can be used to enable and configure built-in remote user authentication:

REMOTE_AUTH_ENABLED = True
REMOTE_AUTH_HEADER = 'HTTP_REMOTE_USER'  # Default
REMOTE_AUTH_AUTO_CREATE_USER = True  # Default
REMOTE_AUTH_DEFAULT_GROUPS = ['Network Engineering', 'Operations']
REMOTE_AUTH_DEFAULT_PERMISSIONS = ['extras.run_script']

This is sufficient to:

1. Configure the HTTP header which indicates the name of the authenticated user
2. Automatically create the user if it doesn't already exist
3. Automatically assign arbitrary groups/permissions

If further customization is needed, there is another setting which can be used to swap out the built-in backend with a custom one:

REMOTE_AUTH_BACKEND = 'path.to.my.custom.backend'

There was some discussion above around manipulating the user name. The backend class does provide a hook for that, but I'm not sure what degree of flexibility would be appropriate. I think at most we can allow the configuration of a regular expression that will match the desired portion of the username passed by the reverse proxy. This would allow us to, for example, strip the username portion from an email address, but anything more complicated than that would require a custom authentication backend.

[chaomodus]

All of this sounds very good, I was wondering how feasible it would be to also provide a remote auth header name for groups or some sort of group mapping mechanism (similar to REMOTE_AUTH_HEADER, but like REMOTE_AUTH_GROUPS), or would the standard suggestion would be to use a remote_auth_backend for that?

[bluikko]

@chaomodus There are ways to provide list of groups in an HTTP header but in my opinion that would never be as good as separating authorization from authentication: the HTTP header would be authentication and NetBox would do authorization via LDAP for example.

Anyhow it might be good to not try to widen the scope too much at this point - a simple external authentication via HTTP headers would be a good start.