wsgi: add peer UNIX credential to a non-standard environment variable #37
Conversation
|
I've attempted to fix automatic tests, but the ones in circle ci seems to be an infrastructure problem. The rest of the tests were green. I'd be really grateful about the feature, especially the naming of the environment variable. I'd like to deploy something like this in production, and I'd like to use a name that will match the 'official' one. thanks. |
|
Any comments? |
|
Oh, sorry @bazsi.. I was busy and didn't check notifications during past few days. Nevermind about CircleCI, it's not configured yet. |
cheroot/wsgi.py
Outdated
| @@ -276,6 +278,11 @@ def get_environ(self): | |||
|
|
|||
| return env | |||
|
|
|||
| def _get_peer_uid(self): | |||
| creds = self.req.server.socket.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize('3i')) | |||
There was a problem hiding this comment.
There's no SO_PEERCRED in Python 2: https://utcc.utoronto.ca/~cks/space/blog/python/AbstractUnixSocketsAndPeercred
cheroot/wsgi.py
Outdated
| @@ -276,6 +278,11 @@ def get_environ(self): | |||
|
|
|||
| return env | |||
|
|
|||
| def _get_peer_uid(self): | |||
There was a problem hiding this comment.
It isn't obvious what this function purpose is, so it's a good practice to leave comments + docstring.
There was a problem hiding this comment.
also done in 831ec2d
I would be squashing that into the base commit as review is complete.
cheroot/wsgi.py
Outdated
| @@ -254,6 +255,7 @@ def get_environ(self): | |||
| # AF_UNIX. This isn't really allowed by WSGI, which doesn't | |||
| # address unix domain sockets. But it's better than nothing. | |||
| env['SERVER_PORT'] = '' | |||
| env['_REMOTE_UID'] = self._get_peer_uid() | |||
There was a problem hiding this comment.
Why do you need this as an environment variable?
cheroot/wsgi.py
Outdated
| @@ -276,6 +278,11 @@ def get_environ(self): | |||
|
|
|||
| return env | |||
|
|
|||
| def _get_peer_uid(self): | |||
| creds = self.req.server.socket.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize('3i')) | |||
There was a problem hiding this comment.
Can this be moved to the request or connection entity?
There was a problem hiding this comment.
I felt this was a lot more connected to REMOTE_ADDR and the likes, as I've felt it is a property of the connection rather than the request itself.
In our internal HTTP based service, we are using the environment to populate our internal data structures and by that time the connection itself is not available.
If you meant that _REMOTE_UID stays in the environment, but the implementation itself moves to HTTPConnection, that could make sense. Is that what you are thinking?
Thanks for the feedback.
There was a problem hiding this comment.
Could you please share your snippet where you cannot access HTTPConnection and still can access env vars? I'm really not sure whether all other users need this as it adds extra syscall overhead.
|
I have now pushed a couple of fixups to address your notes. If you are ok with them, I would rebase the branch and squash them to the same patch. |
cheroot/wsgi.py
Outdated
| @@ -26,8 +26,9 @@ def my_crazy_app(environ, start_response): | |||
| """ | |||
|
|
|||
| import sys | |||
|
|
|||
| import struct | |||
cheroot/wsgi.py
Outdated
| @@ -254,6 +255,9 @@ def get_environ(self): | |||
| # AF_UNIX. This isn't really allowed by WSGI, which doesn't | |||
| # address unix domain sockets. But it's better than nothing. | |||
| env['SERVER_PORT'] = '' | |||
| uid = self.req.conn.get_peer_uid() | |||
| if uid is not None: | |||
| env['_REMOTE_UID'] = str(uid) | |||
There was a problem hiding this comment.
Are there any other WSGI implementations providing similar env var?
There was a problem hiding this comment.
I've found a number of WSGI/HTTP servers/libs that support unix domain sockets:
- https://github.com/eventlet/eventlet/pull/320/files
- nginx with its proxy_pass directive
- support http over unix domain sockets httpie/cli#209
With that said, the only HTTP over UNIX domain socket implementation, that I've found to also support SO_PEERCRED based authentication is cups, which has its embedded http implementation (in C, brr) and does not use CGI like environment variables.
I've also looked up rfc3875, the original specification of CGI, which seems to be the origin for these environment variables: http://www.ietf.org/rfc/rfc3875. In the RFC, there's a standard REMOTE_USER environment variable, which seems to be what we aim for, but would require username lookup instead of just passing on the UID. username resolution could be expensive, especially as it would need to be done on every request.
That same RFC, also specifies that non-standard extensions should be prefixed with an X_.
With all this said, I think:
- unix domain based authentication makes sense, there is at least one implementation, and a use-case that matches mine.
- environment wise, we have three options:
- use X_REMOTE_UID and pass the uid only
- resolve the uid to username (with caching?), and reuse the standard REMOTE_USER attribute
- disregard the CGI origin, and use something different.
I've also found this document about WSGI: http://wsgi.readthedocs.io/en/latest/specifications/simple_authentication.html. This says, that the username needs to go to REMOTE_USER, thus
My opinion and the above spec point to option number 2) above.
If you agree, I can change my patches this way.
There was a problem hiding this comment.
- This shouldn't be a default behavior to avoid overhead for other users who may not want such logic to be triggered, so I would prefer that the calling logic go to CherryPy tool, which would be easy to turn on or switch off easily via config option.
- It would be enough to provide
@properties, which evaluate creads lookup lazily, once per request as I commented earlier. - It would be ok to have some function/method populating all of
REMOTE_USER,X_REMOTE_UID,X_REMOTE_GID,X_REMOTE_PIDor just some of them as configured.
There was a problem hiding this comment.
hmm.. cherrypy does not really have access to HTTPConnection objects, so I don't see how that would go back to the socket to do SO_PEERCRED. This is how it looks like (obviously I don't have intimate knowledge of these parts, I just did a quick browse of the related source code):
- cherrypy instantiates a cheroot.wsgi.Server instance (in ServerAdapter). wsgi.Server is derived from the "native" HTTP server, cheroot.server.HTTPServer
- as a connection arrives, a HTTPConnection is instantiated to encapsulate the connection specific socket
- we parse the incoming request, and a cheroot specific HTTPRequest is created, to encapsulate the low-level HTTP details. Here we still have a reference to the connection/server objects.
- then a cheroot.wsgi.Gateway is created, one that takes the low-level HTTPRequest, and populates a WSGI environment, and passes it to the the WSGI application via a
__call__method. - The environment is the only input to the WSGI app, even headers are translated into and from environment name-value pairs. (with ugly translations like Accept-Language becoming HTTP_ACCEPT_LANGUAGE)
There seems to be an intention to decouple the HTTP specific pieces and the app side, and the means to exchange information is the environment itself. It is thus not possible to go back from the app to HTTPConnection & HTTPRequest where the information is available.
The Native interface is one abstraction short (e.g. environment is not created and HTTPRequest is directly translated to cherrypy request objects), but that is not the default how cherrypy operates. That interface does not support authentication (e.g. request.login is not populated at all).
I could inject the socket into the environment (with a key like "wsgi.socket" to match other attributes), but my opinion is that this would be ugly and would derail the entire idea to decouple apps from HTTP specifics.
Short of adding this ugly hack to environment, we need to do the authentication on the cheroot side and pass the information via environment variables.
Still, REMOTE_USER looks to be the best solution, as that would work out of the box with cherrypy (e.g. request.login is filled automatically).
I could make the username resolution stuff optional by adding an additional constructor parameter to cheroot.wsgi.Server that could be driven by a cherrypy config variable (e.g. server.socket_file_auth True/False).
I can also amortize the performance impact by caching the username lookup, because in most cases, it would only be a very limited number of users that would use the interface.
I can see these courses of action at this point if we are to use REMOTE_USER:
- additional parameter w/o caching
- additional parameter w/ caching
- no additional parameter w/ caching
OR we can use X_REMOTE_UID, GID and PID as you suggested, in which case username resolution is not needed at all, so performance impact should be small. But in that case cherrypy.request.login will not be set automatically.
What do you think?
cheroot/wsgi.py
Outdated
| import six | ||
| import socket |
cheroot/server.py
Outdated
| """ | ||
|
|
||
| if self.socket.family != socket.AF_UNIX: | ||
| return None |
There was a problem hiding this comment.
I would prefer raising exception rather than returning "magic values"
cheroot/server.py
Outdated
| pid, uid, gid = struct.unpack('3i', creds) | ||
| return uid | ||
| except socket.error: | ||
| pass |
There was a problem hiding this comment.
Please, don't swallow exception. Log it and re-raise appropriate one.
There was a problem hiding this comment.
I actually wanted to ignore errors and in those cases not add this to the environment. It seems to be a Linux specific feature. I wouldn't want UNIX domain sockets not to work if this is not available.
perhaps a one-time warning would suffice?
There was a problem hiding this comment.
Alright, warning would be ok (mabe even NOTICE). But I still prefer raising exception instead of returning None.
cheroot/server.py
Outdated
| @@ -1155,6 +1156,28 @@ def close(self): | |||
| # Apache does, but not today. | |||
| pass | |||
|
|
|||
| def get_peer_uid(self): | |||
There was a problem hiding this comment.
I'd prefer a method returning the whole tuple of creds and 3 @property methods returning each of 'em.
cheroot/server.py
Outdated
| # NOTE: the value for SO_PEERCRED can be architecture specific, in | ||
| # which case the getsockopt() will hopefully fail. The arch | ||
| # specific value could be derived from platform.processor() | ||
| SO_PEERCRED = getattr(socket, 'SO_PEERCRED', 17) |
There was a problem hiding this comment.
I'd add this as a monkey-patch next to similar win-specific code @ line 80 to be consistent
|
@bazsi it would be very helpful to have tests and documentation for this feature as well + fix linter errors failing in Travis CI |
|
I'll do another iteration and push the updated patches. |
|
I still have not progressed with this. coming back once I have a timeslice. |
|
No worries. I'll be waiting for updates from your side and maybe even push a few commits to your branch if I have some free time for this. |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
@bazsi I've rebased your branch on top of current master |
|
@bazsi I've added bits of refactoring, but it's not considered as complete yet. I'd like to address:
|
|
This pull request introduces 1 alert - view on lgtm.com new alerts:
Comment posted by lgtm.com |
Codecov Report
@@ Coverage Diff @@
## master #37 +/- ##
==========================================
- Coverage 66.42% 65.97% -0.45%
==========================================
Files 17 17
Lines 2847 2942 +95
==========================================
+ Hits 1891 1941 +50
- Misses 956 1001 +45 |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
I still hope to find some time for this eventionally. |
|
Hi @bazsi! Sorry for being silent for so long. I think I came up with a proper solution now. It's bypassed through an adapter in CherryPy to Server instance in cheroot and then it pre-populates it into connections. WSGI env tries to get this data from connection and it succeeds when enabled and supported by the runtime. Option in CherryPy would be Do you mind taking another round of testing and finish this with me? |
|
@bazsi I think, PoC is ready. Additionally to #37 (review) I'm thinking that we could also looking at I'm almost ready to merge this, but it would be great to see some tests around it. |
Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
Reported by lgtm.com
This patch enables SO_PEERCRED based authentication into the NNX REST server, by adding the _REMOTE_UID to the environment, which then can be used to base authentication on. A similar patch was submitted to CherryPy upstream, here: cherrypy/cheroot#37 Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
|
@bazsi |
PR #37 by @bazsi: Implement PEERCRED lookup over UNIX-socket HTTP connection. * Discover connected process' PID/UID/GID * Respect server switches: ``peercreds_enabled`` and ``peercreds_resolve_enabled`` * ``get_peer_creds`` and ``resolve_peer_creds`` methods on connection * ``peer_pid``, ``peer_uid``, ``peer_gid``, ``peer_user`` and ``peer_group`` properties on connection * ``X_REMOTE_PID``, ``X_REMOTE_UID``, ``X_REMOTE_GID``, ``X_REMOTE_USER`` (``REMOTE_USER``) and ``X_REMOTE_GROUP`` WSGI environment variables when enabled and supported * Per-connection caching to reduce lookup cost
|
New minor version v6.2.0 is being released: https://travis-ci.org/cherrypy/cheroot/builds/364381847 (ETA: 30 min) |
This patch enables SO_PEERCRED based authentication into the NNX REST server, by adding the X_REMOTE_UID to the environment, which then can be used to base authentication on. A similar patch was submitted to CherryPy upstream, here: cherrypy/cheroot#37 The original version of this patch came from our balabit-os-7.0/python3-cherrypy8 package. The corresponding commit ids are: * c8e7fb9d998391e39778f39707e0e3f0819cec02, * cd022ff48d2abcb999d890eef4347b13d943eabd
This patch enables SO_PEERCRED based authentication into the NNX REST server, by adding the X_REMOTE_UID to the environment, which then can be used to base authentication on. A similar patch was submitted to CherryPy upstream, here: cherrypy/cheroot#37 The original version of this patch came from our balabit-os-7.0/python3-cherrypy8 package. The corresponding commit ids are: * c8e7fb9d998391e39778f39707e0e3f0819cec02, * cd022ff48d2abcb999d890eef4347b13d943eabd
Signed-off-by: Balazs Scheidler balazs.scheidler@balabit.com
It adds the UNIX UID of a unix domain socket that connects to our HTTP service over socket.
#)none
peer credentials are not available to apps even if they bind to a unix domain socket
I've added a non-standard _REMOTE_UID value to environ, that is available to apps. I've originally did this patch to an older version of cherrypy and this is a forward port.