Add support for serving on Unix domain socket #478

hsartoris-bard · 2023-04-07T18:14:10Z

Summary

These changes take advantage of jetty-unixsocket to allow serving on a socket, as in the following example:

(run-jetty handler {:http? false, :socket "/run/someapp.sock"})

## hostname is still required for making requests
curl --unix-socket /run/someapp.sock http://my-expected-hostname/url-path

Rationale

Unix sockets are simpler to implement access controls to than TCP sockets, as you can simply use file permissions. They are supported by common reverse proxies such as Apache and NGINX, and are thus reasonable for exposing a service through a proxy that may be handling authentication, without making that service available to all users on the host machine.

Possible enhancements:

Options for file ownership and access controls
A secure-by-default approach might see :http? default false when :socket is set, unless :host or :port are explicitly configured.
To avoid the additional dependency, socket support could be conditional on manually including the jetty-unixsocket library. It is not an overwhelmingly heavy dependency, though.

Note: it has proved nontrivial to get a test working, as clj-http does not support sockets, and the Clojure implementations that I've found do not seem to work as intended out of the box. I have verified that this implementation is operational manually, and I am willing to work through the process of getting a test working as long as you're interested in incorporating this functionality.

weavejester · 2023-04-09T14:19:35Z

I think this is reasonable in theory. Perhaps :unix-socket instead of :socket as an option, so it's not confused with other types of sockets. Also, have you tested to ensure that the inclusion of this library doesn't cause issues when using it under operating systems that don't have sockets (e.g. Windows)? I'd also suggest not automatically deleting the socket file on startup - that feels potentially dangerous. Instead, raise an error.

hsartoris-bard · 2023-04-10T17:24:48Z

I was under the impression that there was now unix socket support on Windows, but indeed it appears not to work.

Changes

check for windows
rename :socket to :unix-socket
make :http? default false when :unix-socket is set, except when :host or :port are set
throw error on extant file instead of deleting
tests for basic functionality and errant attempts to use extant file

Let me know if there's anything else you'd like to see!

hsartoris-bard · 2023-04-10T17:27:57Z

Whoops, wasn't my intention to check in the *warn-on-reflection* line - I can remove that, or resolve the singular reflection warning.

weavejester

Thanks for your hard work on this, and apologies for taking a while to get around to reviewing it.

weavejester · 2023-05-07T16:42:41Z

ring-jetty-adapter/src/ring/adapter/jetty.clj

           [org.eclipse.jetty.util BlockingArrayQueue]
           [org.eclipse.jetty.util.thread ThreadPool QueuedThreadPool]
           [org.eclipse.jetty.util.ssl SslContextFactory$Server KeyStoreScanner]
           [javax.servlet AsyncContext DispatcherType AsyncEvent AsyncListener]
           [javax.servlet.http HttpServletRequest HttpServletResponse]))
+(set! *warn-on-reflection* true)


Can you remove this line, please.

weavejester · 2023-05-07T16:44:33Z

ring-jetty-adapter/src/ring/adapter/jetty.clj

@@ -130,6 +133,25 @@
      (.setHost (options :host))
      (.setIdleTimeout (options :max-idle-time 200000)))))

+(defn- socket-connector
+  ^UnixSocketConnector [^Server server {:keys [unix-socket] :as options}]


Let's use a consistent format in this file, so:

(defn- ^UnixSocketConnector unix-socket-connector [^Server server {:keys [unix-socket] :as options}]

Also I think unix-socket-connector would help differentiate it from TCP sockets.

weavejester · 2023-05-07T16:47:15Z

ring-jetty-adapter/src/ring/adapter/jetty.clj

+  (when (->> (System/getProperty "os.name")
+             (re-find #"(?i)^windows"))
+    (throw (ex-info "Unix sockets not supported on windows"
+                    {:os (System/getProperty "os.name")
+                     :unix-socket unix-socket})))


Let's factor this out into its own function (and also capitalize Windows):

(defn- check-os-socket-compatibility [] (when (->> (System/getProperty "os.name") (re-find #"(?i)^windows")) (throw (ex-info "Unix sockets not supported on Windows" {:os (System/getProperty "os.name") :unix-socket unix-socket}))))

weavejester · 2023-05-07T17:11:43Z

ring-jetty-adapter/src/ring/adapter/jetty.clj

+    (.deleteOnExit socket)
+    (doto (UnixSocketConnector.
+            server
+            #^"[Lorg.eclipse.jetty.server.ConnectionFactory;" (into-array ConnectionFactory [http-factory]))


Can you split the line over 80 characters into two lines? Also can you fix the indentation so it's according to the style guide:

(doto (UnixSocketConnector. server ^"[Lorg.eclipse.jetty.server.ConnectionFactory;" (into-array ConnectionFactory [http-factory]))

weavejester · 2023-05-07T17:15:38Z

ring-jetty-adapter/src/ring/adapter/jetty.clj

+    (when (:http? options (or ;; default is enabled when no socket specified
+                              (not (:unix-socket options))
+                              ;; enabled with socket when host/port set
+                              (some options [:host :port])))


This feels too complex, and isn't what the documentation says. I think let's keep it simple and just leave it defaulting to true. Let's add a note to the :unix-socket option instead to tell people they may want to set :http? to false.

weavejester · 2023-05-07T17:16:18Z

ring-jetty-adapter/src/ring/adapter/jetty.clj

  :ssl?                   - allow connections over HTTPS
  :ssl-port               - the SSL port to listen on (defaults to 443, implies
                            :ssl? is true)
+  :unix-socket            - File to be used as a Unix domain socket; will be
+                            passed to [io/file]. Ensure there is no file at


I don't think [io/file] should be a link, as there's nothing to link it to.

weavejester · 2023-05-07T17:16:35Z

ring-jetty-adapter/test/ring/adapter/test/jetty.clj

+                    IOException
+                    File]


These two lines should be one.

weavejester · 2023-05-07T17:17:42Z

ring-jetty-adapter/test/ring/adapter/test/jetty.clj

+      (with-server hello-world {:unix-socket sock}
+        (is (= (-> (HttpClient/create)
+                   (.remoteAddress
+                     (reify Supplier


Indentation looks off here.

weavejester · 2023-05-07T17:17:59Z

ring-jetty-adapter/test/ring/adapter/test/jetty.clj

+               "Hello World")
+            "able to serve basic request")
+        (is (thrown-with-msg?
+              clojure.lang.ExceptionInfo #"File already exists"


Indentation looks off. Please see Clojure style guide.

weavejester · 2023-05-07T17:21:06Z

ring-jetty-adapter/src/ring/adapter/jetty.clj

+    (when (.exists socket)
+      (throw (ex-info "File already exists at socket path; should be deleted before starting server"
+                      {:unix-socket unix-socket})))


Let's factor this out as well:

(defn- check-socket-file-does-not-exist [socket] (when (.exists socket) (throw (ex-info "Could not create socket file: file already exists" {:socket-file (str socket)}))))

add support for running unix domain socket connector

d7e22d4

hsartoris-bard added 4 commits April 10, 2023 12:24

rename option to :unix-socket; add check for windows

7a0ff20

make HTTP default false when :unix-socket is specified

c0aec47

basic socket server test

9ede539

error on extant file; add doc line

71ec504

cyberb mentioned this pull request May 1, 2023

Make Jetty to support UNIX sockets metabase/metabase#30456

Open

weavejester requested changes May 7, 2023

View reviewed changes

cyberb mentioned this pull request May 23, 2023

local api unix socket support crowdsecurity/crowdsec#2213

Open

weavejester added the waiting on dev label Jul 3, 2023

weavejester force-pushed the master branch 2 times, most recently from 0d7f29a to 0103527 Compare September 15, 2023 18:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for serving on Unix domain socket #478

Add support for serving on Unix domain socket #478

hsartoris-bard commented Apr 7, 2023

weavejester commented Apr 9, 2023

hsartoris-bard commented Apr 10, 2023

hsartoris-bard commented Apr 10, 2023

weavejester left a comment

weavejester May 7, 2023

weavejester May 7, 2023

weavejester May 7, 2023

weavejester May 7, 2023

weavejester May 7, 2023

weavejester May 7, 2023

weavejester May 7, 2023

weavejester May 7, 2023

weavejester May 7, 2023

weavejester May 7, 2023

Add support for serving on Unix domain socket #478

Are you sure you want to change the base?

Add support for serving on Unix domain socket #478

Conversation

hsartoris-bard commented Apr 7, 2023

Summary

Rationale

Possible enhancements:

weavejester commented Apr 9, 2023

hsartoris-bard commented Apr 10, 2023

Changes

hsartoris-bard commented Apr 10, 2023

weavejester left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment