Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow "use any port" feature #1360

Closed
matthijskooijman opened this issue Jul 9, 2017 · 8 comments
Closed

Allow "use any port" feature #1360

matthijskooijman opened this issue Jul 9, 2017 · 8 comments
Labels

Comments

@matthijskooijman
Copy link

@matthijskooijman matthijskooijman commented Jul 9, 2017

Puma supports systemd socket activation by getting a list of open sockets from systemd and matching that against the sockets it would normally open. This means that the listening parameters (procol, ip, port) must be configured twice (in the systemd .socket file and in the application config) and they must match to make things work.

It would be useful if the application could use a wildcard configuration, meaning "just use whatever systemd passes you", so the listen details only need to be given in systemd, not in the application. I'm not entirely sure how this should work (specify a port number of 0 or -1? A special IP or protocol name? Does this replace the explicitely given list of listen sockets, or is it just one element in the list? Perhaps a completely different interface?), but ideally it would work without changes to applications (which suggests a special IP, port or protocol name would be best?).

@nateberkopec
Copy link
Member

@nateberkopec nateberkopec commented Jul 9, 2017

Binding to the unspecified address won't work for your use case? puma -b tcp://0.0.0.0 config.ru

@matthijskooijman
Copy link
Author

@matthijskooijman matthijskooijman commented Jul 10, 2017

Not sure if you understood my meaning. IIUC that command will still bind to port 80 (or whatever is the default), right? And if I specify a different port in systemd, this bit of code will not match the systemd port to the requested port 80, and end up trying to open port 80 and then close the port systemd opened.

As for the usecase that promted this: I was running the rails server, which binds to port 3000 by default. I'm using systemd socket activation to let systemd open up port 80 (so rails/puma does not need root to open a port < 1024). However, to let puma actually use this port passed in from systemd, I have to pass --port 80 to rails, so rails requests the same port systemd offers. I was hoping to do something like --port use-any, or something like that, so it will just use the systemd socket, regardless of which port/listen address it has.

@nateberkopec
Copy link
Member

@nateberkopec nateberkopec commented Jul 10, 2017

Thanks. I am not a systemd guy, so the real answer is "IDK"! I was just trying to see if any of the behavior we already support re: binding could fill this use case.

FWIW, we tend to support a few more things/weirder scenarios when you use puma directly, rather than rails server.

@dekellum
Copy link
Contributor

@dekellum dekellum commented Jul 10, 2017

Consider that puma offers HTTPS support and that is common when using that support to configure 2 ports: one HTTPS and one HTTP port. In such a case:

  1. There needs to be some means of distinguishing which socket is which on the puma side. (The current implementation does handle that.)
  2. Connection parameters like the ssl protocol, key and cert are used only on the puma side.

When I last implemented improvements for socket activation, I did consider adding support in puma to read $LISTEN_FDNAMES as set by systemd and use that value as the complete configuration of the puma connection. This might satisfy your interest in consolidating the config in systemd socket file(s). However, there are some complications:

  • FileDescriptorName oddly applies to all sockets in a single socket unit, so it appears to require one socket unit file per socket for this to work.
  • The ':' character is reserved so we'd need some replacement character like ';' for 'ssl;//', etc. and hope that ';' isn't used elsewhere.

My impression at the time was that this wasn't worth the additional complexity.

@dekellum
Copy link
Contributor

@dekellum dekellum commented Dec 31, 2017

@nateberkopec suggest closing this since there is no further comment and nothing actionable. House cleaning. Happy New Year!

@matthijskooijman
Copy link
Author

@matthijskooijman matthijskooijman commented Dec 31, 2017

Agreed!

@ekohl
Copy link
Contributor

@ekohl ekohl commented Sep 9, 2020

I also ran into this and I attempted to get rid of the duplication in my setup. For reference, my setup is httpd.service -> my_app.socket -> my_app.service. There is just a single bind and Apache, as a reverse proxy, takes care of HTTP/HTTPS.

First I tried to add the following to my config/puma/production.rb:

if ENV['LISTEN_FDS'] && ENV['LISTEN_PID'].to_i == $$
  require 'socket'

  clear_binds!

  ENV['LISTEN_FDS'].to_i.times do |index|
    fd = index + 3 # 3 is the magic number you add to follow the SA protocol
    sock = TCPServer.for_fd(fd)
    url = begin # Try to parse as a path
            "unix://#{Socket.unpack_sockaddr_un(sock.getsockname)}"
          rescue ArgumentError # Try to parse as a port/ip
            port, addr = Socket.unpack_sockaddr_in(sock.getsockname)
            addr = "[#{addr}]" if addr =~ /\:/
            "tcp://#{addr}:#{port}"
          end

    puts "Binding to #{url}"
    bind url
  end
end

However, it turns out the launcher actives the sockets and clears it from ENV.

@binder = Binder.new(@events, conf)
@binder.create_inherited_fds(ENV).each { |k| ENV.delete k }
@binder.create_activated_fds(ENV).each { |k| ENV.delete k }
@environment = conf.environment
# Advertise the Configuration
Puma.cli_config = @config if defined?(Puma.cli_config)
@config.load

That means you can't figure out where systemd wanted to bind at that stage. Since you don't have access to the Binder instance, that's also not a solution.

Would it make sense to add an option bind_to_activated_sockets in the DSL to achieve this? It would not clear the binds, allowing the user to choose this explicitly. Something like the following in the Launcher (right after the above pasted code) could achieve this:

if @config[:bind_to_activated_sockets]
  @binder.activated_sockets.keys.each do |proto, addr, port|
    url = port ? "#{proto}://#{addr}:#{port}" : "#{proto}://#{addr}"
    @config.bind(url) unless @config[:binds].include?(url)
  end
end

Of course this also needs some wiring up elsewhere, but before I start that I'd like to know if such a PR would be accepted.

@nateberkopec
Copy link
Member

@nateberkopec nateberkopec commented Sep 9, 2020

@ekohl sure, I would accept that I think.

@ekohl ekohl mentioned this issue Sep 10, 2020
8 of 8 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants