Replace Unicorn with Puma

Changelog.md

@@ -16,6 +16,20 @@ After [a discussion with our community on Discourse](https://discourse.diasporaf
 
 Although the chat was never enabled per default and was marked as experimental, some production pods did set up the integration and offered an XMPP service to their users. After this release, diaspora\* will no longer contain a chat applet, so users will no longer be able to use the webchat inside diaspora\*. The existing module that is used to enable users to authenticate to Prosody using their diaspora\* credentials will continue to work, but contact list synchronization might not work without further changes to the Prosody module, which is developed independently from this project.
 
+## Changes around the appserver and related configuration
+
+With this release, we switched from `unicorn` to `puma` to run our applications. For podmins running the default setup, this should significantly reduce memory usage, with similar or even better frontend performance! However, as great as this change is, some configuration changes are required.
+
+- The `single_process_mode` and `embed_sidekiq_worker` configurations have been removed. This mode was never truly a "single-process" mode, as it just spawned the Background Workers inside the runserver. If you're using `script/server` to start your pod, this change does not impact you, but if you're running diaspora\* using other means, and you relied on this "single"-process mode, please ensure that Sidekiq workers get started.
+- The format of the `listen` configuration has changed. If you have not set that field in your configuration, you can skip this. Otherwise, make sure to adjust your configuration accordingly:
+  - Listening to Unix sockets with a relative path has changed from `unix:tmp/diaspora.sock` into `unix://tmp/diaspora.sock`.
+  - Listening to Unix sockets with an absolute path has changed from `unix:/run/diaspora/diaspora.sock` to `unix:///run/diaspora/diaspora.sock`.
+  - Listening to a local port has changed from `127.0.0.1:3000` to `tcp://127.0.0.1:3000`.
+- The `PORT` environment variable and the `-p` parameter to `script/server` have been removed. If you used that to run diaspora\* on a non-standard port, please use the `listen` configuration.
+- The `unicorn_worker` configuration has been dropped. With Puma, there should not be a need to increase the number of workers above a single worker in any pod of any size.
+- The `unicorn_timeout` configuration has been renamed to `web_timeout`.
+- **If you don't run your pod with `script/server`**, you have to update your setup. If you previously called `bin/bundle exec unicorn -c config/unicorn.rb` to run diaspora\*, you now have to run `bin/puma -C config/puma.rb`! Please update your systemd-Units or similar accordingly.
+
 ## Yarn for frontend dependencies
 
 We use yarn to install the frontend dependencies now, so you need to have that installed. See here for how to install it: https://yarnpkg.com/en/docs/install

Gemfile

@@ -10,8 +10,7 @@ gem "responders", "3.0.1"
 
 # Appserver
 
-gem "unicorn", "6.1.0", require: false
-gem "unicorn-worker-killer", "0.4.5"
+gem "puma", "5.6.5", require: false
 
 # Federation
 

Gemfile.lock

@@ -310,8 +310,6 @@ GEM
     fuubar (2.5.1)
       rspec-core (~> 3.0)
       ruby-progressbar (~> 1.4)
-    get_process_mem (0.2.7)
-      ffi (~> 1.0)
     gitlab (4.18.0)
       httparty (~> 0.18)
       terminal-table (>= 1.5.1)
@@ -398,7 +396,6 @@ GEM
     jsonpath (1.1.2)
       multi_json
     jwt (2.4.1)
-    kgio (2.11.4)
     kostya-sigar (2.0.10)
     leaflet-rails (1.7.0)
       rails (>= 4.2.0)
@@ -520,6 +517,8 @@ GEM
       byebug (~> 11.0)
       pry (~> 0.10)
     public_suffix (4.0.7)
+    puma (5.6.5)
+      nio4r (~> 2.0)
     raabro (1.4.0)
     racc (1.6.0)
     rack (2.2.4)
@@ -581,7 +580,6 @@ GEM
       rake (>= 12.2)
       thor (~> 1.0)
     rainbow (3.1.1)
-    raindrops (0.20.0)
     rake (12.3.3)
     rash_alt (0.4.12)
       hashie (>= 3.4)
@@ -735,12 +733,6 @@ GEM
       unf_ext
     unf_ext (0.0.8.2)
     unicode-display_width (1.8.0)
-    unicorn (6.1.0)
-      kgio (~> 2.6)
-      raindrops (~> 0.7)
-    unicorn-worker-killer (0.4.5)
-      get_process_mem (~> 0)
-      unicorn (>= 4, < 7)
     uuid (2.3.9)
       macaddr (~> 1.0)
     valid (1.2.0)
@@ -848,6 +840,7 @@ DEPENDENCIES
   pronto-scss (= 0.11.0)
   pry
   pry-byebug
+  puma (= 5.6.5)
   rack-cors (= 1.1.1)
   rack-google-analytics (= 1.2.0)
   rack-piwik (= 0.3.0)
@@ -885,8 +878,6 @@ DEPENDENCIES
   twitter (= 7.0.0)
   twitter-text (= 3.1.0)
   typhoeus (= 1.4.0)
-  unicorn (= 6.1.0)
-  unicorn-worker-killer (= 0.4.5)
   uuid (= 2.3.9)
   versionist (= 2.0.1)
   webmock (= 3.14.0)

Procfile

@@ -1,2 +1,2 @@
-web: bin/bundle exec unicorn -c config/unicorn.rb -p $PORT
+web: bin/puma -C config/puma.rb
 sidekiq: bin/bundle exec sidekiq

app/workers/archive_base.rb

@@ -27,12 +27,17 @@ def perform_archive_job(_args)
     end
 
     def currently_running_archive_jobs
-      return 0 if AppConfig.environment.single_process_mode?
-
       Sidekiq::Workers.new.count do |process_id, thread_id, work|
         !(Process.pid.to_s == process_id.split(":")[1] && Thread.current.object_id.to_s(36) == thread_id) &&
           ArchiveBase.subclasses.map(&:to_s).include?(work["payload"]["class"])
       end
+    rescue Redis::CannotConnectError
+      # If code gets to this point and there is no Redis conenction, we're
+      # running in a Test environment and have not mocked Sidekiq::Workers, so
+      # we're not testing the concurrency-limiting behavior.
+      # There is no way a production pod will run into this code, as diaspora*
+      # refuses to start without redis.
+      0
     end
   end
 end

bin/bundle

@@ -1,3 +1,114 @@
 #!/usr/bin/env ruby
-ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../../Gemfile', __FILE__)
-load Gem.bin_path('bundler', 'bundle')
+# frozen_string_literal: true
+
+#
+# This file was generated by Bundler.
+#
+# The application 'bundle' is installed as part of a gem, and
+# this file is here to facilitate running it.
+#
+
+require "rubygems"
+
+m = Module.new do
+  module_function
+
+  def invoked_as_script?
+    File.expand_path($0) == File.expand_path(__FILE__)
+  end
+
+  def env_var_version
+    ENV["BUNDLER_VERSION"]
+  end
+
+  def cli_arg_version
+    return unless invoked_as_script? # don't want to hijack other binstubs
+    return unless "update".start_with?(ARGV.first || " ") # must be running `bundle update`
+    bundler_version = nil
+    update_index = nil
+    ARGV.each_with_index do |a, i|
+      if update_index && update_index.succ == i && a =~ Gem::Version::ANCHORED_VERSION_PATTERN
+        bundler_version = a
+      end
+      next unless a =~ /\A--bundler(?:[= ](#{Gem::Version::VERSION_PATTERN}))?\z/
+      bundler_version = $1
+      update_index = i
+    end
+    bundler_version
+  end
+
+  def gemfile
+    gemfile = ENV["BUNDLE_GEMFILE"]
+    return gemfile if gemfile && !gemfile.empty?
+
+    File.expand_path("../Gemfile", __dir__)
+  end
+
+  def lockfile
+    lockfile =
+      case File.basename(gemfile)
+      when "gems.rb" then gemfile.sub(/\.rb$/, gemfile)
+      else "#{gemfile}.lock"
+      end
+    File.expand_path(lockfile)
+  end
+
+  def lockfile_version
+    return unless File.file?(lockfile)
+    lockfile_contents = File.read(lockfile)
+    return unless lockfile_contents =~ /\n\nBUNDLED WITH\n\s{2,}(#{Gem::Version::VERSION_PATTERN})\n/
+    Regexp.last_match(1)
+  end
+
+  def bundler_requirement
+    @bundler_requirement ||=
+      env_var_version || cli_arg_version ||
+        bundler_requirement_for(lockfile_version)
+  end
+
+  def bundler_requirement_for(version)
+    return "#{Gem::Requirement.default}.a" unless version
+
+    bundler_gem_version = Gem::Version.new(version)
+
+    requirement = bundler_gem_version.approximate_recommendation
+
+    return requirement unless Gem.rubygems_version < Gem::Version.new("2.7.0")
+
+    requirement += ".a" if bundler_gem_version.prerelease?
+
+    requirement
+  end
+
+  def load_bundler!
+    ENV["BUNDLE_GEMFILE"] ||= gemfile
+
+    activate_bundler
+  end
+
+  def activate_bundler
+    gem_error = activation_error_handling do
+      gem "bundler", bundler_requirement
+    end
+    return if gem_error.nil?
+    require_error = activation_error_handling do
+      require "bundler/version"
+    end
+    return if require_error.nil? && Gem::Requirement.new(bundler_requirement).satisfied_by?(Gem::Version.new(Bundler::VERSION))
+    warn "Activating bundler (#{bundler_requirement}) failed:\n#{gem_error.message}\n\nTo install the version of bundler this project requires, run `gem install bundler -v '#{bundler_requirement}'`"
+    exit 42
+  end
+
+  def activation_error_handling
+    yield
+    nil
+  rescue StandardError, LoadError => e
+    e
+  end
+end
+
+m.load_bundler!
+
+if m.invoked_as_script?
+  load Gem.bin_path("bundler", "bundle")
+end

bin/cucumber

@@ -1,3 +1,27 @@
 #!/usr/bin/env ruby
-require 'bundler/setup'
-load Gem.bin_path('cucumber', 'cucumber')
+# frozen_string_literal: true
+
+#
+# This file was generated by Bundler.
+#
+# The application 'cucumber' is installed as part of a gem, and
+# this file is here to facilitate running it.
+#
+
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__)
+
+bundle_binstub = File.expand_path("bundle", __dir__)
+
+if File.file?(bundle_binstub)
+  if File.read(bundle_binstub, 300) =~ /This file was generated by Bundler/
+    load(bundle_binstub)
+  else
+    abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
+Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
+  end
+end
+
+require "rubygems"
+require "bundler/setup"
+
+load Gem.bin_path("cucumber", "cucumber")

bin/eye

@@ -1,16 +1,27 @@
 #!/usr/bin/env ruby
+# frozen_string_literal: true
+
 #
 # This file was generated by Bundler.
 #
 # The application 'eye' is installed as part of a gem, and
 # this file is here to facilitate running it.
 #
 
-require 'pathname'
-ENV['BUNDLE_GEMFILE'] ||= File.expand_path("../../Gemfile",
-  Pathname.new(__FILE__).realpath)
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__)
+
+bundle_binstub = File.expand_path("bundle", __dir__)
+
+if File.file?(bundle_binstub)
+  if File.read(bundle_binstub, 300) =~ /This file was generated by Bundler/
+    load(bundle_binstub)
+  else
+    abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
+Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
+  end
+end
 
-require 'rubygems'
-require 'bundler/setup'
+require "rubygems"
+require "bundler/setup"
 
-load Gem.bin_path('eye', 'eye')
+load Gem.bin_path("eye", "eye")

bin/haml-lint

@@ -1,16 +1,27 @@
 #!/usr/bin/env ruby
+# frozen_string_literal: true
+
 #
 # This file was generated by Bundler.
 #
 # The application 'haml-lint' is installed as part of a gem, and
 # this file is here to facilitate running it.
 #
 
-require 'pathname'
-ENV['BUNDLE_GEMFILE'] ||= File.expand_path("../../Gemfile",
-  Pathname.new(__FILE__).realpath)
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__)
+
+bundle_binstub = File.expand_path("bundle", __dir__)
+
+if File.file?(bundle_binstub)
+  if File.read(bundle_binstub, 300) =~ /This file was generated by Bundler/
+    load(bundle_binstub)
+  else
+    abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
+Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
+  end
+end
 
-require 'rubygems'
-require 'bundler/setup'
+require "rubygems"
+require "bundler/setup"
 
-load Gem.bin_path('haml_lint', 'haml-lint')
+load Gem.bin_path("haml_lint", "haml-lint")

bin/pronto

@@ -1,16 +1,27 @@
 #!/usr/bin/env ruby
+# frozen_string_literal: true
+
 #
 # This file was generated by Bundler.
 #
 # The application 'pronto' is installed as part of a gem, and
 # this file is here to facilitate running it.
 #
 
-require 'pathname'
-ENV['BUNDLE_GEMFILE'] ||= File.expand_path("../../Gemfile",
-  Pathname.new(__FILE__).realpath)
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__)
+
+bundle_binstub = File.expand_path("bundle", __dir__)
+
+if File.file?(bundle_binstub)
+  if File.read(bundle_binstub, 300) =~ /This file was generated by Bundler/
+    load(bundle_binstub)
+  else
+    abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
+Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
+  end
+end
 
-require 'rubygems'
-require 'bundler/setup'
+require "rubygems"
+require "bundler/setup"
 
-load Gem.bin_path('pronto', 'pronto')
+load Gem.bin_path("pronto", "pronto")