Replace Chef with ECS Deployment (#8314)

* ready files ecs transition

* Non Infrastructure Related variables

* Add more terraform files

* Add ELB and Subnet Files

* add s3 and redis

* Add everything needed to create a staging server

* Add shared module as variable

* Ready to deploy shared resources

* added sidekiq Dockerfile

* switch vault to using an ECS Task role instead of an EC2 instance  role

* fixed typo in S3_avatars_bucket

* fixed typo in S3_AVATARS_ASSET_HOST

* switch to 172. local address for vault while don't have a NAT Gateway set up

* change the azs used to the ones used from our load balancer and added pma healthcheck

* Add tf files for production

* add buildspec for pipeline

* move ECR repo to shared resources for production

* Fix copy paste errors

* Production Deployable

* Add new load balancer

* Move Target Groups to shared so we can define the load balancer rules in terraform

* Add another capacity provider

* rename resources correctly

* delete unused files

* Make sure we load the RDS CA into pma before starting the container

* review changes

* renamed all occurences to rails_startup_time

* make the us-west-2b a private subnet

* enable corepack

* increase the size of the autoscaling group again

* update dockerfiles

* fix dockerfiles

* change networking with a new public subnet

* add yarnrc when copying

* apache config for PMA Docker

* Update Docker files to streamlined Rails 7.3.1 build

* Configure the unicorn server to start from the docker container

* Be a bit more conservative about installing NodeJS

* Move assset building back to docker-entrypoint

* Move assset building back to docker-entrypoint

* Fix ECS Permissions

* add rake to assets:precompile

* change env to prod

* make sidekiq dockerfile also run mailcatcher

* build regulations into the dockerfile

* add git commit BUILD_TAG

* add building docs to script

* make mailcatcher listen on all ips

* move old regulations to the correct folder

* remove building regulations and documents in the dockerfile

* remove font dependencies

* make sure sidekiq can zip up exports

* remove unnecessary / in dockerfile

* change credentials to use Aws::ECSCredentials

* Host Exports on s3 (#8795)

* upload dumps to s3

* serve dumps from s3

* make routes for permalinks

* correct filenames

* make URLs less ugly

* rename method to be more clear

* only cache size and URL for one day

* rename to current_results_export

* use the DumpPublicResultsDatabase start_date as the timestamp for the results exports

* update api_controller

* install zip in docker container

* correct paths for uploading

* make file public-read

* run rubocop

* run rubocop

* change filenames again

* use run_start instead of run_end in test

* delete zip file after upload

* review changes

* rename method

* add .zip to the url

* replace instance profile credentials with ecs credentials

* allow the docker environment to have access to .yarn

* only load s3 bucket when needed

* reuse s3 bucket if possible

* only load regulations in production

* only load regulations on startup in production

* stub regulations loading in omni search test

* Sync infrastructure with current state

* add listener for mailcatcher

* make mailcatcher https

* use specific_install to install a almost working version of mailcatcher

* add task definition to reset db

* install mailcatcher from local fork

* review changes

* don't maintain our own apache config

* correctly rename container_name

* give sidekiq enough memory

* switch from RACK_ENV to RAILS_ENV in unicorn

---------

Co-authored-by: Gregor Billing <gbilling@worldcubeassociation.org>

WcaOnRails/.dockerignore

@@ -0,0 +1,21 @@
+# See https://docs.docker.com/engine/reference/builder/#dockerignore-file for more about ignoring files.
+
+# Ignore bundler config.
+/.bundle
+
+# Ignore all default key files
+config/master.key
+config/credentials/*.key
+
+# Ignore all logfiles and tempfiles.
+/log/*
+/tmp/*
+!/log/.keep
+!/tmp/.keep
+
+# Ignore pidfiles, but keep the directory.
+/tmp/pids/*
+!/tmp/pids/
+!/tmp/pids/.keep
+
+node_modules

WcaOnRails/.gitignore

@@ -39,16 +39,16 @@ pids/*.pid
 # Coveralls stuff
 /coverage/
 
-/public/packs
-/public/packs-test
-/node_modules
 /yarn-error.log
 yarn-debug.log*
 .yarn-integrity
 
 # Healthcheck flag for our Rails docker container
 .db-inited
 
+# GCE secrets file
+**/application_default_credentials.json
+
 # Per https://yarnpkg.com/getting-started/qa#which-files-should-be-gitignored
 .yarn/*
 !.yarn/cache

WcaOnRails/Dockerfile

@@ -1,28 +1,65 @@
 FROM ruby:3.3.0
-EXPOSE 3000
+ARG BUILD_TAG=local
+WORKDIR /rails
 
 ENV DEBIAN_FRONTEND noninteractive
-WORKDIR /app
-ARG NODE_MAJOR=20
 
-# Add PPA needed to install nodejs.
-# From: https://github.com/nodesource/distributions#debian-and-ubuntu-based-distributions
-RUN apt-get update && apt-get install -y ca-certificates curl gnupg
-RUN mkdir -p /etc/apt/keyrings && curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
+# Set production environment
+ENV RAILS_LOG_TO_STDOUT="1" \
+    RAILS_SERVE_STATIC_FILES="true" \
+    RAILS_ENV="production" \
+    BUNDLE_WITHOUT="development:test" \
+    BUNDLE_DEPLOYMENT="1" \
+    BUILD_TAG=$BUILD_TAG
 
-RUN echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_${NODE_MAJOR}.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list
+# Add dependencies necessary to install nodejs.
+# From: https://github.com/nodesource/distributions#debian-and-ubuntu-based-distributions
+RUN apt-get update -qq && \
+    apt-get install --no-install-recommends -y \
+      ca-certificates \
+      curl \
+      gnupg
 
-RUN apt-get update && apt-get install -y \
-  git \
-  build-essential \
-  zip \
-  nodejs \
-  mariadb-client \
-  libssl-dev \
-  libyaml-dev \
-  tzdata
+ARG NODE_MAJOR=20
+RUN curl -fsSL https://deb.nodesource.com/setup_${NODE_MAJOR}.x | bash && \
+    apt-get install -y nodejs
 
 # Enable 'corepack' feature that lets NPM download the package manager on-the-fly as required.
 RUN corepack enable
 
+RUN apt-get update -qq && \
+    apt-get install --no-install-recommends -y \
+      build-essential \
+      git \
+      pkg-config \
+      mariadb-client \
+      libvips \
+      libssl-dev \
+      libyaml-dev \
+      tzdata && \
+    rm -rf /var/lib/apt/lists /var/cache/apt/archives
+
+# Install application gems
+COPY Gemfile Gemfile.lock ./
 RUN gem update --system && gem install bundler
+RUN bundle install && \
+    rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git
+
+# Install node dependencies
+COPY package.json yarn.lock .yarnrc.yml ./
+RUN yarn install --immutable
+
+# Copy built artifacts: gems, application
+COPY . .
+
+# Run and own only the runtime files as a non-root user for security
+RUN useradd rails --create-home --shell /bin/bash && \
+    chown -R rails:rails db log tmp public app pids .yarn
+USER rails:rails
+
+# Entrypoint prepares database and starts app on 0.0.0.0:3000 by default,
+# but can also take a rails command, like "console" or "runner" to start instead.
+ENTRYPOINT ["/rails/bin/docker-entrypoint"]
+
+EXPOSE 3000
+CMD ["./bin/bundle", "exec", "unicorn", "-c", "/rails/config/unicorn.rb"]

WcaOnRails/Dockerfile.dev

@@ -0,0 +1,33 @@
+FROM ruby:3.3.0
+
+WORKDIR /rails
+
+ENV DEBIAN_FRONTEND noninteractive
+
+# Add PPA needed to install nodejs.
+# From: https://github.com/nodesource/distributions#debian-and-ubuntu-based-distributions
+RUN apt-get update -qq && \
+    apt-get install --no-install-recommends -y \
+      ca-certificates \
+      curl \
+      gnupg
+
+ARG NODE_MAJOR=20
+RUN curl -fsSL https://deb.nodesource.com/setup_${NODE_MAJOR}.x | bash && \
+    apt-get install nodejs
+
+# Enable 'corepack' feature that lets NPM download the package manager on-the-fly as required.
+RUN corepack enable
+
+RUN apt-get update -qq && \
+    apt-get install --no-install-recommends -y \
+      git \
+      build-essential \
+      mariadb-client \
+      libssl-dev \
+      libyaml-dev \
+      tzdata
+
+RUN gem update --system && gem install bundler
+
+EXPOSE 3000

WcaOnRails/Dockerfile.sidekiq

@@ -0,0 +1,64 @@
+FROM ruby:3.3.0
+
+WORKDIR /rails
+
+ENV DEBIAN_FRONTEND noninteractive
+
+# Set production environment
+ENV RAILS_LOG_TO_STDOUT="1" \
+    RAILS_SERVE_STATIC_FILES="true" \
+    RAILS_ENV="production" \
+    BUNDLE_WITHOUT="development:test" \
+    BUNDLE_DEPLOYMENT="1" \
+    BUNDLE_PATH="/usr/local/bundle"
+
+# Add dependencies necessary to install nodejs.
+# From: https://github.com/nodesource/distributions#debian-and-ubuntu-based-distributions
+RUN apt-get update -qq && \
+    apt-get install --no-install-recommends -y \
+      ca-certificates \
+      curl \
+      gnupg
+
+ARG NODE_MAJOR=20
+RUN curl -fsSL https://deb.nodesource.com/setup_${NODE_MAJOR}.x | bash && \
+    apt-get install -y nodejs
+
+# Enable 'corepack' feature that lets NPM download the package manager on-the-fly as required.
+RUN corepack enable
+
+RUN apt-get update -qq && \
+    apt-get install --no-install-recommends -y \
+      build-essential \
+      git \
+      pkg-config \
+      zip \
+      python-is-python3 \
+      mariadb-client \
+      libvips \
+      libssl-dev && \
+    rm -rf /var/lib/apt/lists /var/cache/apt/archives
+
+# Install application gems
+COPY Gemfile Gemfile.lock ./
+RUN gem update --system && gem install bundler
+# Workaround for mailcatcher not supporting Ruby 3.3 https://github.com/sj26/mailcatcher/issues/553
+#RUN gem install mailcatcher
+RUN gem install specific_install && gem specific_install -l https://github.com/thewca/mailcatcher.git -b feature/ruby3.3
+RUN bundle install && \
+    rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git
+
+# Install node dependencies
+COPY package.json yarn.lock .yarnrc.yml ./
+RUN yarn install --immutable
+
+COPY . .
+
+# Run and own only the runtime files as a non-root user for security
+RUN useradd rails --create-home --shell /bin/bash && \
+    chown -R rails:rails db log tmp public app .yarn
+USER rails:rails
+
+ENTRYPOINT ["/rails/bin/docker-entrypoint-sidekiq"]
+
+EXPOSE 3000

WcaOnRails/app/controllers/admin_controller.rb

@@ -329,7 +329,7 @@ def generate_db_token
       replica: EnvConfig.READ_REPLICA_HOST,
     }
 
-    role_credentials = Aws::InstanceProfileCredentials.new
+    role_credentials = Aws::ECSCredentials.new
     token_generator = Aws::RDS::AuthTokenGenerator.new credentials: role_credentials
 
     @db_tokens = @db_endpoints.transform_values do |url|

WcaOnRails/app/helpers/documents_helper.rb

@@ -8,7 +8,7 @@ module DocumentsHelper
   private def archive_metadata
     bucket = Aws::S3::Resource.new(
       region: EnvConfig.STORAGE_AWS_REGION,
-      credentials: Aws::InstanceProfileCredentials.new,
+      credentials: Aws::ECSCredentials.new,
     ).bucket(BUCKET_NAME)
 
     prefix = "documents/"

WcaOnRails/app/helpers/regulation_translations_helper.rb

@@ -7,18 +7,14 @@ module RegulationTranslationsHelper
   TRANSLATIONS_DATE_FILE = "#{TRANSLATIONS_FOLDER_PATH}/version-date".freeze
   BUCKET_NAME = 'wca-regulations'
 
-  @@s3 = Aws::S3::Resource.new(
-    region: EnvConfig.STORAGE_AWS_REGION,
-    credentials: Aws::InstanceProfileCredentials.new,
-  ).bucket(BUCKET_NAME)
-
   private def translations_metadata
     @@metadata_cache ||= []
-    build_hash = current_build_hash
+    s3 = regulations_bucket
+    build_hash = current_build_hash(s3)
 
     if @@metadata_cache.empty? || build_hash != @@cached_for_hash
 
-      metadata_objects = @@s3.objects(prefix: TRANSLATIONS_FOLDER_PATH)
+      metadata_objects = s3.objects(prefix: TRANSLATIONS_FOLDER_PATH)
       metadata_index = metadata_objects.filter { |object| File.extname(object.key) == ".json" }
                                        .index_by { |object| File.basename(File.dirname(object.key)) }
                                        .transform_values { |object| object.get.body.read.strip }
@@ -37,12 +33,19 @@ module RegulationTranslationsHelper
     hash
   end
 
-  private def current_build_hash
-    @@s3.object(TRANSLATIONS_HASH_FILE).get.body.read.strip
+  private def regulations_bucket
+    Aws::S3::Resource.new(
+      region: EnvConfig.STORAGE_AWS_REGION,
+      credentials: Aws::ECSCredentials.new,
+    ).bucket(BUCKET_NAME)
+  end
+
+  private def current_build_hash(s3 = regulations_bucket)
+    s3.object(TRANSLATIONS_HASH_FILE).get.body.read.strip
   end
 
-  private def current_base_version
-    @@s3.object(TRANSLATIONS_DATE_FILE).get.body.read.strip
+  private def current_base_version(s3 = regulations_bucket)
+    s3.object(TRANSLATIONS_DATE_FILE).get.body.read.strip
   end
 
   def current_reg_translations

WcaOnRails/app/models/regulation.rb

@@ -13,10 +13,12 @@ def self.reload_regulations(s3)
     @regulations_load_error = e
   end
 
-  reload_regulations(Aws::S3::Resource.new(
-                       region: EnvConfig.STORAGE_AWS_REGION,
-                       credentials: Aws::InstanceProfileCredentials.new,
-                     ))
+  if Rails.env.production?
+    reload_regulations(Aws::S3::Resource.new(
+                         region: EnvConfig.STORAGE_AWS_REGION,
+                         credentials: Aws::ECSCredentials.new,
+                       ))
+  end
 
   class << self
     attr_accessor :regulations_load_error

WcaOnRails/app/views/server_status/index.html.erb

@@ -37,11 +37,10 @@
   <% end %>
 
   <%= alert :info do
-    sha1 = `git rev-parse HEAD`
     {
         "PID": Process.pid,
         "Uptime": duration_to_s(Time.now - WcaOnRails::BOOTED_AT),
-        "Git commit": link_to(sha1.first(7), "https://github.com/thewca/worldcubeassociation.org/commit/#{sha1}"),
+        "Git commit": link_to(EnvConfig.BUILD_TAG, "https://github.com/thewca/worldcubeassociation.org/commit/#{EnvConfig.BUILD_TAG}"),
         "IP Addresses": Socket.ip_address_list.select(&:ipv4_private?).map(&:ip_address).join(","),
     }.map { |key, value| "#{key}: #{value}" }.join(" #{ui_icon("circle")} ").html_safe
   end %>

WcaOnRails/app_secrets.rb

@@ -66,10 +66,10 @@ def vault_file(secret_name, file_path)
     vault :NEW_RELIC_LICENSE_KEY
     vault :SMTP_USERNAME
     vault :SMTP_PASSWORD
+    vault_file :GOOGLE_APPLICATION_CREDENTIALS, "./tmp/application_default_credentials.json"
     vault :JWT_KEY
     vault :OIDC_SECRET_KEY
     vault :SLACK_WST_BOT_TOKEN
-    vault_file :GOOGLE_APPLICATION_CREDENTIALS, "../secrets/application_default_credentials.json"
   else
     mandatory :DATABASE_PASSWORD, :string
     mandatory :GOOGLE_MAPS_API_KEY, :string

WcaOnRails/bin/docker-entrypoint

@@ -0,0 +1,10 @@
+#!/bin/bash -e
+
+# If running the rails server then create or migrate existing database
+if [ "${1}" == "./bin/bundle" ] && [ "${3}" == "unicorn" ]; then
+  ./bin/rails db:prepare
+  ./bin/bundle exec i18n export
+  ./bin/bundle exec rake assets:precompile
+fi
+
+exec "${@}"

WcaOnRails/bin/docker-entrypoint-sidekiq

@@ -0,0 +1,4 @@
+#!/bin/bash -e
+mailcatcher --http-ip=0.0.0.0 --no-quit
+./bin/bundle exec rake assets:precompile
+./bin/bundle exec sidekiq

WcaOnRails/config/unicorn.rb

@@ -8,21 +8,19 @@
   'staging' => true,
   'production' => true,
 }
-rack_env = ENV.fetch('RACK_ENV', nil)
-if !allowed_environments[rack_env]
-  raise "Unrecognized RACK_ENV: #{rack_env}, must be one of #{allowed_environments.keys.join ', '}"
+rails_env = ENV.fetch('RAILS_ENV', nil)
+unless allowed_environments[rails_env]
+  raise "Unrecognized RACK_ENV: #{rails_env}, must be one of #{allowed_environments.keys.join ', '}"
 end
-if rack_env == "development"
-  listen 3000
+if rails_env == "development"
+  puts "Starting Unicorn in Development"
   worker_processes 1
 else
-  stderr_path "#{dir}/log/unicorn-#{rack_env}.log"
-  stdout_path "#{dir}/log/unicorn-#{rack_env}.log"
-
+  puts "Starting Unicorn in production mode"
   worker_processes((Etc.nprocessors * 2).ceil)
 end
 
-listen "/tmp/unicorn.wca.sock"
+listen 3000
 pid "#{dir}/pids/unicorn.pid"
 
 timeout 60

WcaOnRails/env_config.rb

@@ -21,7 +21,7 @@
     mandatory :VAULT_ADDR, :string
     mandatory :VAULT_APPLICATION, :string
     mandatory :VAULT_AWS_REGION, :string
-    mandatory :INSTANCE_ROLE, :string
+    mandatory :TASK_ROLE, :string
     mandatory :WCA_REGISTRATIONS_URL, :string
     mandatory :WCA_REGISTRATIONS_CDN_URL, :string
   else
@@ -67,4 +67,7 @@
   end
 
   optional :ROOT_URL, :string, default_root_url
+
+  # For server status
+  optional :BUILD_TAG, :string, "local"
 end