Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/review setup (#52) #54

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 21 additions & 13 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,16 +1,24 @@
# Gemfile
GEM_PG_VERSION=1.2.2
# Database
DB_HOST=postgis
DB_NAME=gtt
DB_USER=gtt
DB_PASS=gtt
DB_PORT=5432

# docker-compose.yml - postgres
POSTGRES_USER=gtt
POSTGRES_PASSWORD=gtt
POSTGRES_DB=gtt

# config/configuration.yml
SMTP_ENABLE_STARTTLS_AUTO=true # (or false)
SMTP_ADDRESS=smtp.gmail.com
SMTP_PORT=587
SMTP_DOMAIN=smtp.gmail.com
SMTP_AUTHENTICATION=plain # (or login)
# Email
SMTP_ADDRESS=smtp
SMTP_DOMAIN=smtp.example.com
SMTP_PORT=1025
SMTP_USER_NAME=YOUR_EMAIL_ADDRESS
SMTP_PASSWORD=YOUR_PASSWORD
SMTP_ENABLE_STARTTLS_AUTO=true

# Setup
REDMINE_LANG=en
REDMINE_EMAIL=gtt@example.com

# System
RAILS_ENV=production
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkastl (CC: @smellman @nobnisai)
From yesterday's internal discussion, we agreed not to support other development and test environments, so this RAILS_ENV line will not be necessary. (Because, current code doesn't support configuration.)

Note that plugin's test will not be available if we stick to production environment, so to develop plugins with tests, we need to setup another environment.

RAILS_LOG_TO_STDOUT=1
RAILS_MEMORY_LIMIT=1024
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

puma.rb file's RAILS_MEMORY_LIMIT default value is 2048, so it's better to change 1024 to 2048.
https://github.com/gtt-project/docker-gtt/pull/54/files#diff-24409379bdb75ad446bb4e2c18fd4cced1a263b99a3ff96fc3777d8fd8faeab9

PUMA_WORKERS=3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkastl (CC: @smellman, @nobnisai)
This is nice enhancement!

When I set PUMA_WORKERS=0, then it seems not to launch multiple workers and killers.

docker-gtt-redmine-1  | Puma starting in single mode...
docker-gtt-redmine-1  | * Puma version: 5.6.5 (ruby 3.1.2-p20) ("Birdie's Version")
docker-gtt-redmine-1  | *  Min threads: 0
docker-gtt-redmine-1  | *  Max threads: 5
docker-gtt-redmine-1  | *  Environment: production
docker-gtt-redmine-1  | *          PID: 1
docker-gtt-redmine-1  | * Listening on http://0.0.0.0:3000
docker-gtt-redmine-1  | Use Ctrl-C to stop

The following is the default (PUMA_WORKERS=3).

docker-gtt-redmine-1  | [1] Puma starting in cluster mode...
docker-gtt-redmine-1  | [1] * Puma version: 5.6.5 (ruby 3.1.2-p20) ("Birdie's Version")
docker-gtt-redmine-1  | [1] *  Min threads: 0
docker-gtt-redmine-1  | [1] *  Max threads: 5
docker-gtt-redmine-1  | [1] *  Environment: production
docker-gtt-redmine-1  | [1] *   Master PID: 1
docker-gtt-redmine-1  | [1] *      Workers: 3
docker-gtt-redmine-1  | [1] *     Restarts: (✔) hot (✔) phased
docker-gtt-redmine-1  | [1] * Listening on http://0.0.0.0:3000
docker-gtt-redmine-1  | [1] Use Ctrl-C to stop
docker-gtt-redmine-1  | [1] - Worker 0 (PID: 21) booted in 0.0s, phase: 0
docker-gtt-redmine-1  | [1] - Worker 1 (PID: 23) booted in 0.0s, phase: 0
docker-gtt-redmine-1  | [1] - Worker 2 (PID: 25) booted in 0.0s, phase: 0

6 changes: 6 additions & 0 deletions .markdownlint.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"MD029": false,
"MD033": false,
"MD038": false,
"MD041": false
}
116 changes: 39 additions & 77 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,22 @@
FROM ruby:2.7-slim-buster
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# GTT Builder
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
FROM node:16-bullseye-slim as gtt-builder

WORKDIR /app

COPY plugins/redmine_gtt/ ./redmine_gtt/

RUN apt update; \
apt install -y git; \
cd redmine_gtt; \
yarn; \
yarn webpack
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a slight thing, but in redmine_gtt plugin's README.md, we use npx webpack, but is it better to change yarn webpack in the README.md side ?
https://github.com/gtt-project/redmine_gtt#installation


# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# GTT Base
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
FROM ruby:3.1-slim-bullseye as base

# explicitly set uid/gid to guarantee that it won't change in the future
# the values 999:999 are identical to the current user/group id assigned
Expand All @@ -10,19 +28,11 @@ RUN set -eux; \
ca-certificates \
wget \
\
# bzr \
# git \
# mercurial \
openssh-client \
# subversion \
\
# we need "gsfonts" for generating PNGs of Gantt charts
# and "ghostscript" for creating PDF thumbnails (in 4.1+)
ghostscript \
gsfonts \
imagemagick \
# https://github.com/docker-library/ruby/issues/344
shared-mime-info \
# grab gosu for easy step-down from root
gosu \
# grab tini for signal processing and zombie killing
Expand All @@ -44,89 +54,42 @@ RUN set -eux; \
chown redmine:redmine "$HOME"; \
chmod 1777 "$HOME"

ARG REDMINE_VERSION="4.2.3"
ARG REDMICA_VERSION=""
# ENV REDMINE_DOWNLOAD_SHA256 ad4109c3425f1cfe4c8961f6ae6494c76e20d81ed946caa1e297d9eda13b41b4
# Defined in docker-compose.yml
ARG REDMINE_VERSION
ARG REDMINE_DOWNLOAD_SHA256

RUN set -eux; \
if [ -n "$REDMINE_VERSION" ]; then \
wget -O redmine.tar.gz "https://www.redmine.org/releases/redmine-${REDMINE_VERSION}.tar.gz"; \
# echo "$REDMINE_DOWNLOAD_SHA256 *redmine.tar.gz" | sha256sum -c -;
elif [ -n "$REDMICA_VERSION" ]; then \
wget -O redmine.tar.gz "https://github.com/redmica/redmica/archive/v${REDMICA_VERSION}.tar.gz"; \
fi; \
# bullseye hack
wget --no-check-certificate -O redmine.tar.gz "https://www.redmine.org/releases/redmine-${REDMINE_VERSION}.tar.gz"; \
echo "${REDMINE_DOWNLOAD_SHA256} *redmine.tar.gz" | sha256sum -c -; \
tar -xf redmine.tar.gz --strip-components=1; \
rm redmine.tar.gz files/delete.me log/delete.me; \
mkdir -p log public/plugin_assets sqlite tmp/pdf tmp/pids; \
chown -R redmine:redmine ./; \
# log to STDOUT (https://github.com/docker-library/redmine/issues/108)
echo 'config.logger = Logger.new(STDOUT)' > config/additional_environment.rb; \
# fix permissions for running as an arbitrary user
chmod -R ugo=rwX config db sqlite; \
find log tmp -type d -exec chmod 1777 '{}' +
mkdir -p log public/plugin_assets tmp/pdf tmp/pids; \
chown -R redmine:redmine ./ ; \
chmod -R ugo=rwX config ; \
find log tmp -type d -exec chmod 1777 '{}' +

COPY --from=gtt-builder --chown=redmine:redmine /app/ ./plugins/

# for Redmine patches
ARG PATCH_STRIP=1
ARG PATCH_DIRS=""
COPY patches/ ./patches/
COPY --chown=redmine:redmine config/ ./config/

# for GTT gem native extensions
ARG GEM_PG_VERSION="1.2.3"
COPY Gemfile.local ./
COPY plugins/ ./plugins/
COPY --chown=redmine:redmine Gemfile.local ./

RUN set -eux; \
\
savedAptMark="$(apt-mark showmanual)"; \
apt-get update; \
apt-get install -y --no-install-recommends \
# freetds-dev \
gcc \
# libmariadbclient-dev \
libpq-dev \
# libsqlite3-dev \
libgeos-dev \
make \
patch \
# in 4.1+, libmagickcore-dev and libmagickwand-dev are no longer necessary/used: https://www.redmine.org/issues/30492
libmagickcore-dev libmagickwand-dev \
# for GTT dependencies
g++ \
libgeos-dev \
curl \
; \
rm -rf /var/lib/apt/lists/*; \
\
if [ -n "$PATCH_DIRS" ]; then \
for dir in $(echo $PATCH_DIRS | sed "s/,/ /g"); do \
for file in ./patches/"$dir"/*; do \
patch -p"$PATCH_STRIP" < $file; \
done; \
done; \
rm -rf ./patches/*; \
fi; \
curl -sL https://deb.nodesource.com/setup_14.x | bash -; \
apt-get install -y --no-install-recommends nodejs; \
curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -; \
echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list; \
apt-get update; \
apt-get install -y --no-install-recommends yarn; \
for plugin in ./plugins/*; do \
if [ -f "$plugin/webpack.config.js" ]; then \
cd "$plugin" && yarn && npx webpack && rm -rf node_modules && cd ../..; \
fi; \
done; \
export GEM_PG_VERSION="$GEM_PG_VERSION"; \
gosu redmine bundle config --local without 'development test'; \
# fill up "database.yml" with bogus entries so the redmine Gemfile will pre-install all database adapter dependencies
# https://github.com/redmine/redmine/blob/e9f9767089a4e3efbd73c35fc55c5c7eb85dd7d3/Gemfile#L50-L79
echo '# the following entries only exist to force `bundle install` to pre-install all database adapter dependencies -- they can be safely removed/ignored' > ./config/database.yml; \
# for adapter in mysql2 postgresql sqlserver sqlite3; do \
for adapter in postgis; do \
echo "$adapter:" >> ./config/database.yml; \
echo " adapter: $adapter" >> ./config/database.yml; \
done; \
gosu redmine bundle install --jobs "$(nproc)"; \
rm ./config/database.yml; \
# fix permissions for running as an arbitrary user
chmod -R ugo=rwX Gemfile.lock "$GEM_HOME"; \
rm -rf ~redmine/.bundle; \
Expand All @@ -145,11 +108,10 @@ RUN set -eux; \
; \
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false

COPY config/ ./config/
VOLUME /usr/src/redmine/files

COPY docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]
COPY --chown=redmine:redmine docker-entrypoint.sh /

EXPOSE 3000
CMD ["rails", "server", "-b", "0.0.0.0"]

ENTRYPOINT [ "/docker-entrypoint.sh" ]

CMD [ "rails", "s", "-b", "0.0.0.0" ]
8 changes: 3 additions & 5 deletions Gemfile.local
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
source 'https://rubygems.org'

# each plugin's native extensions should be added to here
gem "rgeo"
gem "pg", (ENV['GEM_PG_VERSION'] ? "~> #{ENV['GEM_PG_VERSION']}" : "~> 1.2.3")
gem 'puma'
gem 'puma_worker_killer'
gem 'lograge'
Copy link
Member

@sanak sanak Sep 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkastl (CC: @smellman)
In my opinion, puma_worker_killer is for production environment, but shall we assume this docker-gtt setup for production environment (not for development environment) ?

Also, lograge is for changing rails log as JSON format which is convenience on AWS ECS log and CloudWatch, but shall we assume AWS as first deployment priority ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure ... but we actually run it --without test development and with production environment, so I wouldn't say it's a problem to keep it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkastl

but we actually run it --without test development and with production environment,

Okay, sure at this moment...
By the way, I found the way to support both (production and test development) env in another project, so it can be solved.

For debugging purpose, multiple workers (=3) and puma_worker_killer is quite annoying for me, so personally configurable this part with additional environment variables will be the best way.

Also, lograge is for changing rails log as JSON format which is convenience on AWS ECS log and CloudWatch, but shall we assume AWS as first deployment priority ?

How about this ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, mailcatcher is used in docker-compose.yml, so it's a kind of half production and half development env.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, don't know very much about these details. Could you just make a change that works good for development and push it to this branch?

Copy link
Member

@sanak sanak Sep 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, don't know very much about these details. Could you just make a change that works good for development and push it to this branch?

Do you mean only the following part ?

By the way, I found the way to support both (production and test development) env in another project, so it can be solved.

If so, it's okay for me.

But if it's also gem dependencies (puma, puma_worker_killer and lograge), we need to decide something this project's scope ,etc ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the question is, what the "target audience" of this Docker setup is. I think your perspective is the developer point of view. But there could be also the point of view of someone, who wants to use this to deploy and use the service. For example some SMASH user.

Of course it's best to support both use cases, for example with ENV settings. But we don't need to have this all ready with the first PR.

Maybe the non-developer use case should be priority, because a developer can also adjust the Docker setup. What do you think?

14 changes: 9 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,25 @@
# docker-gtt
# Docker GTT

Docker image for GTT Project

## Quick start

After cloning this repository run:

```
```sh
git submodule update --init
cp .env.example .env
docker-compose up --build
docker compose up --build
```

Open the application on http://localhost:3000/
Open the application on `http://localhost:3000/`

Default user is `admin/admin`.

## How to use GTT:
## How to use GTT

Find more information [how to get started with the GTT plugin](https://github.com/gtt-project/redmine_gtt#how-to-use).

## LICENSE

This program is free software. See [LICENSE](LICENSE) for more information.
11 changes: 3 additions & 8 deletions config/configuration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,10 @@ default:
email_delivery:
delivery_method: :smtp
smtp_settings:
enable_starttls_auto: <%= ENV['SMTP_ENABLE_STARTTLS_AUTO'] %>
enable_starttls_auto: <%= ENV['SMTP_ENABLE_STARTTLS_AUTO'] || true %>
address: <%= ENV['SMTP_ADDRESS'] %>
port: <%= ENV['SMTP_PORT'] %>
port: <%= ENV['SMTP_PORT'] || 587 %>
domain: <%= ENV['SMTP_DOMAIN'] %>
authentication: <%= ENV['SMTP_AUTHENTICATION'] %>
authentication: :plain
dkastl marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkastl (CC: @smellman)
Well, after I read the following Redmine Email Configuration again, I prefer to keep this SMTP_AUTHENTICATION configurable.
https://www.redmine.org/projects/redmine/wiki/EmailConfiguration

Is there some reason to drop this configuration ?

user_name: <%= ENV['SMTP_USER_NAME'] %>
password: <%= ENV['SMTP_PASSWORD'] %>

# ==== Sendmail command
#
# email_delivery:
# delivery_method: :sendmail
17 changes: 17 additions & 0 deletions config/database.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
default: &default
adapter: postgis
database: <%= ENV['DB_NAME'] %>
host: <%= ENV['DB_HOST'] %>
username: <%= ENV['DB_USER'] %>
password: <%= ENV['DB_PASS'] %>
port: <%= ENV['DB_PORT'] || 5432 %>
encoding: utf8

production:
<<: *default

test:
<<: *default

development:
<<: *default
34 changes: 34 additions & 0 deletions config/environments/production.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# frozen_string_literal: true

Rails.application.configure do
# Settings specified here will take precedence over those in config/application.rb

# Code is not reloaded between requests.
config.cache_classes = true

# Eager load code on boot. This eager loads most of Rails and
# your application in memory, allowing both threaded web servers
# and those relying on copy on write to perform better.
# Rake tasks automatically ignore this option for performance.
config.eager_load = true

# Full error reports are disabled and caching is turned on.
config.consider_all_requests_local = false
config.action_controller.perform_caching = true

# Disable delivery errors
config.action_mailer.raise_delivery_errors = false

# No email in production log
config.action_mailer.logger = nil

# Print deprecation notices to the Rails logger.
config.active_support.deprecation = :log

# Logging by JSON
if ENV["RAILS_LOG_TO_STDOUT"].present?
dkastl marked this conversation as resolved.
Show resolved Hide resolved
logger = ActiveSupport::Logger.new(STDOUT)
logger.formatter = config.log_formatter
config.logger = ActiveSupport::TaggedLogging.new(logger)
end
end
14 changes: 14 additions & 0 deletions config/initializers/lograge.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
Rails.application.configure do
config.lograge.enabled = true
config.lograge.formatter = Lograge::Formatters::Json.new
config.lograge.custom_payload do |controller|
current_user = controller.find_current_user
user_name = current_user&.logged? ? "#{current_user.login}" : "anonymous"
user_id = current_user&.logged? ? "#{current_user.id}" : "0"
{
remote_ip: controller.request.remote_ip,
user_name: user_name,
user_id: user_id
}
end
end
15 changes: 15 additions & 0 deletions config/puma.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
workers (ENV['PUMA_WORKERS'] || 3).to_i

before_fork do
require 'puma_worker_killer'

PumaWorkerKiller.config do |config|
config.ram = (ENV['RAILS_MEMORY_LIMIT'] || 2048).to_i
dkastl marked this conversation as resolved.
Show resolved Hide resolved
config.frequency = 10
config.percent_usage = 0.8
config.rolling_restart_frequency = false
config.reaper_status_logs = false
end

PumaWorkerKiller.start
end
Loading