Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[shopsys] made parameters overridable by ENV variables #2055

Merged
merged 5 commits into from
Oct 15, 2020
Merged

[shopsys] made parameters overridable by ENV variables #2055

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a697b04
made parameters overridable by ENV variables
Oct 1, 2020
6174d4d
fix acceptance tests
TomasLudvik Oct 1, 2020
108645b
affected different parameter approach in documentation
Oct 1, 2020
31e0171
set OVERWRITE_DOMAIN_URL via env variable in build_kubernetes.sh
Oct 1, 2020
4920485
added upgrade notes
Oct 3, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 3 additions & 3 deletions .ci/build_kubernetes.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,6 @@ cp project-base/config/parameters.yaml.dist project-base/config/parameters.yaml
yq write --inplace project-base/config/domains_urls.yaml domains_urls[0].url http://${FIRST_DOMAIN_HOSTNAME}:${NGINX_INGRESS_CONTROLLER_HOST_PORT}
grossmannmartin marked this conversation as resolved.
Show resolved Hide resolved
yq write --inplace project-base/config/domains_urls.yaml domains_urls[1].url http://${SECOND_DOMAIN_HOSTNAME}:${NGINX_INGRESS_CONTROLLER_HOST_PORT}

# Change "overwrite_domain_url" parameter for Selenium tests as containers "webserver" and "php-fpm" are bundled together in a pod "webserver-php-fpm"
yq write --inplace project-base/config/parameters_test.yaml parameters.overwrite_domain_url http://webserver-php-fpm:8080

# Pull or build Docker images for the current commit
DOCKER_IMAGE_TAG=ci-commit-${GIT_COMMIT}
DOCKER_ELASTIC_IMAGE_TAG=ci-elasticsearch-7
Expand Down Expand Up @@ -55,6 +52,8 @@ DOCKER_ELASTIC_IMAGE=${DOCKER_USERNAME}/elasticsearch:${DOCKER_ELASTIC_IMAGE_TAG
PATH_CONFIG_DIRECTORY='/var/www/html/project-base/config'
GOOGLE_CLOUD_STORAGE_BUCKET_NAME=''
GOOGLE_CLOUD_PROJECT_ID=''
# Change "OVERWRITE_DOMAIN_URL" parameter for Selenium tests as containers "webserver" and "php-fpm" are bundled together in a pod "webserver-php-fpm"
OVERWRITE_DOMAIN_URL='http://webserver-php-fpm:8080'

FILES=$( find project-base/kubernetes -type f )
VARS=(
Expand All @@ -65,6 +64,7 @@ VARS=(
PATH_CONFIG_DIRECTORY
GOOGLE_CLOUD_STORAGE_BUCKET_NAME
GOOGLE_CLOUD_PROJECT_ID
OVERWRITE_DOMAIN_URL
)

for FILE in $FILES; do
Expand Down
19 changes: 5 additions & 14 deletions composer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,6 @@
"guzzlehttp/guzzle": "^6.3",
"helios-ag/fm-elfinder-bundle": "^10.0.4",
"heureka/overeno-zakazniky": "^2.0.6",
"incenteev/composer-parameter-handler": "^2.1.3",
"intervention/image": "^2.3.14",
"jdorn/sql-formatter": "^1.2",
"jms/metadata": "^1.6",
Expand Down Expand Up @@ -186,12 +185,14 @@
},
"scripts": {
"post-install-cmd": [
"Incenteev\\ParameterHandler\\ScriptHandler::buildParameters",
"cp -n project-base/config/parameters.yaml.dist project-base/config/parameters.yaml || echo '[skiped] parameters.yaml exists'",
"cp -n project-base/config/parameters_test.yaml.dist project-base/config/parameters_test.yaml || echo '[skiped] parameters_test.yaml exists'",
"App\\Environment::checkEnvironment",
"@auto-scripts"
],
"post-update-cmd": [
"Incenteev\\ParameterHandler\\ScriptHandler::buildParameters",
"cp -n project-base/config/parameters.yaml.dist project-base/config/parameters.yaml || echo '[skiped] parameters.yaml exists'",
"cp -n project-base/config/parameters_test.yaml.dist project-base/config/parameters_test.yaml || echo '[skiped] parameters_test.yaml exists'",
"App\\Environment::checkEnvironment",
"@auto-scripts"
],
Expand Down Expand Up @@ -219,17 +220,7 @@
"symfony": {
"allow-contrib": true,
"require": "^4.4.0"
},
"incenteev-parameters": [
{
"file": "project-base/config/parameters.yaml",
"keep-outdated": true
},
{
"file": "project-base/config/parameters_test.yaml",
"keep-outdated": true
}
]
}
},
"replace": {
"shopsys/backend-api": "self.version",
Expand Down
4 changes: 2 additions & 2 deletions docs/cookbook/dumping-and-importing-the-database.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Then you can import the dump:
psql --quiet --username=database_user --host=database_host target_database_name < dump.sql
```

Replace `database_user`, `database_host` and `target_database_name` with the correct values (from your `config/parameters.yaml`).
Replace `database_user`, `database_host` and `target_database_name` with the correct values (from your `.env.local`).
The command will prompt you for the user's password.

## Importing database into a new database
Expand All @@ -49,5 +49,5 @@ Then you can import the dump:
psql --quiet --username=database_user --host=database_host target_database_name < dump.sql
```

Replace `database_user`, `database_host` and `target_database_name` with the correct values (from your `config/parameters.yaml`).
Replace `database_user`, `database_host` and `target_database_name` with the correct values (from your `.env.local`).
The command will prompt you for the user's password.
84 changes: 34 additions & 50 deletions docs/installation/application-configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,71 +1,55 @@
# Application Configuration

The application is configurable by [Symfony configuration files](https://symfony.com/doc/4.4/configuration.html#configuration-parameters) or via environment variables which allows you to overwrite them.
The application is configurable by [Symfony configuration files](https://symfony.com/doc/4.4/configuration.html#configuration-parameters) or via [environment variables](https://symfony.com/doc/4.4/configuration.html#configuration-environments) which allows you to overwrite them.

- [Configuration by parameters](#configuration-by-parameters)
- [Configuration by environment variables](#configuration-by-environment-variables)

## Configuration by parameters

For operating Shopsys Framework it is needed to have correctly set connections to external services via `parameters.yaml` config.
From the clean project, during composer installation process it will prompt you to set the values of parameters (`config/parameters.yaml`):

| Name | Description |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| `database_host` | access data of your PostgreSQL database |
| `database_port` | ... |
| `database_name` | ... |
| `database_user` | ... |
| `database_password` | ... |
| `elasticsearch_host` | host of your Elasticsearch |
| `redis_host` | host of your Redis storage (credentials are not supported right now) |
| `mailer_transport` | access data of your mail server |
| `mailer_host` | ... |
| `mailer_user` | ... |
| `mailer_password` | ... |
| `mailer_disable_delivery` | set to `true` if you don't want to send any emails |
| `mailer_master_email_address` | set if you want to send all emails to one address (useful for development) |
| `mailer_delivery_whitelist` | set as array with regex text items if you want to have master email but allow sending to specific addresses |
| `secret` | randomly generated secret token |
| `trusted_proxies` | proxies that are trusted to pass traffic, used mainly for production |

Composer will then prompt you to set parameters for testing environment (`config/parameters_test.yaml`):

| Name | Description |
| ---------------------------------- | ----------------------------------------------------------------------------- |
| `test_database_host` | access data of your PostgreSQL database for tests |
| `test_database_port` | ... |
| `test_database_name` | ... |
| `test_database_user` | ... |
| `test_database_password` | ... |
| `overwrite_domain_url` | overwrites URL of all domains for acceptance testing (set to `~` to disable) |
| `selenium_server_host` | with native installation the selenium server is on `localhost` |
| `test_mailer_transport` | access data of your mail server for tests |
| `test_mailer_host` | ... |
| `test_mailer_user` | ... |
| `test_mailer_password` | ... |
| `shopsys.content_dir_name` | web/content-test/ directory is used instead of web/content/ during the tests |
## Configuration parameters

For operating Shopsys Framework it is needed to have correctly set connections to external services via ENV variables.

!!! note
All default values use default ports for all external services like PostgreSQL database, elasticsearch, redis, ...

!!! tip
Host values can be modified or can be aliased for your Operating System via `/etc/hosts` or `C:\Windows\System32\drivers\etc\hosts`

## Configuration by environment variables

Environment variables are really handy to configure the right setting in the desired application environment.
You may want to set some settings in a different way (such as production, test, or CI servers).
[Setting environment variables](/introduction/setting-environment-variables) depends on environment of your application.

!!! tip
To improve performance you can optionally run `composer dump-env`. [See Symfony documentation for further information.](https://symfony.com/doc/4.4/configuration.html#configuring-environment-variables-in-production)

### Application

| Name | Default | Description |
| -------------------------------------- | ------- | ------------------------------------------------------------------------------------ |
| `REDIS_PREFIX` | `''` | separates more projects that use the same redis service |
| `ELASTIC_SEARCH_INDEX_PREFIX` | `''` | separates more projects that use the same elasticsearch service |
| `IGNORE_DEFAULT_ADMIN_PASSWORD_CHECK` | `'0'` | set to `true` if you want to allow administrators to log in with default credentials |
!!! note
To propagate the change of `MAILER_DELIVERY_WHITELIST`, `MAILER_DISABLE_DELIVERY` and `MAILER_MASTER_EMAIL_ADDRESS` cache must be cleaned.

| Name | Default | Description |
| -------------------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------ |
| `DATABASE_HOST` | `'postgres'` | access data of your PostgreSQL database |
| `DATABASE_PORT` | `null` | ... |
| `DATABASE_NAME` | `'shopsys'` | ... |
| `DATABASE_USER` | `'root'` | ... |
| `DATABASE_PASSWORD` | `'root'` | ... |
| `ELASTICSEARCH_HOST` | `'elasticsearch:9200'` | host of your Elasticsearch |
| `REDIS_HOST` | `'redis'` | host of your Redis storage (credentials are not supported right now) |
| `REDIS_PREFIX` | `''` | separates more projects that use the same redis service |
| `MAILER_TRANSPORT` | `'smtp'` | access data of your mail server |
| `MAILER_HOST` | `'smtp-server'` | ... |
| `MAILER_USER` | `null` | ... |
| `MAILER_PASSWORD` | `null` | ... |
| `MAILER_DELIVERY_WHITELIST` | `'/@shopsys\.com$/'` | regex text items if you want to have master email but allow sending to specific addresses (set as text separated by comma for multiple values) |
| `MAILER_DISABLE_DELIVERY` | `false` | set to `true` if you don't want to send any emails |
| `MAILER_MASTER_EMAIL_ADDRESS` | `'no-reply@shopsys.com'` | set if you want to send all emails to one address (useful for development) |
| `APP_SECRET` | `'ThisTokenIsNotSoSecretChangeIt'` | randomly generated secret token |
| `ELASTIC_SEARCH_INDEX_PREFIX` | `''` | separates more projects that use the same elasticsearch service |
| `IGNORE_DEFAULT_ADMIN_PASSWORD_CHECK` | `'0'` | set to `true` if you want to allow administrators to log in with default credentials |
| `OVERWRITE_DOMAIN_URL` | `'http://webserver:8080'` | overwrites URL of all domains for acceptance testing (set to `~` to disable) |
| `SELENIUM_SERVER_HOST` | `'selenium-server'` | with native installation the selenium server is on `localhost` |
| `SHOPSYS_CONTENT_DIR_NAME` | `'content-test'` | web/content-test/ directory is used instead of web/content/ during the tests |
| `TRUSTED_PROXIES` | `'127.0.0.1'` | proxies that are trusted to pass traffic, used mainly for production (set as text separated by comma for multiple values) |

henzigo marked this conversation as resolved.
Show resolved Hide resolved

### Google Cloud Bundle

Expand Down
2 changes: 1 addition & 1 deletion docs/installation/installation-using-docker-on-production-server.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ echo $'domains_urls:
' > config/domains_urls.yaml
```

Then we check whether `mailer_master_email_address` property in [`parameters.yaml.dist`](https://github.com/shopsys/shopsys/blob/master/project-base/config/parameters.yaml.dist) is set correctly.
Then we check whether ENV variable `MAILER_MASTER_EMAIL_ADDRESS` in [`.env` or `.env.local`](https://github.com/shopsys/shopsys/blob/master/project-base/.env) is set correctly.

After the project is setup correctly, we launch the build of php-fpm container by docker build command that will build image with composer, npm packages and created assets.

Expand Down
10 changes: 5 additions & 5 deletions docs/introduction/running-acceptance-tests.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ docker exec -it shopsys-framework-php-fpm bash

!!! note
For `selenium-server` to be able to connect to you `webserver` container and access your application, all domains should have URL set to `http://webserver:8000`.
This is done via parameter `%overwrite_domain_url%` defined in `config/parameters_test.yaml`.
This is done via ENV `OVERWRITE_DOMAIN_URL` defined in `.env.test` or `.env.test.local`.
Everything should be configured for you by default but it is important to keep the domain URL overwriting in mind when dealing with acceptance tests.

If you are logged into your `php-fpm` container and have the `%overwrite_domain_url%` parameter properly set,
If you are logged into your `php-fpm` container and have the `OVERWRITE_DOMAIN_URL` ENV properly set,
you can run acceptance tests:
```sh
php phing tests-acceptance
Expand Down Expand Up @@ -55,10 +55,10 @@ For running acceptance tests you need to install [Google Chrome browser](https:/
You must choose compatible versions of Google Chrome and ChromeDriver.
As Chrome browser has auto-update enabled by default this may require you to update ChromeDriver from time to time.

When installing Shopsys Framework natively, it is important to update parameters in `config/parameters_test.yaml`:
When installing Shopsys Framework natively, it is important to update parameters in `.env.test.local`:

* `overwrite_domain_url: ~` (disables domain URL overwriting in `TEST` environment)
* `selenium_server_host: 127.0.0.1`
* `OVERWRITE_DOMAIN_URL=` (disables domain URL overwriting in `TEST` environment)
* `SELENIUM_SERVER_HOST=127.0.0.1`

### Installing Google Chrome browser
Download and install Google Chrome browser from <https://www.google.com/chrome/browser/desktop/>
Expand Down
4 changes: 3 additions & 1 deletion packages/framework/src/Twig/MailerSettingExtension.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,11 @@ class MailerSettingExtension extends AbstractExtension
public function __construct(ContainerInterface $container, Environment $twigEnvironment)
{
$this->container = $container;

$this->mailerWhitelistExpressions = $this->container->getParameter('mailer_delivery_whitelist');
$this->isDeliveryDisabled = $this->container->getParameter('mailer_disable_delivery');
$this->mailerMasterEmailAddress = $this->container->getParameter('mailer_master_email_address');
$this->mailerWhitelistExpressions = $this->container->getParameter('mailer_delivery_whitelist');

$this->twigEnvironment = $twigEnvironment;
}

Expand Down
20 changes: 19 additions & 1 deletion project-base/.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,23 @@
# https://symfony.com/doc/current/best_practices/configuration.html#infrastructure-related-configuration

IGNORE_DEFAULT_ADMIN_PASSWORD_CHECK='0'
REDIS_PREFIX=''
DATABASE_HOST=postgres
DATABASE_PORT=5432
DATABASE_NAME=shopsys
DATABASE_USER=root
DATABASE_PASSWORD=root
s3tezsky marked this conversation as resolved.
Show resolved Hide resolved
ELASTICSEARCH_HOST='elasticsearch:9200'
ELASTIC_SEARCH_INDEX_PREFIX=''
REDIS_HOST=redis
REDIS_PREFIX=''

MAILER_TRANSPORT=smtp
MAILER_HOST=smtp-server
MAILER_USER=
MAILER_PASSWORD=
s3tezsky marked this conversation as resolved.
Show resolved Hide resolved
MAILER_DELIVERY_WHITELIST=/@shopsys\.com$/
MAILER_DISABLE_DELIVERY=0
grossmannmartin marked this conversation as resolved.
Show resolved Hide resolved
MAILER_MASTER_EMAIL_ADDRESS=no-reply@shopsys.com

APP_SECRET=ThisTokenIsNotSoSecretChangeIt
TRUSTED_PROXIES=127.0.0.1
5 changes: 5 additions & 0 deletions project-base/.env.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,6 @@
# define your env variables for the test environment here

DATABASE_NAME=shopsys-test
SELENIUM_SERVER_HOST=selenium-server
OVERWRITE_DOMAIN_URL=http://webserver:8080
SHOPSYS_CONTENT_DIR_NAME=content-test
s3tezsky marked this conversation as resolved.
Show resolved Hide resolved
3 changes: 2 additions & 1 deletion project-base/build/codeception.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ paths:
data: ../tests/App/Acceptance/data
support: ../tests/App/Test/Codeception
params:
- ../config/parameters_test.yaml
- ../.env
- ../.env.test
settings:
colors: true
memory_limit: 1024M