Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] s2i and general extendability support #45

Merged
merged 8 commits into from
Oct 4, 2017

Conversation

hhorak
Copy link
Member

@hhorak hhorak commented Sep 17, 2017

This implements a similar approach to extend the image either using s2i, or by mounting the extending files into /opt/app-root. It also adds some OpenShift tests, together with test-lib-openshift.sh that should be moved to https://github.com/sclorg/container-common-scripts/.

This is very similar to the MongoDB PoC: sclorg/mongodb-container#239

It adds extending support using [source-to-image](https://github.com/openshift/source-to-image).

For example to build customized MariaDB database image `my-mariadb-centos7` with configuration in `~/image-configuration/` run:

```
$ s2i build ~/image-configuration/ centos/mariadb-101-centos7 my-mariadb-centos7
```

The directory passed to `s2i build` can contain these directories:
- `mysql-cfg/`
  - when starting the container, files from this directory will be used as a configuration for the `mysqld` daemon
    - `envsubst` command is run on this file to still allow customization of the image using environmental variables

- `mysql-pre-init/`
  - shell scripts (`*.sh`) available in this directory are sourced before `mysqld` daemon is started

- `mysql-init/`
  - shell scripts (`*.sh`) available in this directory are sourced when `mysqld` daemon is started locally
    - in this phase, use `${mysql_flags}` to connect to the locally running daemon, for example `mysql $mysql_flags < dump.sql`

Variables that can be used in the scripts provided to s2i:

- `$mysql_flags` -- arguments for the `mysql` tool that will connect to the locally running `mysqld` during initialization
- `$MYSQL_RUNNING_AS_MASTER` -- variable defined when the container is run with `run-mysqld-master` command
- `$MYSQL_RUNNING_AS_SLAVE` -- variable defined when the container is run with `run-mysqld-slave` command
- `$MYSQL_DATADIR_FIRST_INIT` -- variable defined when the container was initialized from the empty data dir

During `s2i build` all provided files are copied into `/opt/app-root/src` directory into the resulting image. If some configuration files are present in the destination directory, files with the same name are overwritten. Also only one file with the same name can be used for customization and user provided files are preferred over default files in `/usr/share/container-scripts/mysql/`- so it is possible to overwrite them.

Same configuration directory structure can be used to customize the image every time the image is started using `docker run`. The directory has to be mounted into `/opt/app-root/src/` in the image (`-v ./image-configuration/:/opt/app-root/src/`). This overwrites customization built into the image.
This is very similar to the MongoDB PoC: sclorg/mongodb-container#239

It adds extending support using [source-to-image](https://github.com/openshift/source-to-image).

For example to build customized MariaDB database image `my-mariadb-centos7` with configuration in `~/image-configuration/` run:

```
$ s2i build ~/image-configuration/ centos/mariadb-101-centos7 my-mariadb-centos7
```

The directory passed to `s2i build` can contain these directories:
- `mysql-cfg/`
  - when starting the container, files from this directory will be used as a configuration for the `mysqld` daemon
    - `envsubst` command is run on this file to still allow customization of the image using environmental variables

- `mysql-pre-init/`
  - shell scripts (`*.sh`) available in this directory are sourced before `mysqld` daemon is started

- `mysql-init/`
  - shell scripts (`*.sh`) available in this directory are sourced when `mysqld` daemon is started locally
    - in this phase, use `${mysql_flags}` to connect to the locally running daemon, for example `mysql $mysql_flags < dump.sql`

Variables that can be used in the scripts provided to s2i:

- `$mysql_flags` -- arguments for the `mysql` tool that will connect to the locally running `mysqld` during initialization
- `$MYSQL_RUNNING_AS_MASTER` -- variable defined when the container is run with `run-mysqld-master` command
- `$MYSQL_RUNNING_AS_SLAVE` -- variable defined when the container is run with `run-mysqld-slave` command
- `$MYSQL_DATADIR_FIRST_INIT` -- variable defined when the container was initialized from the empty data dir

During `s2i build` all provided files are copied into `/opt/app-root/src` directory into the resulting image. If some configuration files are present in the destination directory, files with the same name are overwritten. Also only one file with the same name can be used for customization and user provided files are preferred over default files in `/usr/share/container-scripts/mysql/`- so it is possible to overwrite them.

Same configuration directory structure can be used to customize the image every time the image is started using `docker run`. The directory has to be mounted into `/opt/app-root/src/` in the image (`-v ./image-configuration/:/opt/app-root/src/`). This overwrites customization built into the image.
File test-lib-openshift.sh is a candidate for container-common-scripts.
@hhorak hhorak mentioned this pull request Sep 18, 2017
@hhorak
Copy link
Member Author

hhorak commented Sep 18, 2017

As @TomasTomecek suggested in #32, we should document extending images in the Dockerfile, so one can e.g. change USER to 0 to be able to install RPMs..

@hhorak
Copy link
Member Author

hhorak commented Sep 18, 2017

We should also include an SSL example as part of the documentatoin/test.

@hhorak hhorak changed the title s2i and general extendability support [WIP] s2i and general extendability support Sep 18, 2017
@TomasTomecek
Copy link
Contributor

@mkocka promised he would review this PR

@mkocka
Copy link

mkocka commented Sep 20, 2017

[mkocka@warp mariadb-container]$ make build TARGET=centos7 VERSIONS=10.1
make[1]: Entering directory '/home/mkocka/work/mariadb-container'
mkdir -p 10.1/root
go-md2man -in "10.1/README.md" -out "10.1/root/help.1"
/bin/sh: go-md2man: command not found
make[1]: *** [common/common.mk:80: 10.1/root/help.1] Error 127
make[1]: Leaving directory '/home/mkocka/work/mariadb-container'
make: *** [common/common.mk:42: build-serial] Error 2

I am having trouble to test this, see ^^
I am using right branch, up to date:

[mkocka@warp mariadb-container]$ git branch
master

  • s2i-support-3

@TomasTomecek
Copy link
Contributor

/bin/sh: go-md2man: command not found

You should install the package: golang-github-cpuguy83-go-md2man

@omron93
Copy link
Contributor

omron93 commented Sep 20, 2017

@mkocka Or you can run make build TARGET=centos7 VERSIONS=10.1 SKIP_SQUASH=1

@mkocka
Copy link

mkocka commented Sep 26, 2017

Testing s2i mount
chown: changing ownership of '/tmp/tmp.V1N7QPOHRE/test-app/mysql-data/init.sql': Operation not permitted
... This problem occurred for several files in that folder

Tests failed.
make: *** [common/common.mk:63: test] Error 1

@hhorak
Copy link
Member Author

hhorak commented Sep 26, 2017

@mkocka what permission the '/tmp/tmp.V1N7QPOHRE/test-app/mysql-data/init.sql' file has?

@hhorak
Copy link
Member Author

hhorak commented Oct 3, 2017

[test]

1 similar comment
@omron93
Copy link
Contributor

omron93 commented Oct 4, 2017

[test]

@hhorak hhorak merged commit d7774fa into sclorg:master Oct 4, 2017
hhorak added a commit to hhorak/mysql-container that referenced this pull request Oct 22, 2017
This is very similar to the MariaDB: sclorg/mariadb-container#45

It adds extending support using [source-to-image](https://github.com/openshift/source-to-image).

For example to build customized MySQL database image `my-mysql-centos7` with configuration in `~/image-configuration/` run:

```
$ s2i build ~/image-configuration/ centos/mysql-57-centos7 my-mysql-centos7
```

The directory passed to `s2i build` can contain these directories:
- `mysql-cfg/`
  - when starting the container, files from this directory will be used as a configuration for the `mysqld` daemon
  - `envsubst` command is run on this file to still allow customization of the image using environmental variables

- `mysql-pre-init/`
  - shell scripts (`*.sh`) available in this directory are sourced before `mysqld` daemon is started

- `mysql-init/`
  - shell scripts (`*.sh`) available in this directory are sourced when `mysqld` daemon is started locally
  - in this phase, use `${mysql_flags}` to connect to the locally running daemon, for example `mysql $mysql_flags < dump.sql`

Variables that can be used in the scripts provided to s2i:

- `$mysql_flags` -- arguments for the `mysql` tool that will connect to the locally running `mysqld` during initialization
- `$MYSQL_RUNNING_AS_MASTER` -- variable defined when the container is run with `run-mysqld-master` command
- `$MYSQL_RUNNING_AS_SLAVE` -- variable defined when the container is run with `run-mysqld-slave` command
- `$MYSQL_DATADIR_FIRST_INIT` -- variable defined when the container was initialized from the empty data dir

During `s2i build` all provided files are copied into `/opt/app-root/src` directory into the resulting image. If some configuration files are present in the destination directory, files with the same name are overwritten. Also only one file with the same name can be used for customization and user provided files are preferred over default files in `/usr/share/container-scripts/mysql/`- so it is possible to overwrite them.

Same configuration directory structure can be used to customize the image every time the image is started using `docker run`. The directory has to be mounted into `/opt/app-root/src/` in the image (`-v ./image-configuration/:/opt/app-root/src/`). This overwrites customization built into the image.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants