Docker Hub Automated Build #4

alexanderilyin · 2018-10-10T07:34:57Z

RELEASE NOTES

Ignore PHPSorm IDE configuration files.
Improved formatting in README.md.
Created configuration for docker-compose for more convenient development of Dockerfiles.
Use Pinba2 sources from current context instead of cloning latest master.
Planned next iteration of improvements for Dockerfile.

ADDITIONAL INFORMATION

The goal of this change is to clean up existing Dockerfile so ready-to-use docker images could be provided by using existing Docker Hub infrastructure. You can see a configured example of Docker Hub repository at https://hub.docker.com/r/alexanderilyin/pinba2/

Here are build settings I would recommend:

alexanderilyin

OPEN SOURCE FTW

alexanderilyin · 2018-10-10T07:36:43Z

README.md

- [Mariadb/debian](docker/debian-mariadb/) (unfinished)
+### Fedora 25
+
+Kinda works. [`Dockerfile`](docker/fedora-25/)


Once #2 is resolved this description should be updated.

alexanderilyin · 2018-10-10T07:37:14Z

README.md

+
+### MariaDB
+
+Unfinished. [`Dockerfile`](docker/debian-mariadb/).


I would recommend using WIP or Work In Progress instead.

alexanderilyin · 2018-10-10T07:38:35Z

docker/debian-mariadb/Dockerfile

@@ -2,7 +2,7 @@ FROM		mariadb:10.1

 MAINTAINER	Anton Povarov "anton.povarov@gmail.com"

-COPY		build-from-source.sh /opt/
+COPY		./docker/build-from-source.sh /opt/
 COPY		10-pinba-init.sh /docker-entrypoint-initdb.d/


BTW 10-pinba-init.sh is missing in the repository, could someone add it, please?

alexanderilyin · 2018-10-10T07:41:26Z

docker/fedora-25/Dockerfile

@@ -1,13 +1,15 @@
 FROM        fedora:25
 MAINTAINER  Anton Povarov "anton.povarov@gmail.com"

+# TODO: Use multi-stage builds https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds


Using multi-stage Dockerfile will allow removing cleanup which does not work as expected because of nature of layers :

RUN dnf remove -y git-core gcc gcc-c++ boost-devel cmake autoconf automake libtool mariadb-devel && \ dnf autoremove -y && \ dnf clean all

alexanderilyin · 2018-10-10T07:43:54Z

docker/fedora-25/Dockerfile

-			git clone https://github.com/anton-povarov/pinba2 && \
-			git clone https://github.com/anton-povarov/meow && \
-			git clone https://github.com/nanomsg/nanomsg ;
+			git clone --branch master --single-branch --depth 1  https://github.com/anton-povarov/meow && \


Unfortunately, https://github.com/anton-povarov/meow/releases has no releases so we have to use master which is no the best option.

alexanderilyin · 2018-10-10T07:46:33Z

docker/fedora-25/docker-entrypoint.sh

@@ -45,12 +46,14 @@ if [ $1 = "mysqld" ]; then
 	EOSQL

 	# install plugin and create default db
+	# TODO: Use --init-file https://mariadb.com/kb/en/library/server-system-variables/#init_file
 	mysql --protocol=socket -uroot <<-EOSQL
 		INSTALL PLUGIN pinba SONAME 'libpinba_engine2.so';


--init-file is designed for his purpose. I will allow storing all related init queries in the dedicated *.sql file.

alexanderilyin · 2018-10-10T07:50:04Z

docker-compose.yml

+services:
+
+  fedora-25:
+    image: anton-povarov/pinba2:fedora-25


This will allow you to build & push images with predefined names without providing them every time on command line, for example:

$ docker-compose build fedora-25 Building fedora-25 Step 1/13 : FROM fedora:25 ---> 9cffd21a45e3 Step 2/13 : MAINTAINER Anton Povarov "anton.povarov@gmail.com" ---> Using cache ...

$ docker-compose push fedora-25 The push refers to repository [docker.io/anton-povarov/pinba2] ...

alexanderilyin · 2018-10-10T07:51:59Z

BTW I have a question why MariaDB was used and not a Percona?

anton-povarov · 2018-10-10T20:38:20Z

Mariadb and not Percona is not a requirement, just something i had easy access to at the time.
Will happily test with Percona (and generally mysql api versions 5.7/8.0+).

anton-povarov · 2018-10-10T20:39:40Z

Thing is - as pinba is a plugin - it depends on mysql/mariadb/percona internals that tend to change between versions.
So need to adapt the codebase as new versions are released.

anton-povarov · 2018-10-10T20:40:44Z

Using multi-stage Dockerfile will allow removing cleanup which does not work as expected because of nature of layers

could you please elaborate on that, hate the cleanup code, will happily use any mechanism to get rid of it.

alexanderilyin · 2018-10-10T22:45:36Z

@anton-povarov I will update PR to use multi-stage Dockerfile as soon as I will be able to build it successfully. I thing I will have update in next few days on this.

alexanderilyin · 2018-10-16T07:28:16Z

@anton-povarov I've updated PR, unfortunately, to make Automated Builds work on Docker Hub I had to move Dockerfile to the root directory. Here are the main instructions you have to looks at to get the idea:

FROM fedora:25 as builder
...
FROM fedora:25
...
COPY --from=builder /_src/pinba2/mysql_engine/.libs/libpinba_engine2.so /usr/lib64/mysql/plugin/libpinba_engine2.so
...

You can read about it more here:

https://docs.docker.com/develop/develop-images/multistage-build/

Docker image from this PR is available on Docker Hub:

https://hub.docker.com/r/alexanderilyin/pinba2/tags/

Problem is it still fails with error from #2:

AF6469:~ ailyin$ docker run  alexanderilyin/pinba2:fedora-25-docker_hub
...
ERROR 1126 (HY000) at line 1: Can't open shared library '/usr/lib64/mysql/plugin/libpinba_engine2.so' (errno: 13, undefined symbol: my_print_error_service)

alexanderilyin · 2018-10-16T07:31:57Z

Dockerfile

+FROM fedora:25
+MAINTAINER Anton Povarov "anton.povarov@gmail.com"
+
+RUN dnf install -y file hostname mariadb-server


@anton-povarov do you really need file for production instance? The same question is about hostname but IMHO it might be just missing dependencies for mariadb-server.

alexanderilyin · 2018-10-16T07:34:39Z

Dockerfile

+    mariadb-server
+
+RUN git clone --branch master --single-branch --depth 1 https://github.com/anton-povarov/meow /_src/meow
+RUN git clone --branch master --single-branch --depth 1 https://github.com/nanomsg/nanomsg /_src/nanomsg


@anton-povarov I would recommend lock nanomsg to a specific release for example
https://github.com/nanomsg/nanomsg/releases/tag/1.1.5 and download & extract tar.gz using ADD.

alexanderilyin · 2018-10-16T07:35:41Z

Dockerfile

+    hostname \
+    mariadb-server
+
+RUN git clone --branch master --single-branch --depth 1 https://github.com/anton-povarov/meow /_src/meow


@anton-povarov It would be nice to create initial release so we could lock on it and use cur ... | tar ... as well.

alexanderilyin · 2018-10-16T07:37:27Z

Dockerfile

+COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
+
+ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
+EXPOSE 30003


@anton-povarov AFAIK Pinba could listen on TCP/UDP ports for writing and you have to use MySQL port for reading. It would be nice to explicitly set it in Dockerfile.

alexanderilyin · 2018-10-16T08:20:23Z

I found out that Docker Cloud allows to set build context, so it will allow keeping Dockerfile in a directory other than root, I will research it later this week:

https://cloud.docker.com/repository/docker/alexanderilyin/pinba2

Another option is to use CircleCI for building from multiple base images, but it makes sense only after #2 is fixed.

alexanderilyin · 2018-10-26T22:02:48Z

Dockerfile

+FROM fedora:25
+MAINTAINER Anton Povarov "anton.povarov@gmail.com"
+
+RUN dnf install -y file hostname mariadb-server


@anton-povarov / @mkevac maybe it will be better to use official docker image as a base locked on a particular version rather than install latest available mariadb-server during build runtime, WDYT?

image https://hub.docker.com/_/mariadb/

alexanderilyin · 2018-11-02T08:38:39Z

@mkevac please take a look on this PR one more time. Docker Image should be available in 25 minutes here:

https://hub.docker.com/r/alexanderilyin/pinba2/

It's building right now:

Locally you can build & start image like this:

docker-compose build
docker-compose up

P. S.

If you can share your Docker Hub username I can add you to collaborators so you can take a look on configuration inside.

mkevac · 2018-11-02T14:42:30Z

@alexanderilyin it's working great
thank you!

alexanderilyin · 2018-11-02T14:57:26Z

@mkevac sweet. I just checked and the image was built automatically:

docker pull alexanderilyin/pinba2:fedora-25-docker_hub

The only problem that Docker Hub is extremely slow and it took 60+ minutes to build it remotely versus 25 minutes on MBP.

alexanderilyin · 2018-11-02T16:15:27Z

@mkevac any chances that image could be available here?

https://hub.docker.com/u/badoo/

mkevac · 2018-11-03T13:56:27Z

@alexanderilyin we'll try

anton-povarov · 2018-12-11T17:55:37Z

unsuccessful in finding owner as of now :(
@mkevac - found anybody ?

mkevac · 2018-12-11T18:02:03Z

@anton-povarov nope, but I've just contacted my friend in Docker.

alexanderilyin added 3 commits October 9, 2018 21:46

Clenup;

ac5486c

Review docker-entrypoint.sh;

404fd9b

Added TODO with correct URLs;

40574f5

alexanderilyin commented Oct 10, 2018

View reviewed changes

alexanderilyin added 6 commits October 15, 2018 17:03

Merge branch 'master' of github.com:anton-povarov/pinba2 into docker_hub

f692d7a

Use multi-stage docker builds;

8a84c80

Cleanup;

6a218d2

Add debug for Docker Hub;

2a0838b

Docker Hub uses context from Dockerfile dir;

c7f8fc1

Cleanup;

f08deb9

alexanderilyin commented Oct 16, 2018

View reviewed changes

Use 1.1.5 release for nanomsg instead of master;

611f257

alexanderilyin commented Oct 26, 2018

View reviewed changes

This was referenced Nov 2, 2018

fix build by using sources and linking with libmysqlservices.a #5

Merged

Docker Image does NOT build #2

Closed

alexanderilyin added 2 commits November 2, 2018 01:14

Merge branch 'master' of github.com:anton-povarov/pinba2 into docker_hub

963c070

Cleanup;

4030838

mkevac merged commit 98edbb9 into badoo:master Nov 2, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker Hub Automated Build #4

Docker Hub Automated Build #4

alexanderilyin commented Oct 10, 2018

alexanderilyin left a comment

alexanderilyin Oct 10, 2018

alexanderilyin Oct 10, 2018

alexanderilyin Oct 10, 2018

alexanderilyin Oct 10, 2018

alexanderilyin Oct 10, 2018

alexanderilyin Oct 10, 2018

alexanderilyin Oct 10, 2018

alexanderilyin commented Oct 10, 2018

anton-povarov commented Oct 10, 2018

anton-povarov commented Oct 10, 2018

anton-povarov commented Oct 10, 2018

alexanderilyin commented Oct 10, 2018

alexanderilyin commented Oct 16, 2018

alexanderilyin Oct 16, 2018

alexanderilyin Oct 16, 2018

alexanderilyin Oct 16, 2018 •

edited

Loading

alexanderilyin Oct 16, 2018

alexanderilyin commented Oct 16, 2018

alexanderilyin Oct 26, 2018

alexanderilyin commented Nov 2, 2018

mkevac commented Nov 2, 2018

alexanderilyin commented Nov 2, 2018

alexanderilyin commented Nov 2, 2018

mkevac commented Nov 3, 2018

anton-povarov commented Dec 11, 2018

mkevac commented Dec 11, 2018


		### MariaDB

		Unfinished. [`Dockerfile`](docker/debian-mariadb/).

Docker Hub Automated Build #4

Docker Hub Automated Build #4

Conversation

alexanderilyin commented Oct 10, 2018

RELEASE NOTES

ADDITIONAL INFORMATION

alexanderilyin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexanderilyin commented Oct 10, 2018

anton-povarov commented Oct 10, 2018

anton-povarov commented Oct 10, 2018

anton-povarov commented Oct 10, 2018

alexanderilyin commented Oct 10, 2018

alexanderilyin commented Oct 16, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexanderilyin Oct 16, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexanderilyin commented Oct 16, 2018

Choose a reason for hiding this comment

alexanderilyin commented Nov 2, 2018

P. S.

mkevac commented Nov 2, 2018

alexanderilyin commented Nov 2, 2018

alexanderilyin commented Nov 2, 2018

mkevac commented Nov 3, 2018

anton-povarov commented Dec 11, 2018

mkevac commented Dec 11, 2018

alexanderilyin Oct 16, 2018 •

edited

Loading