Run tests in isolation (using Docker) #194

sstok · 2017-11-13T10:33:30Z

Q	A
Bug fix?	no
New feature?	no
BC breaks?	no
Deprecations?	no
Tests pass?	yes
Fixed tickets
License	MIT
Doc PR

This makes the tests less dependent on the developer’s system. and FINALLY! Fixes the ICU versioning problem.

Note: this is a proof of concept, I am open to any suggestions 👍

This makes the tests less dependent on the developer’s system. and FINALLY! Fixes the ICU versioning problem.

sstok · 2017-11-13T15:48:52Z

HAHAHA IT WORKED!

dkarlovi · 2017-11-13T17:32:33Z

docker/Dockerfile-qa

+        pdo_pgsql \
+        pgsql \
+        bcmath \
+        intl \


It depends, I'm not sure this is required for the qa tools 🤔 maybe for the main dev image it's handy for remote debugging and coverage (although php-dbg is more recommended now).

TIL php-dbg can collect coverage data. 👍

BTW @jakzal confirmed in jakzal/phpqa#18 (comment) you can already use phpdbg from his image so this can be ignored.

dkarlovi · 2017-11-13T17:35:09Z

Makefile

+test-full:
+	export SYMFONY_DEPRECATIONS_HELPER=strict
+
+	# These tasks should be executed in paralel.


https://www.gnu.org/software/parallel/

Yes, I know. I already tried this in the past with the script symfony/symfony uses but failed to make it work then 😃

If you can help with this that would be great 👍
FYI I am away from ⌨️ on Wednesday and Thursday.

Just tried it, don't think you actually need to mess with parallel here, DB specific tests here are 7sec on my system in total while the regular part is ~30s.

dkarlovi · 2017-11-17T15:56:15Z

IMO once you merge this, you should consider tagging to see what happens (with the mono-repo and all).

sstok · 2017-11-17T16:15:04Z

The splitting of commits is not automated yet 😛 I will try to contact Fabien if he can set this up for RollerworksSearch 👍

dkarlovi · 2017-11-17T17:59:16Z

Don't know what exactly he does to set it up, but the splitsh-lite tool was made available for your own use, you can do it on your own if he doesn't have time to do it.

I took the scripts from enqueue-dev.

dkarlovi · 2017-11-17T19:03:01Z

Makefile


-dist: cs-full phpstan test-full
+dist: install cs-full phpstan test-full


Install probably not needed here, especially since there's no composer.phar on all systems (mine, for example).

dkarlovi · 2017-11-17T19:04:05Z

Makefile

 ci: cs-full-check phpstan test-full

+install:
+	composer.phar install --no-progress --no-interaction --no-suggest --optimize-autoloader --ansi


Can we use composer from Docker or something? Hardcoding composer.phar here doesn't seem right

dkarlovi · 2017-11-17T19:05:58Z

Makefile

+test-full:
+	export SYMFONY_DEPRECATIONS_HELPER=strict
+
+	# These tasks should be executed in paralel.


Just tried it, don't think you actually need to mess with parallel here, DB specific tests here are 7sec on my system in total while the regular part is ~30s.

dkarlovi · 2017-11-17T19:52:41Z

Makefile

+	phpdbg -qrr vendor/bin/phpunit --verbose --configuration travis/mysql.travis.xml --coverage-php build/cov/coverage-phpunit-mysql.cov
+	phpdbg -qrr phpcov.phar merge --clover build/logs/clover.xml build/cov
+
+test-isolated: docker-up


I'd go with simpler labels for targets you wish developers to use directly. For example, if you'd document the development workflow, you'd want to use simpler labels for human use, sort of like public methods vs private methods. :)

dkarlovi · 2017-11-17T19:53:04Z

Makefile

+test-with-coverage:
+	export SYMFONY_DEPRECATIONS_HELPER=strict
+
+	curl -Ls https://phar.phpunit.de/phpcov.phar > phpcov.phar


Actually not needed as the qa image contains it.

dkarlovi · 2017-11-17T19:54:14Z

docker-compose.yaml

+        volumes:
+            - pg-data:/var/lib/postgresql/data:rw
+        ports:
+            - "5432:5433"


Is this an error? Exporting a port on a off-by-one port?

This is only used for the host. Some people (including me) have a local instance running.
The docker instances still use the original port.

In fact I have been pulling out my hairs why the hell I can't connect from the host to this server instance 💀 or MariaDB for that matter. Elasticsearch connects without any issues.

Yeah, I thought so. If you're only changing the port to avoid collisions, I'd go with something way off instead of just off by one (because it might be considered a typo).

For example, prefixing all ports with 5?

MySQL: 53306
PgSQL: 55432
Elastic: 59200
etc.

That way you see from a mile away it's very different, there's a really low chance of collisions and you can actually document the difference easily. :)

sstok · 2017-11-25T15:41:23Z

I tagged the sub-splits manually for now. I am working on a more permanent solution 👍

sstok · 2017-12-03T15:11:12Z

Good news HubKit now allows to split a monolith repository while merging, and... can synchronize a release for all split repositories! So finally no more outdated releases 🎉

I am continuing work on RollerworksSearch 👍

dkarlovi · 2017-12-03T17:56:14Z

Very nice, LGTM! 🏁

sstok · 2017-12-03T18:55:13Z

Ah shoot I forgot to remove the WIP commits 😫 that's what I get for working after 7 PM 🤦‍♂️

dkarlovi · 2017-12-05T12:30:35Z

docker-compose.yaml

+        volumes:
+            - esdata:/usr/share/elasticsearch/data
+        ports:
+            - "9200:9200"


Note, this was not prefixed with 5. :( (using a prefix here)

For some reasons this broke the tests 😐 (using a prefix for this port).

I know, tried to change it. 😢 Will submit a PR to fix it once I figure out what it is.

It's over 9200 😂 https://stackoverflow.com/questions/113224/what-is-the-largest-tcp-ip-network-port-number-allowable-for-ipv4

The port number seems to high.

But... 59200 < 65535? That's why I proposed 5 as a prefix, and not 6, for example.

Either specify both ports (HOST:CONTAINER), or just the container port (a random host port will be chosen).

See #200

sstok added DX Enhancement labels Nov 13, 2017

sstok force-pushed the docker-travis branch 3 times, most recently from 34bff62 to 553387d Compare November 13, 2017 10:43

sstok added 2 commits November 13, 2017 11:44

Run tests in isolation (using Docker)

553387d

This makes the tests less dependent on the developer’s system. and FINALLY! Fixes the ICU versioning problem.

Fix existing service conflicts

b147259

dkarlovi reviewed Nov 13, 2017

View reviewed changes

Fix Docker image

b0a131b

sstok force-pushed the docker-travis branch 3 times, most recently from 0438fee to b99ea9c Compare November 14, 2017 12:34

Lets try if coverage works

b99ea9c

dkarlovi mentioned this pull request Nov 16, 2017

Add xdebug jakzal/phpqa#18

Closed

dkarlovi reviewed Nov 17, 2017

View reviewed changes

sstok added 2 commits December 3, 2017 17:03

Taking it further. Note: clean-up afterwards

2c1b98b

Could you kindly work

d838cac

sstok added 3 commits December 3, 2017 19:00

Whoops…

126dc14

Fus ro dah!

84bde4a

And now you will work!

c867f2a

sstok merged commit 8e51b5d into rollerworks:master Dec 3, 2017

sstok deleted the docker-travis branch December 3, 2017 18:52

This was referenced Dec 3, 2017

[Merge] Proofreading pull-request hupkit/hupkit#51

Closed

[Meta] Switch the build process to the newly introduced make-based process #193

Closed

dkarlovi reviewed Dec 5, 2017

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run tests in isolation (using Docker) #194

Run tests in isolation (using Docker) #194

sstok commented Nov 13, 2017 •

edited

sstok commented Nov 13, 2017

dkarlovi Nov 13, 2017

sstok Nov 13, 2017 •

edited

dkarlovi Nov 13, 2017

dkarlovi Nov 17, 2017

dkarlovi Nov 13, 2017

sstok Nov 13, 2017 •

edited

dkarlovi Nov 17, 2017

dkarlovi commented Nov 17, 2017

sstok commented Nov 17, 2017

dkarlovi commented Nov 17, 2017

dkarlovi Nov 17, 2017

dkarlovi Nov 17, 2017

dkarlovi Nov 17, 2017

dkarlovi Nov 17, 2017

dkarlovi Nov 17, 2017

dkarlovi Nov 17, 2017

sstok Nov 23, 2017 •

edited

dkarlovi Nov 23, 2017

sstok commented Nov 25, 2017

sstok commented Dec 3, 2017 •

edited

dkarlovi commented Dec 3, 2017

sstok commented Dec 3, 2017

dkarlovi Dec 5, 2017 •

edited by sstok

sstok Dec 5, 2017 •

edited

dkarlovi Dec 5, 2017

sstok Dec 5, 2017

dkarlovi Dec 5, 2017

dkarlovi Dec 6, 2017


		dist: cs-full phpstan test-full
		dist: install cs-full phpstan test-full

Run tests in isolation (using Docker) #194

Run tests in isolation (using Docker) #194

Conversation

sstok commented Nov 13, 2017 • edited

sstok commented Nov 13, 2017

Choose a reason for hiding this comment

sstok Nov 13, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sstok Nov 13, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dkarlovi commented Nov 17, 2017

sstok commented Nov 17, 2017

dkarlovi commented Nov 17, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sstok Nov 23, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sstok commented Nov 25, 2017

sstok commented Dec 3, 2017 • edited

dkarlovi commented Dec 3, 2017

sstok commented Dec 3, 2017

dkarlovi Dec 5, 2017 • edited by sstok

Choose a reason for hiding this comment

sstok Dec 5, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sstok commented Nov 13, 2017 •

edited

sstok Nov 13, 2017 •

edited

sstok Nov 13, 2017 •

edited

sstok Nov 23, 2017 •

edited

sstok commented Dec 3, 2017 •

edited

dkarlovi Dec 5, 2017 •

edited by sstok

sstok Dec 5, 2017 •

edited