Create necessary docker containers intended for unit testing

[jeff1evesque]

We will create the necessary containers required to replicate our application, so it can be unit tested. Once completed, #2628, will link the containers, and perform the basic unit tests.

Note: this issue has been adjusted from its original intention to build all necessary containers, and perform the corresponding unit testing. Now it has been segregated into two different issues.

[jeff1evesque]

Our Travis build indicates that our flask service is not running. Specifically, corresponding puppet logic required to create our upstart script to start our flask service, has not run.

[jeff1evesque]

This issue should be resolved after our vagrant implementation has been replaced with docker (#2087). The reason for this, is because several of the puppet manifests reference /vagrant/... directory structure. However, the travis-docker container, implements docker, instead of vagrant. Therefore, the directory structure will change, respectively.

[jeff1evesque]

This should be resolved immediately, since implementing an automated unit test (per commit) will save time in the long run. The following examples may provide additional insight:

• https://github.com/halberom/ansible_role_iptables/blob/master/.travis.yml
• https://github.com/halberom/ansible_role_iptables/blob/master/tests/Dockerfile.ubuntu1404

Note: the above example, the author uses ansible to execute commands within the docker container. However, we can easily use docker run command, with various test packages installed, with necessary code mounted within the container.

[jeff1evesque]

Necessary attributes during mysql installation is not recognized, and may lead to the larger issue of not having a functional docker container running during our unit testing. Specifically, it seems our hiera attributes is not loading, within setup_tables.py.

[jeff1evesque]

Earlier, implemented hiera attributes from setup_tables.py, was written out to db-trace.txt:

...
    with open("/var/machine-learning/db-trace.txt", "w") as text_file:
        text_file.write("host: %s" % host)
        text_file.write("provisioner: %s" % provisioner)
        text_file.write("provisioner password: %s" % provisioner_password)
        text_file.write("database: %s" % db_ml)
...

So, we cat the values out of db-trace.txt, after the puppet logic containing the above python snippet, had run, within a docker container:

The command "docker run --name database container-database" exited with 0.
0.01s$ cat /var/machine-learning/db-trace.txt
cat: /var/machine-learning/db-trace.txt: No such file or directory

However, the file doesn't seem to exist. So, we need to check the following:

• can we use with open()
• check if argv[1] is not null

[jeff1evesque]

The Max retries exceeded with url error(s) may be related to not listening to the right ports.

[jeff1evesque]

We need to determine the following:

• how to echo all existing processes in each build
  • database docker build, should have mysqld running
  • redis docker build, should have redis process running
  • webserver docker build, should have webserver process running
• determine if corresponding process is running in each run
  • after database docker build, check whether the mysqld process, is active in the corresponding docker run
  • after redis docker build, check whether the redis process, is active in corresponding docker run
  • after webserver docker build, check whether the webserver process, is active in corresponding docker run
• determine if we need special logic to supplement the link between build, or run with another container type
  • link database with webserver, or link redis with webserver

[jeff1evesque]

To simplify this issue, and reduce time, we will debug locally, outside the scope of our Travis CI. This means we will need to install docker locally, and reimplement the docker commands, currently located in our travis.yml.

[jeff1evesque]

This issue has become too long. Therefore, we will comment out the last line within the .travis.yml, since this issue has satisfied the creation of segregated containers. Specifically, it will be continued via #2628.