App falling over due to MySQL load time (approx 10seconds)

[ryantaplin]

After looking in the logs of MySQL container it is taking approx 10 seconds to load up.

This is causing the write/read applications to fall over if trying to run DB scripts while this is happening. Need to check that docker handles the fall over and restarts the application; or I need to add something into the main method to handle the DB not being live (e.g. a wait or similar).

/usr/lib/jvm/java-8-openjdk-amd64/bin/java -javaagent:/home/ryan/IntelliJidea/lib/idea_rt.jar=33783:/home/ryan/IntelliJidea/bin -Dfile.encoding=UTF-8 -classpath /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/charsets.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/cldrdata.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/dnsns.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/icedtea-sound.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/jaccess.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/nashorn.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunec.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/zipfs.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jce.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jsse.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/management-agent.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/resources.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar:/home/ryan/IdeaProjects/docker-write-servlet/target/classes:/home/ryan/.m2/repository/org/eclipse/jetty/jetty-server/9.3.8.v20160314/jetty-server-9.3.8.v20160314.jar:/home/ryan/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar:/home/ryan/.m2/repository/org/eclipse/jetty/jetty-http/9.3.8.v20160314/jetty-http-9.3.8.v20160314.jar:/home/ryan/.m2/repository/org/eclipse/jetty/jetty-util/9.3.8.v20160314/jetty-util-9.3.8.v20160314.jar:/home/ryan/.m2/repository/org/eclipse/jetty/jetty-io/9.3.8.v20160314/jetty-io-9.3.8.v20160314.jar:/home/ryan/.m2/repository/org/eclipse/jetty/jetty-servlet/9.3.8.v20160314/jetty-servlet-9.3.8.v20160314.jar:/home/ryan/.m2/repository/org/eclipse/jetty/jetty-security/9.3.8.v20160314/jetty-security-9.3.8.v20160314.jar:/home/ryan/.m2/repository/org/json/json/20170516/json-20170516.jar:/home/ryan/.m2/repository/mysql/mysql-connector-java/5.1.6/mysql-connector-java-5.1.6.jar:/home/ryan/.m2/repository/junit/junit/4.4/junit-4.4.jar:/home/ryan/.m2/repository/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.jar:/home/ryan/.m2/repository/org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.jar:/home/ryan/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/home/ryan/.m2/repository/commons-codec/commons-codec/1.9/commons-codec-1.9.jar:/home/ryan/.m2/repository/org/mybatis/mybatis/3.0.1/mybatis-3.0.1.jar App
Application starting.

-------------------------
Running database scripts
-------------------------
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

Last packet sent to the server was 0 ms ago.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
	at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1074)
	at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2103)
	at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:718)
	at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
	at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:302)
	at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:282)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:247)
	at database.BasicDatabase.<init>(BasicDatabase.java:15)
	at database.BasicDatabaseBuilder.build(BasicDatabaseBuilder.java:8)
	at App.databaseChecksAndSetup(App.java:33)
	at App.main(App.java:19)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at java.net.Socket.connect(Socket.java:538)
	at java.net.Socket.<init>(Socket.java:434)
	at java.net.Socket.<init>(Socket.java:244)
	at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:253)
	at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:280)
	at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2026)
	... 15 more
Accessing database 'jdbc:mysql://192.168.127.128:3306/' as 'root'.
Exception in thread "main" org.apache.ibatis.jdbc.RuntimeSqlException: Could not set AutoCommit to false. Cause: java.lang.NullPointerException
	at org.apache.ibatis.jdbc.ScriptRunner.setAutoCommit(ScriptRunner.java:127)
	at org.apache.ibatis.jdbc.ScriptRunner.runScript(ScriptRunner.java:65)
	at App.databaseChecksAndSetup(App.java:36)
	at App.main(App.java:19)
Caused by: java.lang.NullPointerException
	at org.apache.ibatis.jdbc.ScriptRunner.setAutoCommit(ScriptRunner.java:123)
	... 3 more

Process finished with exit code 1

[ryantaplin]

Docker swarm service supports the restarting of applications when they fall over,

It does hover cause a spam service creation / termination on the host machine for the duration the database is down for. Maybe add in some handling to time it out and retry after 30-60seconds.

I can use the database.status() method to define whether the database is up and functional. Need to investigate whether it will expected result 'available' when fully ready or just when the service starts up and is accessible.

[ryantaplin]

Fixed under commit 2cc0814