Lost connection to MySQL server during query

[FreakyFreddie]

Environment:

• CTFd Version/Commit: Latest as of 11/11/2017
• Operating System: Ubuntu 16.04 LTS, deploying platform in Docker containers, using docker-compose up
• Web Browser and Version: Firefox: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0 and Chrome: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36

What happened?
When adding a new challenge, I took some time to fill in the form. When sending it, I received an internal server error. I tried it again and it worked normally.

What did you expect to happen?
The challenge to be created normally.

How to reproduce your issue
Click the button to create a new challenge, fill in the form. Then, leave the page open for around 15 minutes and click the button.

Any associated stack traces or error logs
Error.log

[2017-11-13 08:28:13,623] ERROR in app: Exception on /admin/chal/new [POST]

File "/usr/local/lib/python2.7/site-packages/pymysql/connections.py", line 1033, in _read_bytes
    CR.CR_SERVER_LOST, "Lost connection to MySQL server during query")
OperationalError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT config.id AS config_id, config.`key` AS config_key, config.value AS config_value \nFROM config \nWHERE config.`key` = %(key_1)s \n LIMIT %(param_1)s'] [parameters: {u'param_1': 1, u'key_1': 'setup'}]

Access.log

10.0.4.2 - - [13/Nov/2017:08:28:13 +0000] "POST /admin/chal/new HTTP/1.1" 500 2700 "https://10.0.7.4/admin/chal/new" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"

The following errors may also be related to the same issue:
Error.log

[2017-11-13 09:12:36,352] ERROR in app: Exception on /challenges [GET]
File "/usr/local/lib/python2.7/site-packages/pymysql/connections.py", line 1033, in _read_bytes
    CR.CR_SERVER_LOST, "Lost connection to MySQL server during query")
OperationalError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT config.id AS config_id, config.`key` AS config_key, config.value AS config_value \nFROM config \nWHERE config.`key` = %(key_1)s \n LIMIT %(param_1)s'] [parameters: {u'key_1': 'setup', u'param_1': 1}]

Access.log

10.0.4.2 - - [13/Nov/2017:09:12:36 +0000] "GET /challenges HTTP/1.1" 500 2763 "https://10.0.7.4/challenges" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"

Error.log

[2017-11-13 11:55:04,870] ERROR in app: Exception on /admin/chals [GET]
File "/usr/local/lib/python2.7/site-packages/pymysql/connections.py", line 1033, in _read_bytes
    CR.CR_SERVER_LOST, "Lost connection to MySQL server during query")
OperationalError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT config.id AS config_id, config.`key` AS config_key, config.value AS config_value \nFROM config \nWHERE config.`key` = %(key_1)s \n LIMIT %(param_1)s'] [parameters: {u'param_1': 1, u'key_1': 'setup'}]

Access.log

10.0.4.2 - - [13/Nov/2017:11:55:04 +0000] "GET /admin/chals HTTP/1.1" 500 2763 "https://10.0.7.4/admin/teams" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"

access.log
error.log

[ColdHeat]

Is this consistently reproducible for you? I doubt that this is an issue with CTFd considering that the error message is usually related to network issues.

Another thing to note is that the cache clears out about every 5 minutes and given that CTFd wasn't being run for 15 minutes perhaps you entered some situation where the database was getting hammered due to too many configuration queries? Just a random hypothesis.

[FreakyFreddie]

Thank you for your response! I doubt that the database was under stress at the time of error. I also don't know if it can be a network issue, since the database is connected to the application through the local Docker network (if I understand correctly).

Do you have any more information on the cache part? Would it for example help if I configured a redis server to help with Flask caching?

I will test the platform again next monday and will report if the problem reoccurs. I will be monitoring the database more closely.

[ColdHeat]

CTFd uses caches to store database queries so that it doesn't need to constantly look up whether or not it's setup for example. By default CTFd will use an in memory dictionary to cache data so you can get away with not using redis.