Add pidfile support for TiDB

[morgo]

I checked out how this is implemented in MySQL, and it is not a feature of mysqld, but rather mysqld_safe. TiDB does not have a comparable to mysqld_safe, and since many modern oses use something like systemd for this instead, it does not make sense to implement one (it has been a source of many security vulnerabilities too).

So instead I modified the start script to generate a pid file, and modified the stop and status scripts to use it. It seems to work fine (only tested on Linux, but I tried not to use any specific features).

The stop and status scripts check for the pid file, in addition to the start script checking for stale pid files.

[datacharmer]

The goal is to have the a consistent output from the sandbox scripts, as some of the text returned by script execution is evaluated in the functional tests.

• status should return "on" and "off"
• start should check for the server health and write the result accordingly. Currently, it exits with zero regardless. Could we perhaps check for the socket existence in a loop, since the PID is being generated in the script? If we do that, we can remove the fixed sleep instruction.
• For consistency, start should not advertise "starting" but say what happened afterwards ("sandbox server started" or "sandbox server not started yet".

BTW, why do master downloads return different versions for MacOS and Linux? (#62 (comment)) I get 2.1.0 on MacOS and 3.0.0 on Linux.

[morgo]

BTW, why do master downloads return different versions for MacOS and Linux? (#62 (comment)) I get 2.1.0 on MacOS and 3.0.0 on Linux.

I suspect that MacOS builds might not be regenerating daily. Master should not be returning 3.0.0. I will ask the build team to look into it, but in my mind, we should only be targeting 3.0+ since it has support for tcp+unix socket. It will also now advertise socket/port/datadir correctly.

Thank you for the feedback, I will incorporate.

[morgo]

I've updated some temporary packages to:
https://s3.us-east-2.amazonaws.com/files.tocker.ca/tidb-master-darwin-amd64.tar.gz
https://s3.us-east-2.amazonaws.com/files.tocker.ca/tidb-master-linux-amd64.tar.gz

I've verified they work on Mac OS and Linux, but they were manually created.. so no guarantees.

I have incorporated the changes requested in your review. In addition, the use and status commands now use the base templates.

[datacharmer]

Thanks for the update.
The good news is that test_sb passes. The bad ones is that the new start script is now mock-unfriendly.
Could you suggest a new template (similar to mysqld_safe_mock_template) that can emulate tidb-server so that the start script passes?

[morgo]

Sure, I'll take a look.

At quick glance it looks like the mock template writes a pid file (which in the case of TiDB the start script does), so conceptually it only needs to check that a -config argument was passed. i.e.:

#!/bin/sh
# mock-tidb-server

options=$1

# check for -config

if echo $options | grep -F 'config' > /dev/null 2>&1
then
 echo "contains config"
 exit 0
else
 echo "no config!"
 exit 1
fi

Do I need to add a sleep in there so it is a "running server"? I assume I don't need to patch start to accept a custom tidb-server, but rather a custom package is generated and unpacked?

[datacharmer]

Correct. We should not change start to pass a mock test.
However, a no-op mock-script for tidb-server would not be enough, since start waits for a socket to appear.

The mock server is generated with a procedure similar to what we do in set-mock.sh.

The test that is failing is in sandbox_test.go

Run as

cd sandboxes
go test -v -run TestCreateSandbox/mocktidb .

[datacharmer]

I can make the whole thing work (adding a new mock template for tidb-server that generates the right "almost" socket file) but only if we change the restrictive -S in start to a more relaxed -e.
Suggestions?

[datacharmer]

Suggested change

	if [ -S {{.GlobalTmpDir}}/mysql_sandbox{{.Port}}.sock ]
	if [ -e {{.GlobalTmpDir}}/mysql_sandbox{{.Port}}.sock ]

See above comment