Server fails to start if there is a .pid file left over

[jackzampolin]

I'm running into this when running in docker. If I stop the container and then restart it sometimes the blockstack-server.pid file was not cleaned up. The reason I run into this on docker is because the data that needs to be persisted is in the same directory as the .pid file. Moving it out of the $HOME/.blockstack-server directory would fix this issue.

[jcnelson]

Yes; it is expected that you run blockstack-core stop in the event of an unclean shutdown. It will clean up any runtime state.

[jackzampolin]

This is for the container use case. It's not possible to do that. Could we make blockstack-core detect unclean state on start and clear it out?

If there is a server running already, the second one will fail to start due to port conflicts.

[jcnelson]

This should be fixed in master.

[jackzampolin]

I'm still seeing this when using blockstack-core in docker. To reproduce this, I've got a docker-compose file that spins up 4 blockstack-core nodes. I've been using --fast_sync to quickly sync the nodes.

After the --fast_sync, when running docker-compose up (create all 4 nodes) the nodes come up cleanly and can be seen processing remaining blocks in the logs.

After stopping the nodes with docker-compose down (SIGTERM then a SIGKILL after 10s) some of the nodes have .pid files left, but not all, leaving me to think that possibly SIGTERM is handled correctly while nodes that are SIGKILLed are not properly removing the .pid file. The error produced is below:

 [2018-01-16 18:07:40,878] [ERROR] [blockstackd:3097] (1.139761606788864) Blockstackd appears to be running already.  If not, please run '/usr/local/bin/blockstack-core stop'

[jcnelson]

The code to check if the Blockstack version in question is running will kill(2) the PID listed in the PID file with signal 0. If kill(2) succeeds, then another server is assumed to be running, leading to this error.

This behavior may be a consequence of running Blockstack in Docker. Within the Docker container, it could be the case that Blockstack always runs as PID 1. If this is the case, then checking the PID file wouldn't tell us anything, since the PID is the same across invocations. Can you confirm if this is the case?

[jackzampolin]

The process that is being run with the CMD directive is always PID 1 in a docker container, so this is definitely the case.

[jcnelson]

So, the PID file always has 1 written in it?

[jackzampolin]

Yup!

[jcnelson]

This is possibly fixed in feature/clean-chainstate

[jcnelson]

Merged to master. Please reopen if problem persists.