restore task after dm-workers restart

[IANTHEREAL]

What problem does this PR solve?

• refer issue task and subtask would be restarted automatically after we restart dm-worker #77
• refine encrypted password, dm only decrypt password
  • start a task
  • start relay/check unit
  • update worker config

What is changed and how it works?

store subtask in local disk of dm-worker.

In all request start/stop/update task, dm-worker would change meta firstly, then handle the request, we should explain it in document.

And I would modify ansible and add integrate test later

Check List

Tests

• Unit test
• integration test
• Manual test
  • deploy a dm cluster
  • start a task
  • check dm worker meta file content
  • restart dm-worker
  • check whether task is running
  • add another task
  • check dm worker meta file content
  • stop one task
  • check dm worker meta file content
  • update task
  • check dm worker meta file content

[amyangfei]

better return errors.Trace(err)

[amyangfei]

It seems restart_dm_worker has quite similar logic to run_dm_worker script, is there any purpose to add this? In addition, the content of coverage file $TEST_DIR/cov.$TEST_NAME.worker.$port.out will be truncated here, we'd better to use append stream to this file(>>) or choose another file name.

[IANTHEREAL]

remove mkdir and ln opeations, maybe we also add other operation in it later, how do you think about it?

[amyangfei]

mkdir -p is reentrant, and we can also update ln to [ -f $workdir/bin/mydumper ] || ln -s $PWD/bin/mydumper $workdir/bin/mydumper to make it reentrant

[IANTHEREAL]

@csuzhangxc @amyangfei PTAL

[csuzhangxc]

LGTM

[amyangfei]

LGTM