remote worker missing

[gaos1]

We have a concourse TSA in AWS US and a remote worker in AWS China.
We often see the following errors in this worker's logs:
worker for container is missing
Our worker is indeed missing when we executed below commands to check:
fly ws

Here are some related logs in worker job groundcrew:
-- beacon.stderr.log

Write failed: Broken pipe
Write failed: Broken pipe
Write failed: Broken pipe
Connection to concourse.xxx.xxx closed by remote host.

-- beacon.stdout.log

2016/01/07 06:46:00 heartbeat took 2.574034443s
2016/01/07 06:46:31 heartbeat took 1.242054249s
2016/01/07 06:47:03 heartbeat took 1.338272014s
2016/01/07 06:47:34 heartbeat took 1.306922357s
2016/01/07 06:48:05 heartbeat took 756.882852ms
2016/01/07 06:48:36 heartbeat took 791.133605ms
2016/01/07 06:49:06 heartbeat took 847.64856ms
2016/01/07 06:49:37 heartbeat took 786.748922ms
2016/01/07 06:50:13 heartbeat took 5.626379388s

[concourse-bot]

Hi there!

We use Pivotal Tracker to provide visibility into what our team is working on. A story for this issue has been automatically created.

The current status is as follows:

• #111930141 remote worker missing

This comment, as well as the labels on the issue, will be automatically updated as the status in Tracker changes.

[gaos1]

per your suggestion, we set tsa.heartbeat_interval to 60s and the problem still keeps happening

[xoebus]

Does the connection between the two machines drop for more than 60 seconds at a time?

[vito]

There's honestly not a lot we can do about this. If there's a bad enough connection between your workers and the rest of Concourse they'll occasionally go away and come back. The TTLs are fairly generous out of the box; if the worker has become unavailable, it's likely that even if it were still registered you'd just see network failures trying to reach it.

[concourse-bot]

Hello again!

All stories related to this issue have been accepted, so I'm going to automatically close this issue.

At the time of writing, the following stories have been accepted:

• #111930141 remote worker missing

If you feel there is still more to be done, or if you have any questions, leave a comment and we'll reopen if necessary!