task 分布在不同 agent，mysql.extractor: publish timeout, got nats: timeout

[Near-Zhang]

src Task 和 dest Task 分别分配在两个不同的 agent，agent 之间通过外网可以互相 telnet 通 8193 端口，但是一直报错 nats 相关

agent 配置

data_dir = "/usr/local/dtle/data"

bind_addr = "0.0.0.0"

log_level = "DEBUG"

log_file = "/usr/local/dtle/logs/dtle.log"

# Modify our port to avoid a collision with server
ports {
    http = 8190
    rpc = 8191
    serf = 8192
    nats = 8193
}

# Enable the manager
manager {
    enabled = false
    bootstrap_expect = 3
    # Self-elect, should be 3 or 5 for production,
    # Addresses to attempt to join when the server starts.
    #join = ["10.29.67.236"]
}

# Enable the agent
agent {
    enabled = true
    managers = ["119.23.151.150", "47.100.167.118", "47.102.196.3"]
}

advertise {
    http = "47.102.218.202"
    rpc = "47.102.218.202"
    serf = "47.102.218.202"
    nats = "0.0.0.0"
}

错误日志

2019/05/29 11:04:31.263755 [DEBUG] [jobid:baa8a459-0839-11ad-fafa-7efea66b6577] mysql.extractor: publish timeout, got nats: timeout
2019/05/29 11:04:31.263829 [DEBUG] [jobid:baa8a459-0839-11ad-fafa-7efea66b6577] mysql.extractor: publish. gtid: , msg_len: 898

[baoming]

我也有此问题，nats地址配置了public ip，启动会失败；不配置外网ip，另外一个agent又连不上。麻烦官方给个使用指导。

[yaqigithub]

dtle的通讯需要在一个网段下，agent 之间要保证有一个公共网络。dtle.conf的配置与源端目标端的隔离场景无关。
dtle.conf配置如：
first agent

# Setup data dir
data_dir = "/opt/dtle/data"
log_level = "DEBUG"
log_file = "/opt/dtle.log"
#log_to_stdout = true

ui=true
ui_dir="/opt/frontend/"

bind_addr = "172.100.9.4"

# Modify our port to avoid a collision with server
ports {
    http = 8190
}

# Enable the manager
manager {
    enabled = true

    # Self-elect, should be 3 or 5 for production,
    # Addresses to attempt to join when the server starts.
    join = [ "172.100.9.4","172.100.9.5","172.100.9.6" ]
}

# Enable the agent
agent {
    enabled = true
    managers = ["172.100.9.4:8191","172.100.9.5:8191","172.100.9.6:8191"]
}

metric{
   collection_interval = "15s"
   publish_allocation_metrics = "true"
   publish_node_metrics = "true"
}

addresses {
http = "172.100.9.4"
rpc = "172.100.9.4"
serf = "172.100.9.4"
}
advertise {
http = "172.100.9.4"
rpc = "172.100.9.4"
serf = "172.100.9.4"
}

second agent

# Setup data dir
data_dir = "/opt/dtle/data"
log_level = "DEBUG"
log_file = "/opt/dtle.log"
#log_to_stdout = true

ui=true
ui_dir="/opt/frontend/"

bind_addr = "172.100.9.5"

# Modify our port to avoid a collision with server
ports {
    http = 8190
}

# Enable the manager
manager {
    enabled = true

    # Self-elect, should be 3 or 5 for production,
    # Addresses to attempt to join when the server starts.
    join = [ "172.100.9.4","172.100.9.5","172.100.9.6" ]
}

# Enable the agent
agent {
    enabled = true
    managers = ["172.100.9.4:8191","172.100.9.5:8191","172.100.9.6:8191"]
}

metric{
   collection_interval = "15s"
   publish_allocation_metrics = "true"
   publish_node_metrics = "true"
}

addresses {
http = "172.100.9.5"
rpc = "172.100.9.5"
serf = "172.100.9.5"
}
advertise {
http = "172.100.9.5"
rpc = "172.100.9.5"
serf = "172.100.9.5"
}

大体关系入下图：

[baoming]

@yaqigithub @Near-Zhang 在云化环境、容器环境时binding IP和通讯IP不一样，advertise ip无法启动nats server. 我提交了个Patch，目前工作正常，能满足容器和云化虚拟机跨机房通讯场景。

bind_addr = "0.0.0.0"

advertise {
http = "100.x.x.x"
rpc = "100.x.x.x"
serf = "100.x.x.x"
nats = "100.x.x.x" 
}

@baoming “容器环境时binding IP和通讯IP不一样” 是说bind是内网地址, 经过路由器通向外网?

想来之前 #367 2d15837 中把AdvertiseAddrs作为Nats(Bind)Addr是错误做法, 忘了外网地址不一定能Bind.

具体等我再想一下...