In _prepare_sgl_engines (angelspec/inference/factory.py), Step 2 picks a free port for dist_init_addr starting from the default start_port=10000:
https://github.com/Tencent/AngelSpec/blob/main/angelspec/inference/factory.py#L288-L295
ip, port = ray.get(
[head_engine.get_node_ip.remote(), head_engine.find_free_port.remote()],
timeout=120,
)
find_free_port only probes ports; it never reserves them. Step 2.5 then restarts its sequential allocation at the literal next_start = 10000 without accounting for the port just handed out:
https://github.com/Tencent/AngelSpec/blob/main/angelspec/inference/factory.py#L297-L309
next_start = 10000
for i in range(num_engines):
port = ray.get(engines[i].find_free_port.remote(start_port=next_start, consecutive=2), ...)
pre_allocated_ports[i] = port
next_start = port + 2
On a fresh node, Step 2 returns 10000 and Step 2.5 immediately hands engine 0 (which is the replica-0 head on the same node) ports 10000/10001. SglEngine.init then sets port=10000, nccl_port=10001, and dist_init_addr="<same-ip>:10000" — the head node is told to bind its server port and its dist-init TCPStore on the same port. The Step 2.5 comment says "allocate sequentially so engines on the same node never collide", but the Step 2 allocation is invisible to that sequencer.
The vLLM path avoids this by reusing the head's pre-allocated port as the dist-init port rather than drawing a second untracked port. Possible fixes: allocate the dist-init port from the same sequential allocator (e.g. run Step 2 after Step 2.5 using next_start, or advance next_start past the negotiated dist-init port). Only affects sglang_nnodes > 1 with auto-negotiated addr.
In
_prepare_sgl_engines(angelspec/inference/factory.py), Step 2 picks a free port fordist_init_addrstarting from the defaultstart_port=10000:https://github.com/Tencent/AngelSpec/blob/main/angelspec/inference/factory.py#L288-L295
find_free_portonly probes ports; it never reserves them. Step 2.5 then restarts its sequential allocation at the literalnext_start = 10000without accounting for the port just handed out:https://github.com/Tencent/AngelSpec/blob/main/angelspec/inference/factory.py#L297-L309
On a fresh node, Step 2 returns 10000 and Step 2.5 immediately hands engine 0 (which is the replica-0 head on the same node) ports 10000/10001.
SglEngine.initthen setsport=10000,nccl_port=10001, anddist_init_addr="<same-ip>:10000"— the head node is told to bind its server port and its dist-init TCPStore on the same port. The Step 2.5 comment says "allocate sequentially so engines on the same node never collide", but the Step 2 allocation is invisible to that sequencer.The vLLM path avoids this by reusing the head's pre-allocated port as the dist-init port rather than drawing a second untracked port. Possible fixes: allocate the dist-init port from the same sequential allocator (e.g. run Step 2 after Step 2.5 using
next_start, or advancenext_startpast the negotiated dist-init port). Only affectssglang_nnodes > 1with auto-negotiated addr.