--bind-to check prioritized. Use iface address instead of loopback if available.

[amitmurthy]

This PR does the following:

• --bind-to command line option is used to solve the multi-homed node issue. It is is processed before other arguments and LPROC.bind_addr is set the ip-address suggested on the command line (if present), else it is set to whatever is returned by getipaddr().
• detects which of the processes are on the localhost and only uses the loopback address to connect to them - this is resilient across system sleep/wake cycles.
• The machine line in --machinefile file format (as also addprocs(machines::Vector)), is now of the form [user@]host[:port] [bind_addr]. i.e., it supports an optional bind_addr field that is passed as a command line argument to the started workers.

Closes #5995

[jiahao]

On this branch the addprocs command stalls for me:

julia> addprocs(["127.0.0.1", "127.0.0.1"])

ssh: connect to host 127.0.0.1 port 22: Connection refused
ssh: connect to host 127.0.0.1 port 22: Connection refused


^C
Program received signal SIGINT, Interrupt.
0x000000010012d4c9 in add_page (p=0x100cfc200) at gc.c:460
460         v = (gcval_t*)((char*)v + p->osize);

[amitmurthy]

you need to be running the ssh daemon on localhost since you are using the ssh mode of addprocs to launch workers on localhost.

[amitmurthy]

gdb also does not print a stack

julia> addprocs(["127.0.0.1", "127.0.0.1"])
[New Thread 0x7fffeb137700 (LWP 6159)]
...
[New Thread 0x7fffe9934700 (LWP 6162)]
2-element Array{Any,1}:
 2
 3

julia> exit(1)
[Thread 0x7fffe9934700 (LWP 6162) exited]
...
[Thread 0x7fffdce3c700 (LWP 6054) exited]
[Inferior 1 (process 6045) exited with code 01]
(gdb) Segmentation fault

(gdb) where
No stack.
(gdb) 

[jiahao]

Oh right, duh.

[amitmurthy]

Uncommented the above since it does not seem to be a problem now.

[amitmurthy]

@JeffBezanson - please have a look.

[amitmurthy]

Updated to ensure that check_same_host does not use ip-addresses for checking localhost workers. Addresses issues caused by a system sleep/awaken cycle.

[bjarthur]

this PR fixes my problems doing an addprocs() followed by an addprocs_sge() on a cluster whose nodes have two NICs. thanks amit. i tried merging into this branch the latest commits from master, and got a couple conflicts...

[amitmurthy]

Good. I'll rebase once @JeffBezanson takes a look and is OK with this approach.

[JeffBezanson]

Looks good.