DOC: document how to change ipcontroller-engine.json in case controller was started with --ip="*" #4538

jankatins · 2013-11-14T18:38:16Z

When the controller was started and bound to "*" (as the documentation of the config file says), the ipcontroller-engine.json file will include the following line:

  "interface": "tcp://*", 
  "location": "127.0.0.1"

At least in my case it was not enough to change the location to the controller IP, but I had to include the controller IP via the interface line:

  "interface": "tcp://xx.xx.xx.xx", 
  "location": "xx.xx.xx.xx"

It would both be nice to document how to specify the IP of the controller in the tutorial (http://ipython.org/ipython-doc/dev/parallel/parallel_process.html#starting-the-controller-and-engines-on-different-hosts) and/or change the logic to find the IP to bind to in the engine to take the location argument into account when the interface specifies "*".

The text was updated successfully, but these errors were encountered:

minrk · 2013-11-14T19:49:48Z

Under normal circumstances, location will be sufficient to disambiguate the IP (that's what it's there for). It is only '127.0.0.1' if it can't figure out its own location, in which case you may want to specify the --location=w.x.y.z, which skips the guessing.

What version of IPython are you using, and on what system?

jankatins · 2013-11-14T22:59:14Z

I'm using a git version from yesterday on windows. I think the problem is that "ipconfig" is printing it's output in German but the code is expecting it in English

import IPython.utils.localinterfaces
def _populate_from_list(addr):
    print addr
IPython.utils.localinterfaces._populate_from_list = _populate_from_list

from IPython.utils.localinterfaces import _load_ips_ipconfig, get_output_error_code
_load_ips_ipconfig()
['127.0.0.1']
out, err, rc = get_output_error_code('ipconfig')
print out
Windows-IP-Konfiguration


Drahtlos-LAN-Adapter Drahtlosnetzwerkverbindung 3:

   Medienstatus. . . . . . . . . . . : Medium getrennt
   Verbindungsspezifisches DNS-Suffix: 

Drahtlos-LAN-Adapter Drahtlosnetzwerkverbindung 2:

   Medienstatus. . . . . . . . . . . : Medium getrennt
   Verbindungsspezifisches DNS-Suffix: 

Drahtlos-LAN-Adapter Drahtlosnetzwerkverbindung:

   Verbindungsspezifisches DNS-Suffix: fritz.box
   Verbindungslokale IPv6-Adresse  . : fe80::e1e6:ef28:8414:6965%15
   IPv4-Adresse  . . . . . . . . . . : 192.168.181.26
   Subnetzmaske  . . . . . . . . . . : 255.255.255.0
   Standardgateway . . . . . . . . . : 192.168.181.1

Ethernet-Adapter Bluetooth-Netzwerkverbindung:

   Medienstatus. . . . . . . . . . . : Medium getrennt
   Verbindungsspezifisches DNS-Suffix: 

Ethernet-Adapter LAN-Verbindung:

   Medienstatus. . . . . . . . . . . : Medium getrennt
   Verbindungsspezifisches DNS-Suffix: 

Ethernet-Adapter VirtualBox Host-Only Network:

   Verbindungsspezifisches DNS-Suffix: 
   Verbindungslokale IPv6-Adresse  . : fe80::8519:94b0:31c6:8976%31
   IPv4-Adresse  . . . . . . . . . . : 192.168.56.1
   Subnetzmaske  . . . . . . . . . . : 255.255.255.0
   Standardgateway . . . . . . . . . : 

Tunneladapter isatap.{6C6D4B31-DE6C-4D25-A174-D177DBB8802F}:

   Medienstatus. . . . . . . . . . . : Medium getrennt
   Verbindungsspezifisches DNS-Suffix: 

Tunneladapter Teredo Tunneling Pseudo-Interface:

   Medienstatus. . . . . . . . . . . : Medium getrennt
   Verbindungsspezifisches DNS-Suffix: 

Tunneladapter isatap.{F8E56AA0-96E7-4C2A-A278-DBEF0E3382E4}:

   Medienstatus. . . . . . . . . . . : Medium getrennt
   Verbindungsspezifisches DNS-Suffix: 

Tunneladapter isatap.{B8AA2EFE-2AD8-4F10-89DD-203893621C61}:

   Medienstatus. . . . . . . . . . . : Medium getrennt
   Verbindungsspezifisches DNS-Suffix: 

Tunneladapter isatap.fritz.box:

   Medienstatus. . . . . . . . . . . : Medium getrennt
   Verbindungsspezifisches DNS-Suffix: fritz.box

Tunneladapter isatap.{EDB803A5-FE08-4A3E-A8EC-CF86090064F8}:

   Medienstatus. . . . . . . . . . . : Medium getrennt
   Verbindungsspezifisches DNS-Suffix: 

Tunneladapter isatap.{C763A4C3-0F9E-46E2-8147-1836853C4096}:

   Medienstatus. . . . . . . . . . . : Medium getrennt
   Verbindungsspezifisches DNS-Suffix:

The problematic method _load_ips_ipconfig():

def _load_ips_ipconfig():
    """load ip addresses from `ipconfig` output (Windows)"""
    out, err, rc = get_output_error_code('ipconfig')
    if rc:
        raise IOError("no ipconfig: %s" % err)

    lines = out.splitlines()
    addrs = ['127.0.0.1']
    for line in lines:
        line = line.lower().split()
        if line[:2] == ['ipv4', 'address']: # FAIL...
            addrs.append(line.split()[-1])
    _populate_from_list(addrs)

minrk · 2013-11-14T23:01:06Z

Aha - an incorrect English assumption. I'll see what I can do about fixing that.

jankatins · 2013-11-14T23:05:21Z

http://stackoverflow.com/questions/166506/finding-local-ip-addresses-using-pythons-stdlib

import socket
print([ip for ip in socket.gethostbyname_ex(socket.gethostname())[2] ])
['192.168.181.26', '192.168.56.1']

(The first is my current IP in the local net, the second is the virtualbox adapter)

And here the output when in a Cisco VPN:
['xxx.xx.xxx.xxx', '192.168.181.26', '192.168.56.1']
-> It seems that this does "the right thing" :-)

minrk · 2013-11-14T23:08:47Z

That's precisely what this code used to do, and doesn't anymore because it is unreliable, and when it fails it typically does so with a 30 second hang on DNS resolution.

jankatins · 2013-11-14T23:39:43Z

Not sure if that helps:
from http://stackoverflow.com/questions/5898763/how-do-i-get-the-ip-address-into-a-batch-file-variable

C:\Windows\System32>route print
===========================================================================
Schnittstellenliste
 24...8c 70 5a 9d 8e 21 ......Microsoft Virtual WiFi Miniport Adapter #2
 16...8c 70 5a 9d 8e 21 ......Microsoft Virtual WiFi Miniport Adapter
 15...8c 70 5a 9d 8e 20 ......Intel(R) Centrino(R) Advanced-N 6205
 14...c0 18 85 dd 38 4b ......Bluetooth-Gerät (PAN)
 12...3c 97 0e 0e 5c 96 ......Intel(R) 82579LM Gigabit Network Connection
 31...08 00 27 00 84 5d ......VirtualBox Host-Only Ethernet Adapter
  1...........................Software Loopback Interface 1
 34...00 00 00 00 00 00 00 e0 Microsoft-ISATAP-Adapter
 11...00 00 00 00 00 00 00 e0 Teredo Tunneling Pseudo-Interface
 35...00 00 00 00 00 00 00 e0 Microsoft-ISATAP-Adapter #2
 36...00 00 00 00 00 00 00 e0 Microsoft-ISATAP-Adapter #3
 56...00 00 00 00 00 00 00 e0 Microsoft-ISATAP-Adapter #5
 37...00 00 00 00 00 00 00 e0 Microsoft-ISATAP-Adapter #6
 58...00 00 00 00 00 00 00 e0 Microsoft-ISATAP-Adapter #7
===========================================================================

IPv4-Routentabelle
===========================================================================
Aktive Routen:
     Netzwerkziel    Netzwerkmaske          Gateway    Schnittstelle Metrik
          0.0.0.0          0.0.0.0    192.168.181.1   192.168.181.26     25
        127.0.0.0        255.0.0.0   Auf Verbindung         127.0.0.1    306
        127.0.0.1  255.255.255.255   Auf Verbindung         127.0.0.1    306
  127.255.255.255  255.255.255.255   Auf Verbindung         127.0.0.1    306
     192.168.56.0    255.255.255.0   Auf Verbindung      192.168.56.1    276
     192.168.56.1  255.255.255.255   Auf Verbindung      192.168.56.1    276
   192.168.56.255  255.255.255.255   Auf Verbindung      192.168.56.1    276
    192.168.181.0    255.255.255.0   Auf Verbindung    192.168.181.26    281
   192.168.181.26  255.255.255.255   Auf Verbindung    192.168.181.26    281
  192.168.181.255  255.255.255.255   Auf Verbindung    192.168.181.26    281
        224.0.0.0        240.0.0.0   Auf Verbindung         127.0.0.1    306
        224.0.0.0        240.0.0.0   Auf Verbindung      192.168.56.1    276
        224.0.0.0        240.0.0.0   Auf Verbindung    192.168.181.26    281
  255.255.255.255  255.255.255.255   Auf Verbindung         127.0.0.1    306
  255.255.255.255  255.255.255.255   Auf Verbindung      192.168.56.1    276
  255.255.255.255  255.255.255.255   Auf Verbindung    192.168.181.26    281
===========================================================================
Ständige Routen:
  Netzwerkadresse          Netzmaske  Gatewayadresse  Metrik
          0.0.0.0          0.0.0.0    xxx.xxx.xx.xxx  Standard
===========================================================================

IPv6-Routentabelle
===========================================================================
Aktive Routen:
 If Metrik Netzwerkziel             Gateway
  1    306 ::1/128                  Auf Verbindung
 31    276 fe80::/64                Auf Verbindung
 15    281 fe80::/64                Auf Verbindung
 31    276 fe80::8519:94b0:31c6:8976/128
                                    Auf Verbindung
 15    281 fe80::e1e6:ef28:8414:6965/128
                                    Auf Verbindung
  1    306 ff00::/8                 Auf Verbindung
 31    276 ff00::/8                 Auf Verbindung
 15    281 ff00::/8                 Auf Verbindung
===========================================================================
Ständige Routen:
  Keine

C:\Windows\System32>

-> Get to the 0.0.0.0 entry, take the next lines and make a set from all the 4th entries in each line.

jdfreder · 2013-11-14T23:48:38Z

Maybe it's a bug in socket's call to the native WinAPI. How about calling the API ourselves using PyWin32 or CTypes?

minrk · 2013-11-15T00:00:27Z

No, it's generally caused by an issue in system configuration, not any API - the gethostname / gethostbyname approach involves a DNS lookup. If that fails, it will generally fail by timeout, rather than failing immediately.

jdfreder · 2013-11-15T00:04:05Z

Ah I see. ipconfig may get the addresses from the adapters themselves, maybe a WinAPI call to http://msdn.microsoft.com/en-us/library/aa365915%28v=VS.85%29.aspx and then http://msdn.microsoft.com/en-us/library/ms741516%28VS.85%29.aspx ?

minrk · 2013-11-15T00:30:57Z

should be fixed by #4541.

@jdfreder The API call you reference is actually what is used by netifaces, which is our first priority if it's installed. The ipconfig parsing only happens if netifaces is unavailable.

jankatins · 2013-11-16T18:45:19Z

Please leave this BR open, I will add some doc fixes to document what to change if you need to specify the IP adress of the controller by hand.

minrk · 2013-11-16T23:05:43Z

this is where the location key of the connection file is currently discussed, if you want to add to that in the docs.

minrk mentioned this issue Nov 15, 2013

relax ipconfig matching on Windows #4541

Merged

minrk closed this as completed in #4541 Nov 28, 2013

minrk added this to the 2.0 milestone Mar 26, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DOC: document how to change ipcontroller-engine.json in case controller was started with --ip="*" #4538

DOC: document how to change ipcontroller-engine.json in case controller was started with --ip="*" #4538

jankatins commented Nov 14, 2013

minrk commented Nov 14, 2013

jankatins commented Nov 14, 2013

minrk commented Nov 14, 2013

jankatins commented Nov 14, 2013

minrk commented Nov 14, 2013

jankatins commented Nov 14, 2013

jdfreder commented Nov 14, 2013

minrk commented Nov 15, 2013

jdfreder commented Nov 15, 2013

minrk commented Nov 15, 2013

jankatins commented Nov 16, 2013

minrk commented Nov 16, 2013

DOC: document how to change ipcontroller-engine.json in case controller was started with --ip="*" #4538

DOC: document how to change ipcontroller-engine.json in case controller was started with --ip="*" #4538

Comments

jankatins commented Nov 14, 2013

minrk commented Nov 14, 2013

jankatins commented Nov 14, 2013

minrk commented Nov 14, 2013

jankatins commented Nov 14, 2013

minrk commented Nov 14, 2013

jankatins commented Nov 14, 2013

jdfreder commented Nov 14, 2013

minrk commented Nov 15, 2013

jdfreder commented Nov 15, 2013

minrk commented Nov 15, 2013

jankatins commented Nov 16, 2013

minrk commented Nov 16, 2013