Cannot access docker when running VPN (Cisco AnyConnect) #628

Open
mtscout6 opened this Issue Nov 13, 2014 · 61 comments

Comments

Projects
None yet
@mtscout6

I could not get boot2docker to work while running the Cisco AnyConnect VPN client. I did not record the console output when I encountered the error, when I see it again then I will post it.

In my efforts to fix it I found a solution by @frosenberg in his blog post: http://www.devopslife.com/2014/08/08/docker-boot2docker-and-dns-resolution-of-containers.html

I ran his enable-docker-dns.sh script which failed to work with docker 1.3.0. I modified the port 2375 to 2376 which also failed to work. This rendered my boot2docker vm unreachable.

To fix the problem I tried running:

➜  docker-dns-scripts git:(master) ✗ boot2docker destroy
➜  docker-dns-scripts git:(master) ✗ boot2docker init
➜  docker-dns-scripts git:(master) ✗ boot2docker up
Waiting for VM and Docker daemon to start...
.docker@localhost's password:
➜  docker-dns-scripts git:(master) ✗ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
➜  docker-dns-scripts git:(master) ✗ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
ubuntu              latest              5506de2b643b        2 weeks ago         199.3 MB
➜  docker-dns-scripts git:(master) ✗ boot2docker ssh
docker@localhost's password:
➜  docker-dns-scripts git:(master) ✗ rm ~/.ssh/id_boot2docker *
zsh: sure you want to delete all the files in /Users/smithm/dev/docker-dns-scripts [yn]? n
➜  docker-dns-scripts git:(master) ✗ rm ~/.ssh/id_boot2docker*
➜  docker-dns-scripts git:(master) ✗ boot2docker destroy
➜  docker-dns-scripts git:(master) ✗ boot2docker init
Generating public/private rsa key pair.
Your identification has been saved in /Users/smithm/.ssh/id_boot2docker.
Your public key has been saved in /Users/smithm/.ssh/id_boot2docker.pub.
The key fingerprint is:
9a:30:44:b5:0c:c3:89:db:82:bc:eb:f3:6a:73:f3:80 smithm@sll-macc02lw491
The key's randomart image is:
+--[ RSA 2048]----+
|   o+o.          |
|  ..o+ .         |
|.. o. o          |
|..o..            |
|  ..o   S        |
| ..  o o         |
| E..  o          |
| = o.            |
|oo*.o.           |
+-----------------+
➜  docker-dns-scripts git:(master) ✗ boot2docker up
Waiting for VM and Docker daemon to start...
.docker@localhost's password:

I tried the password tcuser as documented at https://docs.docker.com/installation/mac/ but that does not work. Any thoughts on what I need to do to get boot2docker working correctly again?

Also, I would have expected the images I had downloaded to be blown away with boot2docker destroy. I checked and the ~/VirtualBox\ VMs/boot2docker-vm/ directory is removed along with the boot2docker-vm.vmdk disk image. Why are those still hanging around?

@mtscout6

This comment has been minimized.

Show comment
Hide comment
@mtscout6

mtscout6 Nov 13, 2014

I don't know if this helps but running boot2docker -v up:

➜  docker-dns-scripts git:(master) ✗ boot2docker -v up
Boot2Docker-cli version: v1.3.0
Git commit: deafc19
2014/11/13 10:11:08 executing: VBoxManage showvminfo boot2docker-vm --machinereadable
2014/11/13 10:11:08 executing: VBoxManage guestproperty set boot2docker-vm /VirtualBox/GuestAdd/SharedFolders/MountPrefix /
2014/11/13 10:11:08 executing: VBoxManage guestproperty set boot2docker-vm /VirtualBox/GuestAdd/SharedFolders/MountDir /
2014/11/13 10:11:08 executing: VBoxManage sharedfolder add boot2docker-vm --name Users --hostpath /Users --automount
VBoxManage: error: Shared folder named 'Users' already exists
VBoxManage: error: Details: code VBOX_E_OBJECT_IN_USE (0x80bb000c), component SessionMachine, interface IMachine, callee nsISupports
VBoxManage: error: Context: "CreateSharedFolder(Bstr(name).raw(), Bstr(hostpath).raw(), fWritable, fAutoMount)" at line 1009 of file VBoxManageMisc.cpp
2014/11/13 10:11:08 executing: VBoxManage setextradata boot2docker-vm VBoxInternal2/SharedFoldersEnableSymlinksCreate/Users 1
2014/11/13 10:11:08 executing: VBoxManage startvm boot2docker-vm --type headless
Waiting for VM "boot2docker-vm" to power on...
VM "boot2docker-vm" has been successfully started.
2014/11/13 10:11:08 executing: VBoxManage showvminfo boot2docker-vm --machinereadable
Waiting for VM and Docker daemon to start...
.Connecting to tcp://localhost:2022 (attempt #0)2014/11/13 10:11:08 executing: /usr/bin/ssh ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -p 2022 -i /Users/smithm/.ssh/id_boot2docker docker@localhost ip addr show dev eth1
docker@localhost's password:

I don't know if this helps but running boot2docker -v up:

➜  docker-dns-scripts git:(master) ✗ boot2docker -v up
Boot2Docker-cli version: v1.3.0
Git commit: deafc19
2014/11/13 10:11:08 executing: VBoxManage showvminfo boot2docker-vm --machinereadable
2014/11/13 10:11:08 executing: VBoxManage guestproperty set boot2docker-vm /VirtualBox/GuestAdd/SharedFolders/MountPrefix /
2014/11/13 10:11:08 executing: VBoxManage guestproperty set boot2docker-vm /VirtualBox/GuestAdd/SharedFolders/MountDir /
2014/11/13 10:11:08 executing: VBoxManage sharedfolder add boot2docker-vm --name Users --hostpath /Users --automount
VBoxManage: error: Shared folder named 'Users' already exists
VBoxManage: error: Details: code VBOX_E_OBJECT_IN_USE (0x80bb000c), component SessionMachine, interface IMachine, callee nsISupports
VBoxManage: error: Context: "CreateSharedFolder(Bstr(name).raw(), Bstr(hostpath).raw(), fWritable, fAutoMount)" at line 1009 of file VBoxManageMisc.cpp
2014/11/13 10:11:08 executing: VBoxManage setextradata boot2docker-vm VBoxInternal2/SharedFoldersEnableSymlinksCreate/Users 1
2014/11/13 10:11:08 executing: VBoxManage startvm boot2docker-vm --type headless
Waiting for VM "boot2docker-vm" to power on...
VM "boot2docker-vm" has been successfully started.
2014/11/13 10:11:08 executing: VBoxManage showvminfo boot2docker-vm --machinereadable
Waiting for VM and Docker daemon to start...
.Connecting to tcp://localhost:2022 (attempt #0)2014/11/13 10:11:08 executing: /usr/bin/ssh ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -p 2022 -i /Users/smithm/.ssh/id_boot2docker docker@localhost ip addr show dev eth1
docker@localhost's password:
@frosenberg

This comment has been minimized.

Show comment
Hide comment
@frosenberg

frosenberg Nov 13, 2014

Sorry to hear my stuff blows up things. I learned today also that my scripts don't work with docker 1.3.x as security has been added. There may also be a change that my crude way of adding a host-only adapter may to VirtualBox may break things for you.

For the VPN stuff, I run this script (pre-docker 1.3.x but I presume it still works):
https://github.com/frosenberg/docker-dns-scripts/blob/master/vpn-fix.sh
It will remove some firewall stuff that CiscoVPN is setting up. You may wanna give this a try.

Sorry to hear my stuff blows up things. I learned today also that my scripts don't work with docker 1.3.x as security has been added. There may also be a change that my crude way of adding a host-only adapter may to VirtualBox may break things for you.

For the VPN stuff, I run this script (pre-docker 1.3.x but I presume it still works):
https://github.com/frosenberg/docker-dns-scripts/blob/master/vpn-fix.sh
It will remove some firewall stuff that CiscoVPN is setting up. You may wanna give this a try.

@mtscout6

This comment has been minimized.

Show comment
Hide comment
@mtscout6

mtscout6 Nov 13, 2014

@frosenberg I don't blame you, I had a feeling it wouldn't work with 1.3.0 but gave it a try anyways. I figured I'd be able to undo it all anyway, but I didn't count on weird issues with destroying my boot2docker image and recreating it.

The only cleanup I've done on my Mac was to remove the route table entry you add. I ran sudo -i route delete 172.12.0.0/16 172.16.0.11 I assume that's all that I needed to do to remove that change.

@frosenberg I don't blame you, I had a feeling it wouldn't work with 1.3.0 but gave it a try anyways. I figured I'd be able to undo it all anyway, but I didn't count on weird issues with destroying my boot2docker image and recreating it.

The only cleanup I've done on my Mac was to remove the route table entry you add. I ran sudo -i route delete 172.12.0.0/16 172.16.0.11 I assume that's all that I needed to do to remove that change.

@mtscout6

This comment has been minimized.

Show comment
Hide comment
@mtscout6

mtscout6 Nov 13, 2014

Ok, I ran the uninstall script then re-ran the mac installer. Now, I'm getting this:

➜  ~  docker ps
2014/11/13 11:52:04 Get https://192.168.59.103:2376/v1.15/containers/json: dial tcp 192.168.59.103:2376: i/o timeout
➜  ~  boot2docker ssh
                        ##        .
                  ## ## ##       ==
               ## ## ## ##      ===
           /""""""""""""""""\___/ ===
      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
           \______ o          __/
             \    \        __/
              \____\______/
 _                 _   ____     _            _
| |__   ___   ___ | |_|___ \ __| | ___   ___| | _____ _ __
| '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__|
| |_) | (_) | (_) | |_ / __/ (_| | (_) | (__|   <  __/ |
|_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_|
boot2docker: 1.3.0
             master : a083df4 - Thu Oct 16 17:05:03 UTC 2014
docker@boot2docker:~$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
docker@boot2docker:~$

Any thoughts on why I'm getting the timeouts while running docker commands from my mac terminal?

Ok, I ran the uninstall script then re-ran the mac installer. Now, I'm getting this:

➜  ~  docker ps
2014/11/13 11:52:04 Get https://192.168.59.103:2376/v1.15/containers/json: dial tcp 192.168.59.103:2376: i/o timeout
➜  ~  boot2docker ssh
                        ##        .
                  ## ## ##       ==
               ## ## ## ##      ===
           /""""""""""""""""\___/ ===
      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
           \______ o          __/
             \    \        __/
              \____\______/
 _                 _   ____     _            _
| |__   ___   ___ | |_|___ \ __| | ___   ___| | _____ _ __
| '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__|
| |_) | (_) | (_) | |_ / __/ (_| | (_) | (__|   <  __/ |
|_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_|
boot2docker: 1.3.0
             master : a083df4 - Thu Oct 16 17:05:03 UTC 2014
docker@boot2docker:~$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
docker@boot2docker:~$

Any thoughts on why I'm getting the timeouts while running docker commands from my mac terminal?

@mtscout6

This comment has been minimized.

Show comment
Hide comment
@mtscout6

mtscout6 Nov 13, 2014

Issue #392 solved my problem. Looks like the host only network was messed up. This comment's instructions fixed things. I'll have to give @frosenberg 's vpn-fix.sh shell script alone later tonight when I'm at home again.

Issue #392 solved my problem. Looks like the host only network was messed up. This comment's instructions fixed things. I'll have to give @frosenberg 's vpn-fix.sh shell script alone later tonight when I'm at home again.

@johnnyt

This comment has been minimized.

Show comment
Hide comment
@johnnyt

johnnyt Jan 6, 2015

#392 (comment) had the steps that finally worked for me: adding a port forward and then pointing to 127.0.0.1:

boot2docker down
vboxmanage modifyvm "boot2docker-vm" --natpf1 "docker,tcp,127.0.0.1,2376,,2376"
boot2docker up
export DOCKER_HOST=tcp://127.0.0.1:2376

johnnyt commented Jan 6, 2015

#392 (comment) had the steps that finally worked for me: adding a port forward and then pointing to 127.0.0.1:

boot2docker down
vboxmanage modifyvm "boot2docker-vm" --natpf1 "docker,tcp,127.0.0.1,2376,,2376"
boot2docker up
export DOCKER_HOST=tcp://127.0.0.1:2376
@dleute

This comment has been minimized.

Show comment
Hide comment
@dleute

dleute Apr 23, 2015

This fix worked for me running the cisco vpn. I'm also experimenting with kitematic which is awesome. But still needs some work if you use a lot of parameters to run docker containers. For simpler uses, it's fantastic.

dleute commented Apr 23, 2015

This fix worked for me running the cisco vpn. I'm also experimenting with kitematic which is awesome. But still needs some work if you use a lot of parameters to run docker containers. For simpler uses, it's fantastic.

@ryanleary

This comment has been minimized.

Show comment
Hide comment
@ryanleary

ryanleary Jun 2, 2015

Cisco AnyConnect messes with your machine's routes. I've done this as a workaround:
$(boot2docker shellinit 2> /dev/null)
docker_fix_route() {
sudo route delete 192.168.59.0/24 &> /dev/null
sudo route add 192.168.59.0/24 -iface vboxnet0 &> /dev/null
}

Cisco AnyConnect messes with your machine's routes. I've done this as a workaround:
$(boot2docker shellinit 2> /dev/null)
docker_fix_route() {
sudo route delete 192.168.59.0/24 &> /dev/null
sudo route add 192.168.59.0/24 -iface vboxnet0 &> /dev/null
}

@justinclayton justinclayton referenced this issue in docker/machine Jun 8, 2015

Closed

Make VirtualBox network configurable #24

@FrancoisZhang

This comment has been minimized.

Show comment
Hide comment
@FrancoisZhang

FrancoisZhang Jun 9, 2015

Does anybody have a final resolution to resolve this issue? I'm using the Yosemite + Anyconnect. As ipfw is removed from Yosemiti. It cause the script that @frosenberg provided doesn't work. Thanks a lot :-)

Does anybody have a final resolution to resolve this issue? I'm using the Yosemite + Anyconnect. As ipfw is removed from Yosemiti. It cause the script that @frosenberg provided doesn't work. Thanks a lot :-)

@chulkilee

This comment has been minimized.

Show comment
Hide comment
@chulkilee

chulkilee Jun 9, 2015

Using openconnect instead of Anyconnect, I could connect to containers in Kitematic.

Using openconnect instead of Anyconnect, I could connect to containers in Kitematic.

@keenaudio

This comment has been minimized.

Show comment
Hide comment
@keenaudio

keenaudio Jun 10, 2015

Setting up port forwarding in Virtual Box worked for me, as described here: #628 (comment) (thanks @johnnyt)
I used the Virtual Box UI (Settings / Network) Network type should be NAT. Click on port forwarding and add a rule to forward on 127.0.0.1:2376, then update your ~/.profile file:

export DOCKER_HOST=tcp://127.0.0.1:2376

Setting up port forwarding in Virtual Box worked for me, as described here: #628 (comment) (thanks @johnnyt)
I used the Virtual Box UI (Settings / Network) Network type should be NAT. Click on port forwarding and add a rule to forward on 127.0.0.1:2376, then update your ~/.profile file:

export DOCKER_HOST=tcp://127.0.0.1:2376

@norbertpy

This comment has been minimized.

Show comment
Hide comment
@norbertpy

norbertpy Jul 17, 2015

@chulkilee, openconnect fixed my problem. Thank you. I hate AnyConnect.

@chulkilee, openconnect fixed my problem. Thank you. I hate AnyConnect.

@tresbailey

This comment has been minimized.

Show comment
Hide comment
@tresbailey

tresbailey Jul 23, 2015

Thanks @johnnyt - I am using Yosemite with docker 1.6 with Cisco AnyConnect. Changing the docker host to 127.0.0.1 and adding the port forwarding worked for me.

Thanks @johnnyt - I am using Yosemite with docker 1.6 with Cisco AnyConnect. Changing the docker host to 127.0.0.1 and adding the port forwarding worked for me.

@eelcocramer

This comment has been minimized.

Show comment
Hide comment
@eelcocramer

eelcocramer Aug 18, 2015

So last week I installed the new set of docker tools including docker-machine. I also installed a new version of Cisco AnyConnect (4.1.00028). Things are working without any problems for me at the moment. Before, after and while on the VPN connection.

So last week I installed the new set of docker tools including docker-machine. I also installed a new version of Cisco AnyConnect (4.1.00028). Things are working without any problems for me at the moment. Before, after and while on the VPN connection.

@rickpeters

This comment has been minimized.

Show comment
Hide comment
@rickpeters

rickpeters Aug 20, 2015

Hi,
I also (have to) use anyconnect. Setting the port forwarding rule to 127.0.0.1 doesn't work anymore for me. Reason is that docker-machine creates secured docker hosts by default. When I forward using 127.0.0.1 i get a message from the docker host that the certificate is created for 192.168.99.100.
Also several commands (i.e. docker-machine env default) don't respond anymore.
I tried re-adding the route to the docker ip-range again (sudo route -nv add -net 192.168.99 -interface vboxnet1) but this doesn't work, probably because anyconnect doesn't allow this :-(
However, the already running containers for which I added a portforward (a webserver on port 8080) still respond, so some part of the solution still works?
Since anyconnect routes everything through the utun0 it seems like part of the traffic related to the docker-machine call gets blown into the tunnel and disappears?

Hi,
I also (have to) use anyconnect. Setting the port forwarding rule to 127.0.0.1 doesn't work anymore for me. Reason is that docker-machine creates secured docker hosts by default. When I forward using 127.0.0.1 i get a message from the docker host that the certificate is created for 192.168.99.100.
Also several commands (i.e. docker-machine env default) don't respond anymore.
I tried re-adding the route to the docker ip-range again (sudo route -nv add -net 192.168.99 -interface vboxnet1) but this doesn't work, probably because anyconnect doesn't allow this :-(
However, the already running containers for which I added a portforward (a webserver on port 8080) still respond, so some part of the solution still works?
Since anyconnect routes everything through the utun0 it seems like part of the traffic related to the docker-machine call gets blown into the tunnel and disappears?

@atomantic

This comment has been minimized.

Show comment
Hide comment
@atomantic

atomantic Aug 25, 2015

Anyone yet found a fix for this with docker-machine? it hangs when connected to VPN too, but doing the port forwarding to 127.0.0.1 doesn't seem to have an affect on it.

Anyone yet found a fix for this with docker-machine? it hangs when connected to VPN too, but doing the port forwarding to 127.0.0.1 doesn't seem to have an affect on it.

@Kalle80

This comment has been minimized.

Show comment
Hide comment
@Kalle80

Kalle80 Aug 28, 2015

I got it working yesterday with this flow (Cisco VPN Client and Win VirtualBox):

  1. VPN off
  2. Create new VM:

docker-machine create -d virtualbox --virtualbox-hostonly-cidr "10.32.21.100/24" default (IP would be in the same network than your VPN)

  1. Run eval for it

eval "$(docker-machine env default --shell ssh)"

4 Turn on VPN

But today after Win boot it could not connect to that VM. So I had to remove it and recreated it...

Kalle80 commented Aug 28, 2015

I got it working yesterday with this flow (Cisco VPN Client and Win VirtualBox):

  1. VPN off
  2. Create new VM:

docker-machine create -d virtualbox --virtualbox-hostonly-cidr "10.32.21.100/24" default (IP would be in the same network than your VPN)

  1. Run eval for it

eval "$(docker-machine env default --shell ssh)"

4 Turn on VPN

But today after Win boot it could not connect to that VM. So I had to remove it and recreated it...

@joshskinner

This comment has been minimized.

Show comment
Hide comment
@joshskinner

joshskinner Sep 17, 2015

@atomantic i was able to get docker-machine to work using @johnnyt solution with a few changes.

myhost=my_vb_name
myip=`docker-machine ip $myhost`
docker-machine stop $myhost
vboxmanage modifyvm "$myhost" --natpf1 "docker,tcp,$myip,2376,,2376"
docker-machine start $myhost

@atomantic i was able to get docker-machine to work using @johnnyt solution with a few changes.

myhost=my_vb_name
myip=`docker-machine ip $myhost`
docker-machine stop $myhost
vboxmanage modifyvm "$myhost" --natpf1 "docker,tcp,$myip,2376,,2376"
docker-machine start $myhost
@daagar

This comment has been minimized.

Show comment
Hide comment
@daagar

daagar Sep 28, 2015

@joshskinner's solution appears to have been the key in a Windows 7 environment with docker-machine and Cisco AnyConnect. Setting the port-forward IP to the docker-machine IP rather than 127.0.0.1 worked around certificate issues that were present with @johnnyt's solution.

As stated by @Kalle80, be sure to create the VM before connecting to VPN. Also, once having run AnyConnect it was necessary to reboot before AnyConnect was truly shutdown, even after killing the visible Cisco services.

daagar commented Sep 28, 2015

@joshskinner's solution appears to have been the key in a Windows 7 environment with docker-machine and Cisco AnyConnect. Setting the port-forward IP to the docker-machine IP rather than 127.0.0.1 worked around certificate issues that were present with @johnnyt's solution.

As stated by @Kalle80, be sure to create the VM before connecting to VPN. Also, once having run AnyConnect it was necessary to reboot before AnyConnect was truly shutdown, even after killing the visible Cisco services.

@onejli

This comment has been minimized.

Show comment
Hide comment
@onejli

onejli Oct 17, 2015

I hacked my way through a similar solution some time back, but never had time to automate it. It was working fine until I got bitten by the certificate issue that @rickpeters mentioned. I just put together a small helper script to re-apply the fixes https://github.com/onejli/docker-vpn-helper. The script will:

  • help VirtualBox fix the routing table
  • insert a port forwarding rule
  • regenerate the ssl cert

In its current state (with more than a few TODOs), the helper script assumes that you're creating (or using) a VM named default. This matches the VM name when using the Docker Quickstart Terminal. It patches the VM and outputs some environment variables that you'll need to export.

After running the helper script, you'll be able to:

  • Manage the VM using the standard docker-machine commands (e.g., stop, start, ssh, etc.)
  • Execute docker commands regardless of whether or not you're connected to the VPN

Cisco AnyConnect removes/redirects routes upon connection, but doesn't restore them after disconnecting. This seems to make the VirtualBox network kernel modules very unhappy. After dropping off of VPN, VirtualBox is able to add host-only network adapters, but it is NOT able to add the routes needed to connect them. I stumbled across this thread and found a solution in the last post that I integrated it into my helper script.

@daagar There's no need to reboot after disconnecting from AnyConnect. You just need to:

  1. Disconnect from AnyConnect (you can actually leave the application/services running)
  2. Stop all VirtualBox processes (i.e., all VMs and the GUI)
  3. Restart the VirtualBox kernel modules
    sudo /Library/Application\ Support/VirtualBox/LaunchDaemons/VirtualBoxStartup.sh restart

onejli commented Oct 17, 2015

I hacked my way through a similar solution some time back, but never had time to automate it. It was working fine until I got bitten by the certificate issue that @rickpeters mentioned. I just put together a small helper script to re-apply the fixes https://github.com/onejli/docker-vpn-helper. The script will:

  • help VirtualBox fix the routing table
  • insert a port forwarding rule
  • regenerate the ssl cert

In its current state (with more than a few TODOs), the helper script assumes that you're creating (or using) a VM named default. This matches the VM name when using the Docker Quickstart Terminal. It patches the VM and outputs some environment variables that you'll need to export.

After running the helper script, you'll be able to:

  • Manage the VM using the standard docker-machine commands (e.g., stop, start, ssh, etc.)
  • Execute docker commands regardless of whether or not you're connected to the VPN

Cisco AnyConnect removes/redirects routes upon connection, but doesn't restore them after disconnecting. This seems to make the VirtualBox network kernel modules very unhappy. After dropping off of VPN, VirtualBox is able to add host-only network adapters, but it is NOT able to add the routes needed to connect them. I stumbled across this thread and found a solution in the last post that I integrated it into my helper script.

@daagar There's no need to reboot after disconnecting from AnyConnect. You just need to:

  1. Disconnect from AnyConnect (you can actually leave the application/services running)
  2. Stop all VirtualBox processes (i.e., all VMs and the GUI)
  3. Restart the VirtualBox kernel modules
    sudo /Library/Application\ Support/VirtualBox/LaunchDaemons/VirtualBoxStartup.sh restart

@onejli onejli referenced this issue in docker/machine Oct 18, 2015

Closed

Docker Machine create hangs (OSX) #1819

@bfarrell

This comment has been minimized.

Show comment
Hide comment
@bfarrell

bfarrell Oct 21, 2015

Thanks @onejli !! This has been holding me back for months. Finally something worked.
osx 10.10.5

Thanks @onejli !! This has been holding me back for months. Finally something worked.
osx 10.10.5

@rickpeters

This comment has been minimized.

Show comment
Hide comment
@rickpeters

rickpeters Oct 22, 2015

@onejli , we have moved on beyond this issue :-)
Instead of looking at getting it to work while using the anyconnect vpn on the Mac we turned it around.
What we did was put anyconnect itself in a container :-)
The advantage of this is that openvpn (the opensource anyconnect client) just breaks the complete stack inside the vpn container itself and not on my osx itself.
I'm still thinking on sharing this solution, however there are some secrets in the images I use that I cannot share with others, so I would need to do some clean up.
We use this solution in our team on a daily basis and have (almost) no need for Cisco AnyConnect anymore.
Global overview of the solution is that we have a vpn container which uses oneconnect to connect to the corporate vpn. Then we have a second container that uses apache httpd as a proxy server to the vpn container. The http proxy delivers a proxy.pac file so the mac browser knows which adresses go to the corporate vpn and which should go to the real (separate) internet connection.
SSH into corporate servers is done using a docker exec into the vpn container.

Second part of the solution is that sometimes we create a really transparant tunnel from the osx host (using sshuttle) to the apache container and the vpn container and just tunnel the complete 10.0.0.0/8 range of addresses through the ssh tunnel. Also works great, but is sometimes a bit slower.

The big advantage is that my local mac is not touched by the vpn at all and everything works (and also all docker tools) work like a charm. Even a local Docker swarm is not a problem anymore :-)

grtz,
Rick

@onejli , we have moved on beyond this issue :-)
Instead of looking at getting it to work while using the anyconnect vpn on the Mac we turned it around.
What we did was put anyconnect itself in a container :-)
The advantage of this is that openvpn (the opensource anyconnect client) just breaks the complete stack inside the vpn container itself and not on my osx itself.
I'm still thinking on sharing this solution, however there are some secrets in the images I use that I cannot share with others, so I would need to do some clean up.
We use this solution in our team on a daily basis and have (almost) no need for Cisco AnyConnect anymore.
Global overview of the solution is that we have a vpn container which uses oneconnect to connect to the corporate vpn. Then we have a second container that uses apache httpd as a proxy server to the vpn container. The http proxy delivers a proxy.pac file so the mac browser knows which adresses go to the corporate vpn and which should go to the real (separate) internet connection.
SSH into corporate servers is done using a docker exec into the vpn container.

Second part of the solution is that sometimes we create a really transparant tunnel from the osx host (using sshuttle) to the apache container and the vpn container and just tunnel the complete 10.0.0.0/8 range of addresses through the ssh tunnel. Also works great, but is sometimes a bit slower.

The big advantage is that my local mac is not touched by the vpn at all and everything works (and also all docker tools) work like a charm. Even a local Docker swarm is not a problem anymore :-)

grtz,
Rick

@onejli

This comment has been minimized.

Show comment
Hide comment
@onejli

onejli Oct 22, 2015

@rickpeters I wish I could run my VPN client from within a VM or container 😢. Unfortunately, there are some corporate security rules that prohibit us from going down this road.

@bfarrell Happy to help! Just keep in mind that this solution is more of a band-aid. Unfortunately, it doesn't help when you want to expose a port from within a container to the physical host.
e.g. docker run -p 5000:5000 registry

A host-only adapter would normally allow you to access any port mapped from a docker container. Due to VPN crippling communication between the physical host and the VM over the host-only adapter, you'll need to manually insert a port forwarding rule over the NAT interface for each container port that you want to expose.
e.g. (in the case above)
VBoxManage controlvm default natpf1 registry_port,tcp,127.0.0.1,5000,,5000
or via the GUI

onejli commented Oct 22, 2015

@rickpeters I wish I could run my VPN client from within a VM or container 😢. Unfortunately, there are some corporate security rules that prohibit us from going down this road.

@bfarrell Happy to help! Just keep in mind that this solution is more of a band-aid. Unfortunately, it doesn't help when you want to expose a port from within a container to the physical host.
e.g. docker run -p 5000:5000 registry

A host-only adapter would normally allow you to access any port mapped from a docker container. Due to VPN crippling communication between the physical host and the VM over the host-only adapter, you'll need to manually insert a port forwarding rule over the NAT interface for each container port that you want to expose.
e.g. (in the case above)
VBoxManage controlvm default natpf1 registry_port,tcp,127.0.0.1,5000,,5000
or via the GUI

@rickpeters

This comment has been minimized.

Show comment
Hide comment
@rickpeters

rickpeters Oct 23, 2015

@onejli I think we have the same corporate security rules ;-) However my take on this is that since BYOD is possible and even corporate laptops are able to function on a normal internet connection (and allowed to do so), there is really no big risk in using Docker as a vpn tunnel in this way :-)
However everyone is free in his own choices! Good Luck!

@onejli I think we have the same corporate security rules ;-) However my take on this is that since BYOD is possible and even corporate laptops are able to function on a normal internet connection (and allowed to do so), there is really no big risk in using Docker as a vpn tunnel in this way :-)
However everyone is free in his own choices! Good Luck!

@gcarre

This comment has been minimized.

Show comment
Hide comment
@gcarre

gcarre Jan 6, 2016

I installed Cisco Anyconnect 4.2 and it fixed my Docker issues: I could use Docker (1.9.1) while connected to the VPN, but as soon as I was disconnected I couldn't use docker anymore, I had to add the route again manually.

gcarre commented Jan 6, 2016

I installed Cisco Anyconnect 4.2 and it fixed my Docker issues: I could use Docker (1.9.1) while connected to the VPN, but as soon as I was disconnected I couldn't use docker anymore, I had to add the route again manually.

@bilal

This comment has been minimized.

Show comment
Hide comment
@bilal

bilal Jan 13, 2016

If possible, running Cisco VPN in split tunnel mode is an option and it does't have all these problems. It might be a good tradeoff where you can still access your corporate network and still work with docker!

bilal commented Jan 13, 2016

If possible, running Cisco VPN in split tunnel mode is an option and it does't have all these problems. It might be a good tradeoff where you can still access your corporate network and still work with docker!

@eoliphan

This comment has been minimized.

Show comment
Hide comment
@eoliphan

eoliphan Jan 23, 2016

Just as a quick FYI, I've been pulling my hair out about this for several days (I'm on osx) , been over this issue and all the related ones several times. Well, I just thought to check to see if there was a VMWare Fusion driver for docker-machine, there is, and zero problems. I know it's not free, but it's headache-free for this issue at least.

Just as a quick FYI, I've been pulling my hair out about this for several days (I'm on osx) , been over this issue and all the related ones several times. Well, I just thought to check to see if there was a VMWare Fusion driver for docker-machine, there is, and zero problems. I know it's not free, but it's headache-free for this issue at least.

@alexandrem

This comment has been minimized.

Show comment
Hide comment
@alexandrem

alexandrem Jan 27, 2016

I've struggled with these problems for months (Cisco AnyConnect v3.1.xxx)

Just switched to openconnect (http://www.infradead.org/openconnect/) and not looking back. Solves all the issues mentioned here.

Works like a charm on Yosemite (10.11.x)

brew install openconnect

I've struggled with these problems for months (Cisco AnyConnect v3.1.xxx)

Just switched to openconnect (http://www.infradead.org/openconnect/) and not looking back. Solves all the issues mentioned here.

Works like a charm on Yosemite (10.11.x)

brew install openconnect
@mohdaliiqbal

This comment has been minimized.

Show comment
Hide comment
@mohdaliiqbal

mohdaliiqbal Jan 29, 2016

Thanks @onejli - the VBox launch daemon restart part was critical for me. I use your vpn-helper everyday now.

Thanks @onejli - the VBox launch daemon restart part was critical for me. I use your vpn-helper everyday now.

@onejli

This comment has been minimized.

Show comment
Hide comment
@onejli

onejli Feb 8, 2016

@bilal @gcarre Most of my complaints about AnyConnect are around a particularly draconian configuration. No split tunneling and no restoration of the routing table after disconnect make me sad.

@mohdaliiqbal Great to hear it!

onejli commented Feb 8, 2016

@bilal @gcarre Most of my complaints about AnyConnect are around a particularly draconian configuration. No split tunneling and no restoration of the routing table after disconnect make me sad.

@mohdaliiqbal Great to hear it!

@mchiang0610

This comment has been minimized.

Show comment
Hide comment
@mchiang0610

mchiang0610 Mar 15, 2016

If anyone still has problems with this, please contact me at mchiang@docker.com.

We are working on something to help address this issue.

If anyone still has problems with this, please contact me at mchiang@docker.com.

We are working on something to help address this issue.

@atomantic

This comment has been minimized.

Show comment
Hide comment
@atomantic

atomantic Mar 15, 2016

I finally found a solution:
Stop using Cisco AnyConnect client and use openconnect instead. Openconnect is basically a direct port of AnyConnect without all the stuff that messes up your networking: http://www.infradead.org/openconnect/manual.html

brew install openconnect
sudo openconnect --user=$USERNAME $VPNHOST

It will prompt for password, which is your pin+rsa token.

Then just leave this terminal open and open another to do your work.

I finally found a solution:
Stop using Cisco AnyConnect client and use openconnect instead. Openconnect is basically a direct port of AnyConnect without all the stuff that messes up your networking: http://www.infradead.org/openconnect/manual.html

brew install openconnect
sudo openconnect --user=$USERNAME $VPNHOST

It will prompt for password, which is your pin+rsa token.

Then just leave this terminal open and open another to do your work.

@mchiang0610

This comment has been minimized.

Show comment
Hide comment
@mchiang0610

mchiang0610 Mar 15, 2016

@atomantic That might not be an option for enterprise users.

@atomantic That might not be an option for enterprise users.

@atomantic

This comment has been minimized.

Show comment
Hide comment
@atomantic

atomantic Mar 15, 2016

I'm an enterprise user ;)

I'm an enterprise user ;)

@mchiang0610

This comment has been minimized.

Show comment
Hide comment
@mchiang0610

mchiang0610 Mar 15, 2016

@atomantic Some companies force you to stay with AnyConnect as a policy.

We've been working on something new to help address this issue. Still in early private beta

@atomantic Some companies force you to stay with AnyConnect as a policy.

We've been working on something new to help address this issue. Still in early private beta

@dantran

This comment has been minimized.

Show comment
Hide comment
@dantran

dantran Mar 23, 2016

@mchiang0610, i am interesting on any solution you have :-)

dantran commented Mar 23, 2016

@mchiang0610, i am interesting on any solution you have :-)

@davidkarlsen

This comment has been minimized.

Show comment
Hide comment
@davidkarlsen

davidkarlsen Mar 23, 2016

@mchiang0610 I have sent you a couple of mails on the subject. Will you get back in touch? I'd be happy to test.

@mchiang0610 I have sent you a couple of mails on the subject. Will you get back in touch? I'd be happy to test.

@chino

This comment has been minimized.

Show comment
Hide comment
@chino

chino Mar 24, 2016

Has anyone heard back yet?

On Wed, Mar 23, 2016 at 9:08 AM, David J. M. Karlsen <
notifications@github.com> wrote:

@mchiang0610 https://github.com/mchiang0610 I have sent you a couple of
mails on the subject. Will you get back in touch? I'd be happy to test.


You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#628 (comment)

chino commented Mar 24, 2016

Has anyone heard back yet?

On Wed, Mar 23, 2016 at 9:08 AM, David J. M. Karlsen <
notifications@github.com> wrote:

@mchiang0610 https://github.com/mchiang0610 I have sent you a couple of
mails on the subject. Will you get back in touch? I'd be happy to test.


You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#628 (comment)

@bfarrell

This comment has been minimized.

Show comment
Hide comment
@bfarrell

bfarrell Mar 24, 2016

Count me in the list that havent received a response.

Count me in the list that havent received a response.

@dantran

This comment has been minimized.

Show comment
Hide comment
@dantran

dantran Mar 24, 2016

docker/machine#2632 ssh port forwarding fits the bill for me for now

dantran commented Mar 24, 2016

docker/machine#2632 ssh port forwarding fits the bill for me for now

@yoplait

This comment has been minimized.

Show comment
Hide comment
@yoplait

yoplait Mar 24, 2016

@mchiang0610 I am sending you a mail with this... Thanks!

yoplait commented Mar 24, 2016

@mchiang0610 I am sending you a mail with this... Thanks!

@gcarre

This comment has been minimized.

Show comment
Hide comment
@gcarre

gcarre Apr 13, 2016

has anyone been able to make this work with Docker for Mac beta?

gcarre commented Apr 13, 2016

has anyone been able to make this work with Docker for Mac beta?

@rickpeters

This comment has been minimized.

Show comment
Hide comment
@rickpeters

rickpeters Apr 13, 2016

Yes, works as documented. Put it in VPN compatibility mode start your VPN and use the address of the docker daemon VM that's in pinata list

Yes, works as documented. Put it in VPN compatibility mode start your VPN and use the address of the docker daemon VM that's in pinata list

@chino

This comment has been minimized.

Show comment
Hide comment
@chino

chino Apr 13, 2016

It will just bind to local host I imagine instead of using host-only
network?

On Wed, Apr 13, 2016, 1:41 AM Rick Peters notifications@github.com wrote:

Yes, works as documented. Put it in VPN compatibility mode start your VPN
and use the address of the docker daemon VM that's in pinata list


You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#628 (comment)

chino commented Apr 13, 2016

It will just bind to local host I imagine instead of using host-only
network?

On Wed, Apr 13, 2016, 1:41 AM Rick Peters notifications@github.com wrote:

Yes, works as documented. Put it in VPN compatibility mode start your VPN
and use the address of the docker daemon VM that's in pinata list


You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#628 (comment)

@rickpeters

This comment has been minimized.

Show comment
Hide comment
@rickpeters

rickpeters Apr 13, 2016

Don't know the real magic. However to me it looks mostly like it uses a special IP address that your VPN leaves alone. Furthermore if I understand correctly you are not (yet) able to expose ports to your host machine itself. So the main part is that the docker daemon will stay available while you are on your VPN connection. Also, the docker.local alias for addressing the docker VM does not (yet) work when in VPN compatibility mode.

Don't know the real magic. However to me it looks mostly like it uses a special IP address that your VPN leaves alone. Furthermore if I understand correctly you are not (yet) able to expose ports to your host machine itself. So the main part is that the docker daemon will stay available while you are on your VPN connection. Also, the docker.local alias for addressing the docker VM does not (yet) work when in VPN compatibility mode.

@chino

This comment has been minimized.

Show comment
Hide comment
@chino

chino Apr 14, 2016

Can the containers communicate together still? Will host be able to act as
a nat gateway for them? If you can't expose ports seems confusing how you
could even access the vm at that point since at least ssh is needed?

On Wed, Apr 13, 2016, 2:37 PM Rick Peters notifications@github.com wrote:

Don't know the real magic. However to me it looks mostly like it uses a
special IP address that your VPN leaves alone. Furthermore if I understand
correctly you are not (yet) able to expose ports to your host machine
itself. So the main part is that the docker daemon will stay available
while you are on your VPN connection. Also, the docker.local alias for
addressing the docker VM does not (yet) work when in VPN compatibility mode.


You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#628 (comment)

chino commented Apr 14, 2016

Can the containers communicate together still? Will host be able to act as
a nat gateway for them? If you can't expose ports seems confusing how you
could even access the vm at that point since at least ssh is needed?

On Wed, Apr 13, 2016, 2:37 PM Rick Peters notifications@github.com wrote:

Don't know the real magic. However to me it looks mostly like it uses a
special IP address that your VPN leaves alone. Furthermore if I understand
correctly you are not (yet) able to expose ports to your host machine
itself. So the main part is that the docker daemon will stay available
while you are on your VPN connection. Also, the docker.local alias for
addressing the docker VM does not (yet) work when in VPN compatibility mode.


You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#628 (comment)

@shaunsenecal

This comment has been minimized.

Show comment
Hide comment
@shaunsenecal

shaunsenecal Aug 20, 2016

My situation was that I was that as soon as I enabled my VPN (using openconnect), I was no longer able to reach my containers.

$> ping 172.17.0.2
PING 172.17.0.2 (172.17.0.2) 56(84) bytes of data.                                                                                                                                         
From 172.17.0.2 icmp_seq=3 Destination Host Unreachable                                                                                                                                    
^C                                                                                                                                                                                         
--- 172.17.0.2 ping statistics ---                                                                                                                                                         
5 packets transmitted, 0 received, +1 errors, 100% packet loss, time 4018ms

I took a look at the routes on my host and noticed that there was a duplicate route

$> ip route show | grep 172.17
172.17.0.0/16 dev vpn0  proto static 
172.17.0.0/16 dev docker0  proto kernel  scope link  src 172.17.0.1 

So I deleted the extraneous route. I didnt want docker traffic going over the VPN, so that is the route I removed

$> sudo ip route delete 172.17.0.0/16 dev vpn0

Now I'm able to ping my containers, and my containers have network access again

$> ping 172.17.0.2            
PING 172.17.0.2 (172.17.0.2) 56(84) bytes of data.
64 bytes from 172.17.0.2: icmp_seq=1 ttl=64 time=0.057 ms
64 bytes from 172.17.0.2: icmp_seq=2 ttl=64 time=0.032 ms
64 bytes from 172.17.0.2: icmp_seq=3 ttl=64 time=0.030 ms
^C
--- 172.17.0.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms

It seems that the important thing is that there is a route on your host going to the docker interface and that there is not another route ahead of it. If you are missing the route, simply add it and you should be back in business

sudo ip route add 172.17.0.0/16 dev docker0

Of course, this needs to be run each time you reconnect to your VPN, but its easily scripted.

My situation was that I was that as soon as I enabled my VPN (using openconnect), I was no longer able to reach my containers.

$> ping 172.17.0.2
PING 172.17.0.2 (172.17.0.2) 56(84) bytes of data.                                                                                                                                         
From 172.17.0.2 icmp_seq=3 Destination Host Unreachable                                                                                                                                    
^C                                                                                                                                                                                         
--- 172.17.0.2 ping statistics ---                                                                                                                                                         
5 packets transmitted, 0 received, +1 errors, 100% packet loss, time 4018ms

I took a look at the routes on my host and noticed that there was a duplicate route

$> ip route show | grep 172.17
172.17.0.0/16 dev vpn0  proto static 
172.17.0.0/16 dev docker0  proto kernel  scope link  src 172.17.0.1 

So I deleted the extraneous route. I didnt want docker traffic going over the VPN, so that is the route I removed

$> sudo ip route delete 172.17.0.0/16 dev vpn0

Now I'm able to ping my containers, and my containers have network access again

$> ping 172.17.0.2            
PING 172.17.0.2 (172.17.0.2) 56(84) bytes of data.
64 bytes from 172.17.0.2: icmp_seq=1 ttl=64 time=0.057 ms
64 bytes from 172.17.0.2: icmp_seq=2 ttl=64 time=0.032 ms
64 bytes from 172.17.0.2: icmp_seq=3 ttl=64 time=0.030 ms
^C
--- 172.17.0.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms

It seems that the important thing is that there is a route on your host going to the docker interface and that there is not another route ahead of it. If you are missing the route, simply add it and you should be back in business

sudo ip route add 172.17.0.0/16 dev docker0

Of course, this needs to be run each time you reconnect to your VPN, but its easily scripted.

@DiJu519

This comment has been minimized.

Show comment
Hide comment
@DiJu519

DiJu519 Aug 24, 2016

Thoughts if i wanted to hit an internal private docker repo, over VPN?

DiJu519 commented Aug 24, 2016

Thoughts if i wanted to hit an internal private docker repo, over VPN?

@shaunsenecal

This comment has been minimized.

Show comment
Hide comment
@shaunsenecal

shaunsenecal Aug 25, 2016

Can you still hit the repo from your docker host? If you can, then I think the container should be able to hit the repo as well, since its traffic should go from the container over the docker0 interface, then the docker host should route the data to the repo over vpn0

Can you still hit the repo from your docker host? If you can, then I think the container should be able to hit the repo as well, since its traffic should go from the container over the docker0 interface, then the docker host should route the data to the repo over vpn0

@volkertb

This comment has been minimized.

Show comment
Hide comment
@volkertb

volkertb Sep 17, 2016

There is another workaround for this frustrating VPN problem: you can talk to the Docker machine (boot2docker VM) through an (emulated) serial console. I didn't see it documented anywhere, but I noticed in boot2docker's Dockerfile that boot2docker does indeed start up serial consoles on /dev/ttyS0 (COM1) and /dev/ttyS1 (COM2) if those serial ports are available.

I tried it with one emulated serial port and it worked. Instructions for Windows are described below. The instructions will be somewhat different for Linux and Mac environments, but setting up emulated serial connections between a host computer (regardless of the OS) and a VM on VirtualBox is well-documented on many sites.

To enable emulated serial ports in your Docker machine, shut it down first, then open the VirtualBox GUI, right-click on your Docker machine VM, select Settings... -> select list item "Serial Ports" -> select tab "Port 1" (if not already selected). Configure the settings under the tab as follows (assuming Windows, it will be somewhat different on Linux or Mac hosts):

  • Check (enable) "Enable Serial Port"
  • Port Mode: "Host Pipe"
  • Uncheck (disable) "Connect to existing pipe/socket"
  • Path/Address: ".\pipe\vbox_boot2docker_com1" (without the quotes)
  • Click OK.

Now start the Docker machine (boot2docker) VM up again. Do this preferably through the "docker-machine" command, unless of course you're already running a VPN connection, in which case you'll have little choice but to start the VM through VirtualBox, through either the GUI or the VBoxManage command.

To connect to the emulated serial port on the now running VM, use a console application that can initiate serial connections to host pipes. I recommend PuTTY for this. In the case of PuTTY, you can connect to the emulated serial port by starting PuTTY and creating a PuTTY session with the following settings/parameters:

  • Connection type: Serial (select the radio button next to "Serial")
  • Serial line: ".\pipe\vbox_boot2docker_com1" (without the quotes, the same string you entered behind "Path/Address" earlier in VirtualBox)
  • Speed: 115200
  • Text field right under the title "Saved Sessions": (enter any name here that is convenient and practical to you)
  • Click Save to save the session for later reuse.
  • Either double-click on the saved session in the list or click on the "Open" button below to start the session.
  • When a new PuTTY console pops up with a green cursor in it, press ENTER to request a login prompt.
  • When prompted for a user name, enter "docker" (without the quotes). It should not ask you for a password.

You should now have regained control over your Docker machine. And since you are now controlling it over a serial connection (albeit an emulated one), the connection will not be disrupted by any changes in your host computer's TCP/IP configuration. You can continue to enter Docker commands and manage your container instances, regardless of whether you are connected through a VPN or not. Access to the outside Internet from within the Docker machine should also still be possible, since the NAT interface managed by VirtualBox should also remain unaffected by any changes on the host machine, except for possible proxy server issues on the VPN of course. But you can work around those as well by configuring the http_proxy and https_proxy environment variables on the Docker machine and inside your container instances.

WARNING: when you disconnect the serial console by closing it (without entering the command "exit"), you will still be logged in the next time you connect to the same serial console. Make sure no one else has access to the user session on your computer.

You should still be able to access any local files in your home folder(s) through the path /c/Users if you created the docker VM using the standard docker-machine settings. If you need access to any other local folders, you will have to stop the docker machine VM and add it as a shared folder using the VirtualBox VM and then restart the VM and mount the newly added shared volume.

By the way, it would be a very practical and convenient enhancement if the docker-machine client tool could be improved to automatically fall back to an emulated serial connection to the boot2docker VM, whenever SSH connections to it failed.

Lastly, it might be a good idea to simply create a separate "client VM" in VirtualBox (alongside your Docker machine) and run a light-weight OS with a GUI and a web browser in it. I'd recommend Lubuntu in Live CD mode for this, but you could for instance also use one of the Windows/IE VMs that Microsoft has officially made available for testing purposes, in case you need to test anything with Internet Explorer. The client VM can then serve to both control and test the Docker machine. To virtually "connect" it to the Docker machine, add a host-only network interface to this new client VM and connect it to the same host-only adapter that is already connected to the Docker machine. This way, the new client VM will be able to access the Docker machine directly. And since the connection between these two VMs will be taking place over an internal virtual "network switch" managed by VirtualBox, this connection will remain in tact, even once the host machine no longer has access to it due to changed routes by a VPN client. You will then be able to continue accessing and testing your docker container instances on the Docker machine by accessing it from the new client VM. In addition, you could also add a second network interface in NAT mode to the client VM, so you can access both the Docker machine and the outside internet (again, taking account any possible proxy issues that pop up while connecting to a corporate VPN).

I hope these suggestions help! 😃

There is another workaround for this frustrating VPN problem: you can talk to the Docker machine (boot2docker VM) through an (emulated) serial console. I didn't see it documented anywhere, but I noticed in boot2docker's Dockerfile that boot2docker does indeed start up serial consoles on /dev/ttyS0 (COM1) and /dev/ttyS1 (COM2) if those serial ports are available.

I tried it with one emulated serial port and it worked. Instructions for Windows are described below. The instructions will be somewhat different for Linux and Mac environments, but setting up emulated serial connections between a host computer (regardless of the OS) and a VM on VirtualBox is well-documented on many sites.

To enable emulated serial ports in your Docker machine, shut it down first, then open the VirtualBox GUI, right-click on your Docker machine VM, select Settings... -> select list item "Serial Ports" -> select tab "Port 1" (if not already selected). Configure the settings under the tab as follows (assuming Windows, it will be somewhat different on Linux or Mac hosts):

  • Check (enable) "Enable Serial Port"
  • Port Mode: "Host Pipe"
  • Uncheck (disable) "Connect to existing pipe/socket"
  • Path/Address: ".\pipe\vbox_boot2docker_com1" (without the quotes)
  • Click OK.

Now start the Docker machine (boot2docker) VM up again. Do this preferably through the "docker-machine" command, unless of course you're already running a VPN connection, in which case you'll have little choice but to start the VM through VirtualBox, through either the GUI or the VBoxManage command.

To connect to the emulated serial port on the now running VM, use a console application that can initiate serial connections to host pipes. I recommend PuTTY for this. In the case of PuTTY, you can connect to the emulated serial port by starting PuTTY and creating a PuTTY session with the following settings/parameters:

  • Connection type: Serial (select the radio button next to "Serial")
  • Serial line: ".\pipe\vbox_boot2docker_com1" (without the quotes, the same string you entered behind "Path/Address" earlier in VirtualBox)
  • Speed: 115200
  • Text field right under the title "Saved Sessions": (enter any name here that is convenient and practical to you)
  • Click Save to save the session for later reuse.
  • Either double-click on the saved session in the list or click on the "Open" button below to start the session.
  • When a new PuTTY console pops up with a green cursor in it, press ENTER to request a login prompt.
  • When prompted for a user name, enter "docker" (without the quotes). It should not ask you for a password.

You should now have regained control over your Docker machine. And since you are now controlling it over a serial connection (albeit an emulated one), the connection will not be disrupted by any changes in your host computer's TCP/IP configuration. You can continue to enter Docker commands and manage your container instances, regardless of whether you are connected through a VPN or not. Access to the outside Internet from within the Docker machine should also still be possible, since the NAT interface managed by VirtualBox should also remain unaffected by any changes on the host machine, except for possible proxy server issues on the VPN of course. But you can work around those as well by configuring the http_proxy and https_proxy environment variables on the Docker machine and inside your container instances.

WARNING: when you disconnect the serial console by closing it (without entering the command "exit"), you will still be logged in the next time you connect to the same serial console. Make sure no one else has access to the user session on your computer.

You should still be able to access any local files in your home folder(s) through the path /c/Users if you created the docker VM using the standard docker-machine settings. If you need access to any other local folders, you will have to stop the docker machine VM and add it as a shared folder using the VirtualBox VM and then restart the VM and mount the newly added shared volume.

By the way, it would be a very practical and convenient enhancement if the docker-machine client tool could be improved to automatically fall back to an emulated serial connection to the boot2docker VM, whenever SSH connections to it failed.

Lastly, it might be a good idea to simply create a separate "client VM" in VirtualBox (alongside your Docker machine) and run a light-weight OS with a GUI and a web browser in it. I'd recommend Lubuntu in Live CD mode for this, but you could for instance also use one of the Windows/IE VMs that Microsoft has officially made available for testing purposes, in case you need to test anything with Internet Explorer. The client VM can then serve to both control and test the Docker machine. To virtually "connect" it to the Docker machine, add a host-only network interface to this new client VM and connect it to the same host-only adapter that is already connected to the Docker machine. This way, the new client VM will be able to access the Docker machine directly. And since the connection between these two VMs will be taking place over an internal virtual "network switch" managed by VirtualBox, this connection will remain in tact, even once the host machine no longer has access to it due to changed routes by a VPN client. You will then be able to continue accessing and testing your docker container instances on the Docker machine by accessing it from the new client VM. In addition, you could also add a second network interface in NAT mode to the client VM, so you can access both the Docker machine and the outside internet (again, taking account any possible proxy issues that pop up while connecting to a corporate VPN).

I hope these suggestions help! 😃

@sudtek

This comment has been minimized.

Show comment
Hide comment
@sudtek

sudtek Jan 3, 2017

• Path/Address: ".\pipe\vbox_boot2docker_com1" (without the quotes)

not on virtual box windows must be "//./pipe/docker_engine" (without the quotes) (the default for kinematic)

sudtek commented Jan 3, 2017

• Path/Address: ".\pipe\vbox_boot2docker_com1" (without the quotes)

not on virtual box windows must be "//./pipe/docker_engine" (without the quotes) (the default for kinematic)

@cowwoc

This comment has been minimized.

Show comment
Hide comment
@cowwoc

cowwoc Jan 25, 2017

@volkertb's suggestion worked for me but I had to set the pipe name to \\.\pipe\vbox_boot2docker_com1 (notice the two slashes added to the beginning).

I never could get kinematic to work, even following @sudtek's suggestion.

All of this is under Windows 10.

cowwoc commented Jan 25, 2017

@volkertb's suggestion worked for me but I had to set the pipe name to \\.\pipe\vbox_boot2docker_com1 (notice the two slashes added to the beginning).

I never could get kinematic to work, even following @sudtek's suggestion.

All of this is under Windows 10.

@blockloop blockloop referenced this issue in kubernetes/minikube Feb 8, 2017

Closed

Can't use Minikube on VPN #1099

@vhosakot

This comment has been minimized.

Show comment
Hide comment
@vhosakot

vhosakot Feb 8, 2017

Due to this issue in Cisco AnyConnect VPN client version 4.3.03086, I'm not able to run minikube (https://kubernetes.io/docs/tutorials/stateless-application/hello-minikube/) when I'm connected to the Cisco AnyConnect VPN.

Interestingly, I see in the Cisco AnyConnect VPN logs that it flaps every time I start minikube.

    12:45:51 AM    Reconnecting to Boxborough - SSL...
    12:45:53 AM    Establishing VPN - Examining system...
    12:45:53 AM    Establishing VPN - Activating VPN adapter...
    12:45:53 AM    Establishing VPN - Configuring system...
    12:45:56 AM    Establishing VPN...
    12:45:56 AM    Connected to Boxborough - SSL.
    12:46:32 AM    Reconnecting to Boxborough - SSL...
    12:46:34 AM    Establishing VPN - Examining system...
    12:46:34 AM    Establishing VPN - Activating VPN adapter...
    12:46:34 AM    Establishing VPN - Configuring system...
    12:46:36 AM    Establishing VPN...
    12:46:36 AM    Connected to Boxborough - SSL.
    12:49:55 AM    Reconnecting to Boxborough - SSL...
    12:49:57 AM    Establishing VPN - Examining system...
    12:49:57 AM    Establishing VPN - Activating VPN adapter...
    12:49:57 AM    Establishing VPN - Configuring system...
    12:49:59 AM    Establishing VPN...
    12:49:59 AM    Connected to Boxborough - SSL.
    12:51:11 AM    Reconnecting to Boxborough - SSL...
    12:51:13 AM    Establishing VPN - Examining system...
    12:51:13 AM    Establishing VPN - Activating VPN adapter...
    12:51:13 AM    Establishing VPN - Configuring system...
    12:51:15 AM    Establishing VPN...
    12:51:16 AM    Connected to Boxborough - SSL.
    12:52:38 AM    Reconnecting to Boxborough - SSL...
    12:52:40 AM    Establishing VPN - Examining system...
    12:52:40 AM    Establishing VPN - Activating VPN adapter...
    12:52:40 AM    Establishing VPN - Configuring system...
    12:52:43 AM    Establishing VPN...
    12:52:43 AM    Connected to Boxborough - SSL.
    12:56:46 AM    Reconnecting to Boxborough - SSL...
    12:56:48 AM    Establishing VPN - Examining system...
    12:56:48 AM    Establishing VPN - Activating VPN adapter...
    12:56:48 AM    Establishing VPN - Configuring system...
    12:56:50 AM    Establishing VPN...
    12:56:50 AM    Connected to Boxborough - SSL.

vhosakot commented Feb 8, 2017

Due to this issue in Cisco AnyConnect VPN client version 4.3.03086, I'm not able to run minikube (https://kubernetes.io/docs/tutorials/stateless-application/hello-minikube/) when I'm connected to the Cisco AnyConnect VPN.

Interestingly, I see in the Cisco AnyConnect VPN logs that it flaps every time I start minikube.

    12:45:51 AM    Reconnecting to Boxborough - SSL...
    12:45:53 AM    Establishing VPN - Examining system...
    12:45:53 AM    Establishing VPN - Activating VPN adapter...
    12:45:53 AM    Establishing VPN - Configuring system...
    12:45:56 AM    Establishing VPN...
    12:45:56 AM    Connected to Boxborough - SSL.
    12:46:32 AM    Reconnecting to Boxborough - SSL...
    12:46:34 AM    Establishing VPN - Examining system...
    12:46:34 AM    Establishing VPN - Activating VPN adapter...
    12:46:34 AM    Establishing VPN - Configuring system...
    12:46:36 AM    Establishing VPN...
    12:46:36 AM    Connected to Boxborough - SSL.
    12:49:55 AM    Reconnecting to Boxborough - SSL...
    12:49:57 AM    Establishing VPN - Examining system...
    12:49:57 AM    Establishing VPN - Activating VPN adapter...
    12:49:57 AM    Establishing VPN - Configuring system...
    12:49:59 AM    Establishing VPN...
    12:49:59 AM    Connected to Boxborough - SSL.
    12:51:11 AM    Reconnecting to Boxborough - SSL...
    12:51:13 AM    Establishing VPN - Examining system...
    12:51:13 AM    Establishing VPN - Activating VPN adapter...
    12:51:13 AM    Establishing VPN - Configuring system...
    12:51:15 AM    Establishing VPN...
    12:51:16 AM    Connected to Boxborough - SSL.
    12:52:38 AM    Reconnecting to Boxborough - SSL...
    12:52:40 AM    Establishing VPN - Examining system...
    12:52:40 AM    Establishing VPN - Activating VPN adapter...
    12:52:40 AM    Establishing VPN - Configuring system...
    12:52:43 AM    Establishing VPN...
    12:52:43 AM    Connected to Boxborough - SSL.
    12:56:46 AM    Reconnecting to Boxborough - SSL...
    12:56:48 AM    Establishing VPN - Examining system...
    12:56:48 AM    Establishing VPN - Activating VPN adapter...
    12:56:48 AM    Establishing VPN - Configuring system...
    12:56:50 AM    Establishing VPN...
    12:56:50 AM    Connected to Boxborough - SSL.
@vhosakot

This comment has been minimized.

Show comment
Hide comment
@vhosakot

vhosakot Feb 8, 2017

I generated the DART (Diagnostic And Reporting Tool) report from Cisco AnyConnect VPN, and see these errors in system.log. Clearly, we can see below docker is flapping Cisco AnyConnect VPN.

Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Adding filter rule: pass in log quick on awdl0 inet6 proto ipv6-icmp from ::0 to FF02:0:0:0:0:1:FF50:A62A icmp6-type 135 code 0
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Adding filter rule: pass out log quick on awdl0 inet6 proto ipv6-icmp from FE80:0:0:0:50D4:61FF:FE50:A62A to FF02::1 icmp6-type 136 code 0
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Adding filter rule: pass in log quick on awdl0 inet6 proto ipv6-icmp from any to FF02::1 icmp6-type 136 code 0
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Adding filter rule: block log quick inet all ! tagged cisco_anyconnect_vpn_pass
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Adding filter rule: block log quick inet6 all ! tagged cisco_anyconnect_vpn_pass
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: saveRulesToConfigFile File: ../../vpn/AgentUtilities/UnixFwUtil_OSX.cpp Line: 758 Updating pf configuration file /opt/cisco/anyconnect/ac_pf_final.conf with: 0 options, 0 scrubs, 1 IPv4 rules, 1 IPv6 rules
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: saveRulesToConfigFile File: ../../vpn/AgentUtilities/UnixFwUtil_OSX.cpp Line: 758 Updating pf configuration file /opt/cisco/anyconnect/ac_pf_filt.conf with: 1 options, 1 scrubs, 11 IPv4 rules, 43 IPv6 rules
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: enablePf File: ../../vpn/AgentUtilities/UnixFwUtil_OSX.cpp Line: 1165 Command '/sbin/pfctl -E -f /etc/pf.conf 2>&1' returned output pfctl: Use of -f option, could result in flushing of rules present in the main ruleset added by the system at startup. See /etc/pf.conf for further details.  No ALTQ support in kernel ALTQ related functions disabled pf enabled Token : 18380007534171971645  , extracted token 18380007534171971645
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: reloadPfRules File: ../../vpn/AgentUtilities/UnixFwUtil_OSX.cpp Line: 1251 Command '/sbin/pfctl -f /etc/pf.conf 2>&1' returned output pfctl: Use of -f option, could result in flushing of rules present in the main ruleset added by the system at startup. See /etc/pf.conf for further details.  No ALTQ support in kernel ALTQ related functions disabled 
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: applyFirewallConfiguration File: ../../vpn/AgentUtilities/HostConfigMgr.cpp Line: 900 No Firewall Rules to configure
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: The network control state changed to restricted.
Feb  8 00:45:32 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: Message type information sent to the user: Establishing VPN...
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: OnTunnelStateChange File: ../../vpn/Agent/TND.cpp Line: 1970 tunnel state change notification (new 1, old 0)
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: The VPN connection has been established and can now pass data.
Feb  8 00:45:32 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: VPN state: Connected Network state: Network Accessible Network control state: Network Access: Restricted Network type: Untrusted
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: The Primary DTLS connection to the secure gateway is being established.
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: OnTunnelStateChange File: ../../vpn/Agent/TND.cpp Line: 1970 tunnel state change notification (new 1, old 1)
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: connectTransport File: ../../vpn/Common/IPC/SocketTransport.cpp Line: 856 Invoked Function: ::bind Return Code: 22 (0x00000016) Description: unknown 
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: initiateTransport File: ../../vpn/Agent/DtlsTunnelTransport.cpp Line: 222 Opened DTLS socket from [192.168.0.29]:64858 to [198.135.0.166]:443
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpndownloader[8247]: Function: WaitForCompletion File: /tmp/build/thehoff/Ironman2_MR30.980020412858/Ironman2_MR3/vpn/Common/Utility/Thread.cpp Line: 286 The thread has successfully completed execution.
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpndownloader[8247]: Cisco AnyConnect Secure Mobility Client Downloader (VPN) exiting, version 4.3.03086 , return code 0 [0x00000000]
Feb  8 00:45:32 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: Function: launchCachedDownloader File: ../../vpn/Api/ConnectMgr.cpp Line: 7016 Invoked Function: ConnectMgr :: launchCachedDownloader Return Code: 0 (0x00000000) Description: Cached Downloader terminated normally 
Feb  8 00:45:32 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: VPN state: Connected Network state: Network Accessible Network control state: Network Access: Restricted Network type: Untrusted
Feb  8 00:45:32 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: Message type information sent to the user: Connected to Boxborough - SSL.
Feb  8 00:45:32 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: Launching script "/opt/cisco/anyconnect/script/OnConnect_Mac_AnyConnect_Screenlock_D13v2.sh".
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: A DTLS connection has been established using cipher AES256-SHA
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: The Primary DTLS connection to the secure gateway has been established.
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: OnTunnelStateChange File: ../../vpn/Agent/TND.cpp Line: 1970 tunnel state change notification (new 1, old 1)
Feb  8 00:45:34 VHOSAKOT-M-H6X5 Docker[814]: hosts file has bindings for localhost broadcasthost localhost
Feb  8 00:45:34 VHOSAKOT-M-H6X5 defaults[8385]: 
	The domain/default pair of (/Library/Preferences/com.jamfsoftware.jamf, global_log_level) does not exist
Feb  8 00:45:34 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: PluginManager: Monitor - No Plugins Changes observed! [4->4]
Feb  8 00:45:36 VHOSAKOT-M-H6X5 defaults[8426]: 
	The domain/default pair of (/Library/Preferences/com.jamfsoftware.jamf, global_log_level) does not exist
Feb  8 00:45:41 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: Script "/opt/cisco/anyconnect/script/OnConnect_Mac_AnyConnect_Screenlock_D13v2.sh" exited with code 0.
Feb  8 00:45:50 VHOSAKOT-M-H6X5 diskimages-helper[8520]: *** -[NSMachPort handlePortMessage:]: dropping incoming DO message because the connection is invalid
Feb  8 00:45:51 VHOSAKOT-M-H6X5 com.apple.xpc.launchd[1] (com.apple.ftp-proxy): Unknown key for Boolean: ForceEnableHack
Feb  8 00:45:51 VHOSAKOT-M-H6X5 com.apple.xpc.launchd[1] (com.apple.bootpd): Unknown key for Boolean: ForceEnableHack
Feb  8 00:45:51 VHOSAKOT-M-H6X5 kernel[0]: bridge100: attached with 1 suspended link-layer multicast membership(s)
Feb  8 00:45:51 VHOSAKOT-M-H6X5 kernel[0]: en5: promiscuous mode enable succeeded
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]: BCAST is ready [anyExternal, mtu=1406 ]
Feb  8 00:45:51 VHOSAKOT-M-H6X5 kernel[0]: bridge100: successfully restored 1 suspended link-layer multicast membership(s) (err=0)
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]: added addr=192.168.64.1 mask=255.255.255.0 on bridge100
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: A new network interface has been detected.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: logInterfaces File: ../../vpn/AgentUtilities/Routing/InterfaceRouteMonitorCommon.cpp Line: 477 IP Address Interface List: FE80:0:0:0:CA69:CDFF:FEA0:4C88 192.168.0.29 2601:18B:4100:9505:CA69:CDFF:FEA0:4C88 2601:18B:4100:9505:E1CF:A829:66A:2A7A FE80:0:0:0:50D4:61FF:FE50:A62A 10.86.247.79 FE80:0:0:0:CA69:CDFF:FEA0:4C88 2001:420:C0E4:1002:0:0:0:AC 192.168.64.1 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Reconfigure reason code 15: New network interface.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: netInterfaceNoticeCategoryHandler File: ../../vpn/Agent/MainThread.cpp Line: 7965 Network Interface change detected, refreshing physical MAC addresses
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: A routing table change notification has been received.  Starting automatic correction of the routing table.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]: started: [DHCP subnet=192.168.64/24 on bridge100 mtu=1500 <---> anyExternal mtu=1406] max-mss=1366
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]:   dns: 192.168.64.1
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: findMatchingRouteChange File: ../../vpn/AgentUtilities/Routing/RouteHandlerCommon.cpp Line: 4300 Found matching non-LL IPv4 VA default route.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: WaitWhileProcessingEvents File: ../../vpn/Agent/MainThread.cpp Line: 9794 Invoked Function: CMainThread::internalProcessEvents Return Code: -32702455 (0xFE0D0009) Description: MAINTHREAD_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]: attempted to start dns proxy on anyExternal
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]: docker-machine-driver-xhyve: com.apple.NetworkSharing.broadcast-0 has been started
Feb  8 00:45:51 VHOSAKOT-M-H6X5 mDNSResponder[109]: SetupDNSProxySkts: 14, 20, 25, 27
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: setDefaultRouteViaSysConfig File: ../../vpn/AgentUtilities/Routing/RouteHandlerMac.cpp Line: 605 Failed to get primary service IPv4 information
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: specialDefaultRouteAutoCorrect File: ../../vpn/AgentUtilities/Routing/RouteHandlerMac.cpp Line: 382 Invoked Function: CRouteHandlerMac::setDefaultRouteViaSysConfig Return Code: -24117239 (0xFE900009) Description: ROUTETABLE_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: VerifyRouteTable File: ../../vpn/AgentUtilities/Routing/RouteHandlerCommon.cpp Line: 1049 Invoked Function: CRouteHandlerCommon::specialDefaultRouteAutoCorrect Return Code: -24117239 (0xFE900009) Description: ROUTETABLE_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table - Original     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric         0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.29                                                             en0       4  N       0    10.86.247.79  255.255.255.255     10.86.247.79     10.86.247.79                                                           utun0       9  N       0       127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1                                                             lo0       1  N       0       127.0.0.1  255.255.255.255        127.0.0.1        127.0.0.1                                                             lo0       1  N       0     169.254.0.0      255.255.0.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     192.168.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0    192.168.0.29  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0       224.0.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0 255.255.255.255  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table - Modified     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric         0.0.0.0          0.0.0.0     10.86.247.79     10.86.247.79                                                           utun0       9  N       0         0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.29                                                             en0       4  N       0         0.0.0.0          0.0.0.0          0.0.0.0     192.168.64.1                                                       bridge100      11  Y       0    10.86.247.79  255.255.255.255          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0       127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1                                                             lo0       1  N       0       127.0.0.1  255.255.255.255        127.0.0.1        127.0.0.1                                                             lo0       1  N       0     169.254.0.0      255.255.0.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     192.168.0.0    255.255.255.0          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0     192.168.0.1  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0    192.168.0.29  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0    192.168.64.0    255.255.255.0          0.0.0.0     192.168.64.1                                                       bridge100      11  Y       0   198.135.0.166  255.255.255.255      192.168.0.1     192.168.0.29                                                             en0       4  N       0       224.0.0.0    255.255.255.0          0.0.0.0     192.168.0.29      
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Index of questionable route entry in 'Modified' table: 2
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table changes: index  Action  Found     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric     0     NOP      Y         0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.29                                                             en0       4  N       0     1     DEL      N     192.168.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     2     ADD      N   198.135.0.166  255.255.255.255      192.168.0.1     192.168.0.29                                                             en0       4  N       0     3     ADD      N     192.168.0.0    255.255.255.0          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0     4     NOP      Y         0.0.0.0          0.0.0.0          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0     5     NOP      N 255.255.255.255  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     6     NOP      N       224.0.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     7     NOP      N    192.168.0.29  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     8     NOP      N       127.0.0.1  255.255.255.255        127.0.0.1        127.0.0.1                                                             lo0       1  N       0     9     NOP      N       127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1                                                             lo0       1  N       0 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: OnRouteTableChange File: ../../vpn/AgentUtilities/Routing/RouteMgr.cpp Line: 478 Invoked Function: IRouteHandler::VerifyRouteTable Return Code: -24117239 (0xFE900009) Description: ROUTETABLE_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 com.apple.pfd[8524]: clearing states for internet-sharingshared_v4
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]: dns proxy successfully enabled
Feb  8 00:45:51 VHOSAKOT-M-H6X5 configd[60]: network changed: v4(en0:192.168.0.29) v6(utun0:2001:420:c0e4:1002::ac, en0) DNS! Proxy SMB
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database lists search domains: cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: ; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 75.75.75.75, 75.75.76.76, 2001:558:feed::1, 2001:558:feed::2; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 161.44.124.122, 64.102.6.247; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database has domain name: cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: WaitWhileProcessingEvents File: ../../vpn/Agent/MainThread.cpp Line: 9794 Invoked Function: CMainThread::internalProcessEvents Return Code: -32702455 (0xFE0D0009) Description: MAINTHREAD_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[814]: updating resolvers to nameserver 2001:558:feed::2#53
	timeout 2000
	order 200000
	nameserver 2001:558:feed::1#53
	timeout 2000
	order 200000
	nameserver 161.44.124.122#53
	timeout 2000
	order 200000
	nameserver 75.75.76.76#53
	timeout 2000
	order 200000
	nameserver 75.75.75.75#53
	timeout 2000
	order 200000
	nameserver 64.102.6.247#53
	timeout 2000
	order 200000
	search cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 configd[60]: network changed: v4(utun0+:10.86.247.79, en0) v6(utun0:2001:420:c0e4:1002::ac, en0) DNS! Proxy SMB
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database lists search domains: cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 161.44.124.122, 64.102.6.247; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 75.75.75.75, 75.75.76.76, 2001:558:feed::1, 2001:558:feed::2; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 161.44.124.122, 64.102.6.247; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: WaitWhileProcessingEvents File: ../../vpn/Agent/MainThread.cpp Line: 9794 Invoked Function: CMainThread::internalProcessEvents Return Code: -32702455 (0xFE0D0009) Description: MAINTHREAD_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: executeRouteCmd File: ../../vpn/AgentUtilities/Routing/RouteTableMac.cpp Line: 219 route cmd success: route delete - dest 192.168.64.0/24, defGw 0.0.0.0, intf bridge100 (idx 11), metric 0, link-level
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table - fixed - deleted route     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric    192.168.64.0    255.255.255.0          0.0.0.0     192.168.64.1                                                       bridge100      11  Y       0 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database has domain name: cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: WaitWhileProcessingEvents File: ../../vpn/Agent/MainThread.cpp Line: 9794 Invoked Function: CMainThread::internalProcessEvents Return Code: -32702455 (0xFE0D0009) Description: MAINTHREAD_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 configd[60]: network changed: v4(en0:192.168.0.29) v6(utun0:2001:420:c0e4:1002::ac, en0) DNS! Proxy SMB
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[814]: updating resolvers to nameserver 2001:558:feed::2#53
	timeout 2000
	order 200000
	nameserver 2001:558:feed::1#53
	timeout 2000
	order 200000
	nameserver 161.44.124.122#53
	timeout 2000
	order 200000
	nameserver 75.75.76.76#53
	timeout 2000
	order 200000
	nameserver 75.75.75.75#53
	timeout 2000
	order 200000
	nameserver 64.102.6.247#53
	timeout 2000
	order 200000
	search cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 com.apple.pfd[8524]: clearing states for internet-sharingshared_v4
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: setDefaultRouteViaSysConfig File: ../../vpn/AgentUtilities/Routing/RouteHandlerMac.cpp Line: 605 Failed to get primary service IPv4 information
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: specialVADefaultRouteAutoCorrect File: ../../vpn/AgentUtilities/Routing/RouteHandlerMac.cpp Line: 449 Invoked Function: CRouteHandlerMac::setDefaultRouteViaSysConfig Return Code: -24117239 (0xFE900009) Description: ROUTETABLE_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: VerifyRouteTable File: ../../vpn/AgentUtilities/Routing/RouteHandlerCommon.cpp Line: 1197 Invoked Function: CRouteHandlerCommon::specialVADefaultRouteAutoCorrect Return Code: -24117239 (0xFE900009) Description: ROUTETABLE_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table - Original     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric         0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.29                                                             en0       4  N       0    10.86.247.79  255.255.255.255     10.86.247.79     10.86.247.79                                                           utun0       9  N       0       127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1                                                             lo0       1  N       0       127.0.0.1  255.255.255.255        127.0.0.1        127.0.0.1                                                             lo0       1  N       0     169.254.0.0      255.255.0.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     192.168.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0    192.168.0.29  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0       224.0.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0 255.255.255.255  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table - Modified     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric         0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.29                                                             en0       4  N       0         0.0.0.0          0.0.0.0          0.0.0.0     192.168.64.1                                                       bridge100      11  Y       0       127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1                                                             lo0       1  N       0       127.0.0.1  255.255.255.255        127.0.0.1        127.0.0.1                                                             lo0       1  N       0     169.254.0.0      255.255.0.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     192.168.0.0    255.255.255.0          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0     192.168.0.1  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0    192.168.0.29  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0    192.168.64.0    255.255.255.0          0.0.0.0     192.168.64.1                                                       bridge100      11  Y       0   198.135.0.166  255.255.255.255      192.168.0.1     192.168.0.29                                                             en0       4  N       0       224.0.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0 255.255.255.255  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table changes: index  Action  Found     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric     0     NOP      Y         0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.29                                                             en0       4  N       0     1     DEL      N     192.168.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     2     ADD      Y   198.135.0.166  255.255.255.255      192.168.0.1     192.168.0.29                                                             en0       4  N       0     3     ADD      Y     192.168.0.0    255.255.255.0          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0     4     NOP      N         0.0.0.0          0.0.0.0          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0     5     NOP      Y 255.255.255.255  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     6     NOP      Y       224.0.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     7     NOP      Y    192.168.0.29  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     8     NOP      Y       127.0.0.1  255.255.255.255        127.0.0.1        127.0.0.1                                                             lo0       1  N       0     9     NOP      Y       127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1                                                             lo0       1  N       0    10     DEL      N    192.168.64.0    255.255.255.0          0.0.0.0     192.168.64.1                                                       b
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: OnRouteTableChange File: ../../vpn/AgentUtilities/Routing/RouteMgr.cpp Line: 478 Invoked Function: IRouteHandler::VerifyRouteTable Return Code: -24117239 (0xFE900009) Description: ROUTETABLE_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database lists search domains: cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: ; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 75.75.75.75, 75.75.76.76, 2001:558:feed::1, 2001:558:feed::2; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 161.44.124.122, 64.102.6.247; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database has domain name: cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Automatic correction of the routing table has failed.  Notifying higher levels of the routing change notification for possible further corrective action.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Reconfigure reason code 5: IP forwarding table modification.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: The entire VPN connection is being reconfigured.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: OnTunnelStateChange File: ../../vpn/Agent/TND.cpp Line: 1970 tunnel state change notification (new 2, old 1)
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: VPN state: Reconnecting Network state: Network Accessible Network control state: Network Access: Restricted Network type: Untrusted
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: Message type information sent to the user: Reconnecting to Boxborough - SSL...

vhosakot commented Feb 8, 2017

I generated the DART (Diagnostic And Reporting Tool) report from Cisco AnyConnect VPN, and see these errors in system.log. Clearly, we can see below docker is flapping Cisco AnyConnect VPN.

Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Adding filter rule: pass in log quick on awdl0 inet6 proto ipv6-icmp from ::0 to FF02:0:0:0:0:1:FF50:A62A icmp6-type 135 code 0
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Adding filter rule: pass out log quick on awdl0 inet6 proto ipv6-icmp from FE80:0:0:0:50D4:61FF:FE50:A62A to FF02::1 icmp6-type 136 code 0
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Adding filter rule: pass in log quick on awdl0 inet6 proto ipv6-icmp from any to FF02::1 icmp6-type 136 code 0
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Adding filter rule: block log quick inet all ! tagged cisco_anyconnect_vpn_pass
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Adding filter rule: block log quick inet6 all ! tagged cisco_anyconnect_vpn_pass
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: saveRulesToConfigFile File: ../../vpn/AgentUtilities/UnixFwUtil_OSX.cpp Line: 758 Updating pf configuration file /opt/cisco/anyconnect/ac_pf_final.conf with: 0 options, 0 scrubs, 1 IPv4 rules, 1 IPv6 rules
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: saveRulesToConfigFile File: ../../vpn/AgentUtilities/UnixFwUtil_OSX.cpp Line: 758 Updating pf configuration file /opt/cisco/anyconnect/ac_pf_filt.conf with: 1 options, 1 scrubs, 11 IPv4 rules, 43 IPv6 rules
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: enablePf File: ../../vpn/AgentUtilities/UnixFwUtil_OSX.cpp Line: 1165 Command '/sbin/pfctl -E -f /etc/pf.conf 2>&1' returned output pfctl: Use of -f option, could result in flushing of rules present in the main ruleset added by the system at startup. See /etc/pf.conf for further details.  No ALTQ support in kernel ALTQ related functions disabled pf enabled Token : 18380007534171971645  , extracted token 18380007534171971645
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: reloadPfRules File: ../../vpn/AgentUtilities/UnixFwUtil_OSX.cpp Line: 1251 Command '/sbin/pfctl -f /etc/pf.conf 2>&1' returned output pfctl: Use of -f option, could result in flushing of rules present in the main ruleset added by the system at startup. See /etc/pf.conf for further details.  No ALTQ support in kernel ALTQ related functions disabled 
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: applyFirewallConfiguration File: ../../vpn/AgentUtilities/HostConfigMgr.cpp Line: 900 No Firewall Rules to configure
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: The network control state changed to restricted.
Feb  8 00:45:32 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: Message type information sent to the user: Establishing VPN...
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: OnTunnelStateChange File: ../../vpn/Agent/TND.cpp Line: 1970 tunnel state change notification (new 1, old 0)
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: The VPN connection has been established and can now pass data.
Feb  8 00:45:32 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: VPN state: Connected Network state: Network Accessible Network control state: Network Access: Restricted Network type: Untrusted
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: The Primary DTLS connection to the secure gateway is being established.
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: OnTunnelStateChange File: ../../vpn/Agent/TND.cpp Line: 1970 tunnel state change notification (new 1, old 1)
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: connectTransport File: ../../vpn/Common/IPC/SocketTransport.cpp Line: 856 Invoked Function: ::bind Return Code: 22 (0x00000016) Description: unknown 
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: initiateTransport File: ../../vpn/Agent/DtlsTunnelTransport.cpp Line: 222 Opened DTLS socket from [192.168.0.29]:64858 to [198.135.0.166]:443
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpndownloader[8247]: Function: WaitForCompletion File: /tmp/build/thehoff/Ironman2_MR30.980020412858/Ironman2_MR3/vpn/Common/Utility/Thread.cpp Line: 286 The thread has successfully completed execution.
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpndownloader[8247]: Cisco AnyConnect Secure Mobility Client Downloader (VPN) exiting, version 4.3.03086 , return code 0 [0x00000000]
Feb  8 00:45:32 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: Function: launchCachedDownloader File: ../../vpn/Api/ConnectMgr.cpp Line: 7016 Invoked Function: ConnectMgr :: launchCachedDownloader Return Code: 0 (0x00000000) Description: Cached Downloader terminated normally 
Feb  8 00:45:32 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: VPN state: Connected Network state: Network Accessible Network control state: Network Access: Restricted Network type: Untrusted
Feb  8 00:45:32 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: Message type information sent to the user: Connected to Boxborough - SSL.
Feb  8 00:45:32 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: Launching script "/opt/cisco/anyconnect/script/OnConnect_Mac_AnyConnect_Screenlock_D13v2.sh".
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: A DTLS connection has been established using cipher AES256-SHA
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: The Primary DTLS connection to the secure gateway has been established.
Feb  8 00:45:32 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: OnTunnelStateChange File: ../../vpn/Agent/TND.cpp Line: 1970 tunnel state change notification (new 1, old 1)
Feb  8 00:45:34 VHOSAKOT-M-H6X5 Docker[814]: hosts file has bindings for localhost broadcasthost localhost
Feb  8 00:45:34 VHOSAKOT-M-H6X5 defaults[8385]: 
	The domain/default pair of (/Library/Preferences/com.jamfsoftware.jamf, global_log_level) does not exist
Feb  8 00:45:34 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: PluginManager: Monitor - No Plugins Changes observed! [4->4]
Feb  8 00:45:36 VHOSAKOT-M-H6X5 defaults[8426]: 
	The domain/default pair of (/Library/Preferences/com.jamfsoftware.jamf, global_log_level) does not exist
Feb  8 00:45:41 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: Script "/opt/cisco/anyconnect/script/OnConnect_Mac_AnyConnect_Screenlock_D13v2.sh" exited with code 0.
Feb  8 00:45:50 VHOSAKOT-M-H6X5 diskimages-helper[8520]: *** -[NSMachPort handlePortMessage:]: dropping incoming DO message because the connection is invalid
Feb  8 00:45:51 VHOSAKOT-M-H6X5 com.apple.xpc.launchd[1] (com.apple.ftp-proxy): Unknown key for Boolean: ForceEnableHack
Feb  8 00:45:51 VHOSAKOT-M-H6X5 com.apple.xpc.launchd[1] (com.apple.bootpd): Unknown key for Boolean: ForceEnableHack
Feb  8 00:45:51 VHOSAKOT-M-H6X5 kernel[0]: bridge100: attached with 1 suspended link-layer multicast membership(s)
Feb  8 00:45:51 VHOSAKOT-M-H6X5 kernel[0]: en5: promiscuous mode enable succeeded
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]: BCAST is ready [anyExternal, mtu=1406 ]
Feb  8 00:45:51 VHOSAKOT-M-H6X5 kernel[0]: bridge100: successfully restored 1 suspended link-layer multicast membership(s) (err=0)
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]: added addr=192.168.64.1 mask=255.255.255.0 on bridge100
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: A new network interface has been detected.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: logInterfaces File: ../../vpn/AgentUtilities/Routing/InterfaceRouteMonitorCommon.cpp Line: 477 IP Address Interface List: FE80:0:0:0:CA69:CDFF:FEA0:4C88 192.168.0.29 2601:18B:4100:9505:CA69:CDFF:FEA0:4C88 2601:18B:4100:9505:E1CF:A829:66A:2A7A FE80:0:0:0:50D4:61FF:FE50:A62A 10.86.247.79 FE80:0:0:0:CA69:CDFF:FEA0:4C88 2001:420:C0E4:1002:0:0:0:AC 192.168.64.1 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Reconfigure reason code 15: New network interface.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: netInterfaceNoticeCategoryHandler File: ../../vpn/Agent/MainThread.cpp Line: 7965 Network Interface change detected, refreshing physical MAC addresses
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: A routing table change notification has been received.  Starting automatic correction of the routing table.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]: started: [DHCP subnet=192.168.64/24 on bridge100 mtu=1500 <---> anyExternal mtu=1406] max-mss=1366
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]:   dns: 192.168.64.1
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: findMatchingRouteChange File: ../../vpn/AgentUtilities/Routing/RouteHandlerCommon.cpp Line: 4300 Found matching non-LL IPv4 VA default route.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: WaitWhileProcessingEvents File: ../../vpn/Agent/MainThread.cpp Line: 9794 Invoked Function: CMainThread::internalProcessEvents Return Code: -32702455 (0xFE0D0009) Description: MAINTHREAD_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]: attempted to start dns proxy on anyExternal
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]: docker-machine-driver-xhyve: com.apple.NetworkSharing.broadcast-0 has been started
Feb  8 00:45:51 VHOSAKOT-M-H6X5 mDNSResponder[109]: SetupDNSProxySkts: 14, 20, 25, 27
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: setDefaultRouteViaSysConfig File: ../../vpn/AgentUtilities/Routing/RouteHandlerMac.cpp Line: 605 Failed to get primary service IPv4 information
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: specialDefaultRouteAutoCorrect File: ../../vpn/AgentUtilities/Routing/RouteHandlerMac.cpp Line: 382 Invoked Function: CRouteHandlerMac::setDefaultRouteViaSysConfig Return Code: -24117239 (0xFE900009) Description: ROUTETABLE_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: VerifyRouteTable File: ../../vpn/AgentUtilities/Routing/RouteHandlerCommon.cpp Line: 1049 Invoked Function: CRouteHandlerCommon::specialDefaultRouteAutoCorrect Return Code: -24117239 (0xFE900009) Description: ROUTETABLE_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table - Original     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric         0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.29                                                             en0       4  N       0    10.86.247.79  255.255.255.255     10.86.247.79     10.86.247.79                                                           utun0       9  N       0       127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1                                                             lo0       1  N       0       127.0.0.1  255.255.255.255        127.0.0.1        127.0.0.1                                                             lo0       1  N       0     169.254.0.0      255.255.0.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     192.168.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0    192.168.0.29  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0       224.0.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0 255.255.255.255  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table - Modified     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric         0.0.0.0          0.0.0.0     10.86.247.79     10.86.247.79                                                           utun0       9  N       0         0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.29                                                             en0       4  N       0         0.0.0.0          0.0.0.0          0.0.0.0     192.168.64.1                                                       bridge100      11  Y       0    10.86.247.79  255.255.255.255          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0       127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1                                                             lo0       1  N       0       127.0.0.1  255.255.255.255        127.0.0.1        127.0.0.1                                                             lo0       1  N       0     169.254.0.0      255.255.0.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     192.168.0.0    255.255.255.0          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0     192.168.0.1  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0    192.168.0.29  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0    192.168.64.0    255.255.255.0          0.0.0.0     192.168.64.1                                                       bridge100      11  Y       0   198.135.0.166  255.255.255.255      192.168.0.1     192.168.0.29                                                             en0       4  N       0       224.0.0.0    255.255.255.0          0.0.0.0     192.168.0.29      
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Index of questionable route entry in 'Modified' table: 2
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table changes: index  Action  Found     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric     0     NOP      Y         0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.29                                                             en0       4  N       0     1     DEL      N     192.168.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     2     ADD      N   198.135.0.166  255.255.255.255      192.168.0.1     192.168.0.29                                                             en0       4  N       0     3     ADD      N     192.168.0.0    255.255.255.0          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0     4     NOP      Y         0.0.0.0          0.0.0.0          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0     5     NOP      N 255.255.255.255  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     6     NOP      N       224.0.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     7     NOP      N    192.168.0.29  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     8     NOP      N       127.0.0.1  255.255.255.255        127.0.0.1        127.0.0.1                                                             lo0       1  N       0     9     NOP      N       127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1                                                             lo0       1  N       0 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: OnRouteTableChange File: ../../vpn/AgentUtilities/Routing/RouteMgr.cpp Line: 478 Invoked Function: IRouteHandler::VerifyRouteTable Return Code: -24117239 (0xFE900009) Description: ROUTETABLE_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 com.apple.pfd[8524]: clearing states for internet-sharingshared_v4
Feb  8 00:45:51 VHOSAKOT-M-H6X5 InternetSharing[8523]: dns proxy successfully enabled
Feb  8 00:45:51 VHOSAKOT-M-H6X5 configd[60]: network changed: v4(en0:192.168.0.29) v6(utun0:2001:420:c0e4:1002::ac, en0) DNS! Proxy SMB
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database lists search domains: cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: ; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 75.75.75.75, 75.75.76.76, 2001:558:feed::1, 2001:558:feed::2; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 161.44.124.122, 64.102.6.247; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database has domain name: cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: WaitWhileProcessingEvents File: ../../vpn/Agent/MainThread.cpp Line: 9794 Invoked Function: CMainThread::internalProcessEvents Return Code: -32702455 (0xFE0D0009) Description: MAINTHREAD_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[814]: updating resolvers to nameserver 2001:558:feed::2#53
	timeout 2000
	order 200000
	nameserver 2001:558:feed::1#53
	timeout 2000
	order 200000
	nameserver 161.44.124.122#53
	timeout 2000
	order 200000
	nameserver 75.75.76.76#53
	timeout 2000
	order 200000
	nameserver 75.75.75.75#53
	timeout 2000
	order 200000
	nameserver 64.102.6.247#53
	timeout 2000
	order 200000
	search cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 configd[60]: network changed: v4(utun0+:10.86.247.79, en0) v6(utun0:2001:420:c0e4:1002::ac, en0) DNS! Proxy SMB
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database lists search domains: cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 161.44.124.122, 64.102.6.247; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 75.75.75.75, 75.75.76.76, 2001:558:feed::1, 2001:558:feed::2; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 161.44.124.122, 64.102.6.247; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: WaitWhileProcessingEvents File: ../../vpn/Agent/MainThread.cpp Line: 9794 Invoked Function: CMainThread::internalProcessEvents Return Code: -32702455 (0xFE0D0009) Description: MAINTHREAD_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: executeRouteCmd File: ../../vpn/AgentUtilities/Routing/RouteTableMac.cpp Line: 219 route cmd success: route delete - dest 192.168.64.0/24, defGw 0.0.0.0, intf bridge100 (idx 11), metric 0, link-level
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table - fixed - deleted route     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric    192.168.64.0    255.255.255.0          0.0.0.0     192.168.64.1                                                       bridge100      11  Y       0 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database has domain name: cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: WaitWhileProcessingEvents File: ../../vpn/Agent/MainThread.cpp Line: 9794 Invoked Function: CMainThread::internalProcessEvents Return Code: -32702455 (0xFE0D0009) Description: MAINTHREAD_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 configd[60]: network changed: v4(en0:192.168.0.29) v6(utun0:2001:420:c0e4:1002::ac, en0) DNS! Proxy SMB
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[814]: updating resolvers to nameserver 2001:558:feed::2#53
	timeout 2000
	order 200000
	nameserver 2001:558:feed::1#53
	timeout 2000
	order 200000
	nameserver 161.44.124.122#53
	timeout 2000
	order 200000
	nameserver 75.75.76.76#53
	timeout 2000
	order 200000
	nameserver 75.75.75.75#53
	timeout 2000
	order 200000
	nameserver 64.102.6.247#53
	timeout 2000
	order 200000
	search cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 com.apple.pfd[8524]: clearing states for internet-sharingshared_v4
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: setDefaultRouteViaSysConfig File: ../../vpn/AgentUtilities/Routing/RouteHandlerMac.cpp Line: 605 Failed to get primary service IPv4 information
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: specialVADefaultRouteAutoCorrect File: ../../vpn/AgentUtilities/Routing/RouteHandlerMac.cpp Line: 449 Invoked Function: CRouteHandlerMac::setDefaultRouteViaSysConfig Return Code: -24117239 (0xFE900009) Description: ROUTETABLE_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: VerifyRouteTable File: ../../vpn/AgentUtilities/Routing/RouteHandlerCommon.cpp Line: 1197 Invoked Function: CRouteHandlerCommon::specialVADefaultRouteAutoCorrect Return Code: -24117239 (0xFE900009) Description: ROUTETABLE_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table - Original     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric         0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.29                                                             en0       4  N       0    10.86.247.79  255.255.255.255     10.86.247.79     10.86.247.79                                                           utun0       9  N       0       127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1                                                             lo0       1  N       0       127.0.0.1  255.255.255.255        127.0.0.1        127.0.0.1                                                             lo0       1  N       0     169.254.0.0      255.255.0.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     192.168.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0    192.168.0.29  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0       224.0.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0 255.255.255.255  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table - Modified     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric         0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.29                                                             en0       4  N       0         0.0.0.0          0.0.0.0          0.0.0.0     192.168.64.1                                                       bridge100      11  Y       0       127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1                                                             lo0       1  N       0       127.0.0.1  255.255.255.255        127.0.0.1        127.0.0.1                                                             lo0       1  N       0     169.254.0.0      255.255.0.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     192.168.0.0    255.255.255.0          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0     192.168.0.1  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0    192.168.0.29  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0    192.168.64.0    255.255.255.0          0.0.0.0     192.168.64.1                                                       bridge100      11  Y       0   198.135.0.166  255.255.255.255      192.168.0.1     192.168.0.29                                                             en0       4  N       0       224.0.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0 255.255.255.255  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Routing table changes: index  Action  Found     Destination          Netmask          Gateway           IfAddr                                                          IfName IfIndex LL  Metric     0     NOP      Y         0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.29                                                             en0       4  N       0     1     DEL      N     192.168.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     2     ADD      Y   198.135.0.166  255.255.255.255      192.168.0.1     192.168.0.29                                                             en0       4  N       0     3     ADD      Y     192.168.0.0    255.255.255.0          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0     4     NOP      N         0.0.0.0          0.0.0.0          0.0.0.0     10.86.247.79                                                           utun0       9  Y       0     5     NOP      Y 255.255.255.255  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     6     NOP      Y       224.0.0.0    255.255.255.0          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     7     NOP      Y    192.168.0.29  255.255.255.255          0.0.0.0     192.168.0.29                                                             en0       4  Y       0     8     NOP      Y       127.0.0.1  255.255.255.255        127.0.0.1        127.0.0.1                                                             lo0       1  N       0     9     NOP      Y       127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1                                                             lo0       1  N       0    10     DEL      N    192.168.64.0    255.255.255.0          0.0.0.0     192.168.64.1                                                       b
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: OnRouteTableChange File: ../../vpn/AgentUtilities/Routing/RouteMgr.cpp Line: 478 Invoked Function: IRouteHandler::VerifyRouteTable Return Code: -24117239 (0xFE900009) Description: ROUTETABLE_ERROR_UNEXPECTED 
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database lists search domains: cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: ; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 75.75.75.75, 75.75.76.76, 2001:558:feed::1, 2001:558:feed::2; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database includes DNS service: { Addresses: 161.44.124.122, 64.102.6.247; Order: 200000; Zones:  }
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Docker[816]: SC database has domain name: cisco.com
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Automatic correction of the routing table has failed.  Notifying higher levels of the routing change notification for possible further corrective action.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Reconfigure reason code 5: IP forwarding table modification.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: The entire VPN connection is being reconfigured.
Feb  8 00:45:51 VHOSAKOT-M-H6X5 acvpnagent[55]: Function: OnTunnelStateChange File: ../../vpn/Agent/TND.cpp Line: 1970 tunnel state change notification (new 2, old 1)
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: VPN state: Reconnecting Network state: Network Accessible Network control state: Network Access: Restricted Network type: Untrusted
Feb  8 00:45:51 VHOSAKOT-M-H6X5 Cisco AnyConnect Secure Mobility Client[8211]: Message type information sent to the user: Reconnecting to Boxborough - SSL...

@CyberKoz CyberKoz referenced this issue in department-of-veterans-affairs/vets-api Feb 10, 2017

Closed

Added docker build and docker compose file #631

@sarusso

This comment has been minimized.

Show comment
Hide comment
@sarusso

sarusso Aug 16, 2017

This should do all the machinery required for setting up docker-machine with local port forwarding: https://github.com/onejli/docker-vpn-helper. Plus explains very well where problems are in using docker-machine with a VPN that intercepts all the traffic.

sarusso commented Aug 16, 2017

This should do all the machinery required for setting up docker-machine with local port forwarding: https://github.com/onejli/docker-vpn-helper. Plus explains very well where problems are in using docker-machine with a VPN that intercepts all the traffic.

@caleyg

This comment has been minimized.

Show comment
Hide comment

caleyg commented Aug 31, 2017

Thank you!! @volkertb

@dcp12345678

This comment has been minimized.

Show comment
Hide comment
@dcp12345678

dcp12345678 Sep 22, 2017

@sarusso - Do you have a solution for windows users?

@sarusso - Do you have a solution for windows users?

@sarusso

This comment has been minimized.

Show comment
Hide comment
@sarusso

sarusso Oct 1, 2017

The same code with minor adjustments worked for me even on Windows (with Bash). But I could not find the time to cleanly publish it here on GitHub yet..

sarusso commented Oct 1, 2017

The same code with minor adjustments worked for me even on Windows (with Bash). But I could not find the time to cleanly publish it here on GitHub yet..

@jecot

This comment has been minimized.

Show comment
Hide comment
@jecot

jecot Oct 25, 2017

@sarusso Would be interested in those 'minor adjustments' for windows user!

jecot commented Oct 25, 2017

@sarusso Would be interested in those 'minor adjustments' for windows user!

@CarlosOVillanueva

This comment has been minimized.

Show comment
Hide comment
@CarlosOVillanueva

CarlosOVillanueva Mar 15, 2018

I haven't read completely through this thread yet, but has anyone here tried to set "Allow Local (LAN) access when using VPN (if configured)" within the Cisco AnyConnect VPN client? I know that in Windows 10, this was the smoking bullet. My symptoms were simply that I could not mount a vol when AnyConnect was connected to my company VPN - even with the local firewall disabled. Enabling this setting within the AnyConnect client fixed the issue immediately.

I haven't read completely through this thread yet, but has anyone here tried to set "Allow Local (LAN) access when using VPN (if configured)" within the Cisco AnyConnect VPN client? I know that in Windows 10, this was the smoking bullet. My symptoms were simply that I could not mount a vol when AnyConnect was connected to my company VPN - even with the local firewall disabled. Enabling this setting within the AnyConnect client fixed the issue immediately.

@ceridwen

This comment has been minimized.

Show comment
Hide comment
@ceridwen

ceridwen Mar 15, 2018

That option doesn't seem to be available on the Mac OS X version of the client.

That option doesn't seem to be available on the Mac OS X version of the client.

@wglambert wglambert added the question label Jul 11, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment