Refactor scutil parts of client scripts, fix #379 #452

stek29 · 2018-04-27T11:58:39Z

Don't edit old service but create new service instead

Fix IPv6 addresses not working (macOS does resolve AAAA too now)

Fix DNS resolver for scoped queries becoming unreachable (old interface
was used with address given by vpn server, probably it could've even
caused dns leak)

Fix copy-paste error in down script

Handle dhcp-option DNS6

I have not tested it on anything other than macOS High Sierra with IPv4 only network connecting to openvpn server giving out IPv4 and IPv6 addresses. Haven't tested on older OS X or on different configurations.

Also, Probably other scripts/executables (i.e. process-network-changes) need to be updated -- please tell me if there's more stuff to fix.

IMHO these scripts need to be carefully reviewed/rewritten, with different versions for different OSVER if dropping older OS X support isn't an option.
And SystemConfiguration usage/"schema" needs to be documented too.

stek29 · 2018-04-27T12:01:02Z

Huh, meant #379 and not #440.

jkbullard · 2018-04-27T12:28:28Z

@stek29 - Thanks. This looks very interesting!

One tiny point from my first look: currently Tunnelblick supports only OS X 10.7.5 and higher. So the conditional checking for OS X 10.4 in (new) line 1172 of client.up.tunnelblick.sh isn't needed. (There may be other 10.4 - 10.6 references that should be removed, but let's not add new ones!)

stek29 · 2018-04-27T17:17:43Z

@jkbullard done :)

stek29 · 2018-05-12T08:21:01Z

bump

jkbullard · 2018-06-29T12:06:40Z

@stek29 - Thanks again for this. It points the way to do several things. However, I don't want to merge it because

It makes a lot of different, unrelated changes all in one commit; and
As you note, changes to other parts of Tunnelblick (process-network-changes) may also be needed.

I would like to take pieces of this PR and implement them separately if you don't want to, but that means that you wouldn't get "PR merged" credit on GitHub for this. Would that be OK with you, @stek29?

Edited 2018-06-29 09:48 EDT to remove "2. I think it leaves artifacts around in SC because there aren't corresponding changes to the down script;", which was just plain wrong. My apologies to @stek29.

stek29 · 2018-06-30T10:00:45Z

@jkbullard

It makes a lot of different, unrelated changes all in one commit

I disagree. All of changes made (except DNS6 maybe) just can't be separated – you can't separate full rewrite of something.

As you note, changes to other parts of Tunnelblick (process-network-changes) may also be needed.

I was only mentioning this because I was looking for feedback and hints on which parts of Tunnelblick should be changed.

I would like to take pieces of this PR and implement them separately if you don't want to, but that means that you wouldn't get "PR merged" credit on GitHub for this.

I'm OK with not getting "PR merged" credit, and I think that just picking a commit would be ok (see my comments on point 1).

Also, I've been using this for more than 2 months, and I've only encountered one issue:

Connect to VPN
Put macOS into sleep so it disconnects from WiFi
When flashing "reconnecting" window appears, click on disconnect there
Tunnelblick crashes

However, I'm not sure if this is actually caused by changed shell script. But I have not tried to put old script back and check if it still crashes.

jkbullard · 2018-06-30T10:21:19Z

@stek29 -

It makes a lot of different, unrelated changes all in one commit

I disagree. All of changes made (except DNS6 maybe) just can't be separated – you can't separate full rewrite of something.

Actually, it was the IPv6 (and the copy/paste error) that I am most interested in – that's where my comment came from : )

I would like to take pieces of this PR and implement them separately if you don't want to, but that means that you wouldn't get "PR merged" credit on GitHub for this.

I'm OK with not getting "PR merged" credit, and I think that just picking a commit would be ok

OK, thanks. I plan to do that, along with some other changes.

Thanks also for mentioning the crashes and for your clear description of when they happen. Since they were crashes of Tunnelblick itself, they probably aren't related to the up/down scripts, which are run directly by OpenVPN and would crash that process instead (although it is possible that such an unusual situation could cause Tunnelblick to crash). Sleep/wake often caused Tunnelblick crashes in versions earlier than 3.7.5beta06 build 5000 (2018-02-15) but I'll try to reproduce this since you presumably are using a more recent version.

stek29 · 2018-06-30T10:26:02Z

@jkbullard I would split it into multiple commits

stek29 · 2018-06-30T15:48:45Z

@jkbullard now it's split into ~~3 commits~~ 2 commits and additional PR (#466)

jkbullard · 2018-07-06T17:15:54Z

@stek29 - I have merged the PR that fixed the typo in the down script – thanks!

I have decided not to accept this PR, which changes the default up/down scripts, because I do not want to risk breaking existing setups.

However I am still interested in the changes and would like you to submit a new PR which will modify two new scripts that I have just committed as 35842e5: client.4.up.tunnelblick.sh and client.4.down.tunnelblick.sh. They are available as "Set nameserver (IPv6)" under the "Set DNS/WINS" setting (in anticipation of your changes that implement IPv6 support).

@stek29, I'd like you to, using the newly-updated head of master, overwrite client.4.up.tunnelblick.sh and client.4.down.tunnelblick.sh with your corresponding modified copies. (Because the existing scripts are copies of client.up.tunnelblick.sh and client.down.tunnelblick.sh, the diffs should be the same as they are now, but on different files.)

Then create a PR and I'll merge it.

Or let me know if you want me to do this myself (but you get no credit).

Again, thanks for your work on this.

jkbullard · 2018-07-07T13:27:48Z

I've done some more testing, and this fails, at least on macOS 10.11.6 – DNS stops working.

I'm not sure why, but suspect it's "Don't edit old service but create new service instead".

Don't edit old service but create new service instead Fix IPv6 addresses not working (macOS does resolve AAAA too now) Fix DNS resolver for scoped queries becoming unreachable (old interface was used with address given by vpn server, probably it could've even caused dns leak)

stek29 · 2018-07-08T14:45:32Z

@jkbullard what is the output of scutil --dns?

jkbullard · 2018-07-08T16:05:08Z

--dns shows

$ scutil --dns
DNS configuration

resolver #1
  flags    : Request A records
Not Reachable

resolver #2
  domain   : local
  options  : mdns
  timeout  : 5
  flags    : Request A records
Not Reachable
  order    : 300000

resolver #3
  domain   : 254.169.in-addr.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
Not Reachable
  order    : 300200

resolver #4
  domain   : 8.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
Not Reachable
  order    : 300400

resolver #5
  domain   : 9.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
Not Reachable
  order    : 300600

resolver #6
  domain   : a.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
Not Reachable
  order    : 300800

resolver #7
  domain   : b.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
Not Reachable
  order    : 301000

DNS configuration (for scoped queries)

resolver #1
  search domain[0] : openvpn
  nameserver[0] : 8.8.8.8
  nameserver[1] : 8.8.4.4
  if_index : 13 (utun0)
  flags    : Scoped, Request A records
Reachable

The output from the up script is

Start of output from client.4.up.tunnelblick.sh
Retrieved from OpenVPN: name server(s) [ 8.8.8.8 8.8.4.4 ], search domain(s) [  ] and SMB server(s) [  ] and using default domain name [ openvpn ]
  No such key
Will allow changes to manually-set ServerAddresses '8.8.8.8 8.8.4.4'
Setting search domains to 'openvpn' because running under OS X 10.6 or higher and the search domains were not set manually (or are allowed to be changed) and 'Prepend domain name to search domains' was not selected
NOT Setting up IPv6 service
Saved the DNS and SMB configurations so they can be restored
Did not change DNS ServerAddresses setting of '8.8.8.8 8.8.4.4' (but re-set it)
Changed DNS SearchDomains setting from '' to 'openvpn'
Changed DNS DomainName setting from '' to 'openvpn'
Did not change SMB NetBIOSName setting of ''
Did not change SMB Workgroup setting of ''
Did not change SMB WINSAddresses setting of ''
DNS servers '8.8.8.8 8.8.4.4' will be used for DNS queries when the VPN is active
The DNS servers include only free public DNS servers known to Tunnelblick.
Flushed the DNS cache via dscacheutil
/usr/sbin/discoveryutil not present. Not flushing the DNS cache via discoveryutil
Notified mDNSResponder that the DNS cache was flushed
End of output from client.4.up.tunnelblick.sh

jkbullard · 2018-07-08T16:06:50Z

@stek29 -

And here is what it is before connecting to the VPN:

$ scutil --dns
DNS configuration

resolver #1
 nameserver[0] : 8.8.8.8
 nameserver[1] : 8.8.4.4
 flags    : Request A records
Reachable

resolver #2
 domain   : local
 options  : mdns
 timeout  : 5
 flags    : Request A records
Not Reachable
 order    : 300000

resolver #3
 domain   : 254.169.in-addr.arpa
 options  : mdns
 timeout  : 5
 flags    : Request A records
Not Reachable
 order    : 300200

resolver #4
 domain   : 8.e.f.ip6.arpa
 options  : mdns
 timeout  : 5
 flags    : Request A records
Not Reachable
 order    : 300400

resolver #5
 domain   : 9.e.f.ip6.arpa
 options  : mdns
 timeout  : 5
 flags    : Request A records
Not Reachable
 order    : 300600

resolver #6
 domain   : a.e.f.ip6.arpa
 options  : mdns
 timeout  : 5
 flags    : Request A records
Not Reachable
 order    : 300800

resolver #7
 domain   : b.e.f.ip6.arpa
 options  : mdns
 timeout  : 5
 flags    : Request A records
Not Reachable
 order    : 301000

DNS configuration (for scoped queries)

resolver #1
 nameserver[0] : 8.8.8.8
 nameserver[1] : 8.8.4.4
 if_index : 4 (en0)
 flags    : Scoped, Request A records
Reachable

stek29 · 2018-07-08T19:33:16Z

Does DNS work in Google Chrome? Do you see anything from mDNSResponder in Console.app?

jkbullard · 2018-07-09T01:36:26Z

Does DNS work in Google Chrome?

No, I get "This site can’t be reached. tunnelblick.net’s server IP address could not be found."

Do you see anything from mDNSResponder in Console.app?

Only the mDNSResponder[95]: SIGHUP: Purge cache message from when Tunnelblick's up script purged the DNS cache.

stek29 · 2018-07-09T07:44:19Z

@jkbullard does dig @8.8.8.8 A tunnelblick.net work? what about host tunnelblick.net

jkbullard · 2018-07-09T10:04:09Z

dig @8.8.8.8 A tunnelblick.net works

host tunnelblick.net does not work. It says ";; connection timed out; no servers could be reached".

stek29 · 2018-07-09T15:09:50Z

Ok, I've been looking into how to get more verbose logging out of configd and IPMonitor, but it didn't work on macOS 10.13 -- maybe it would on OS X 10.11:

Run this with sudo: https://gist.github.com/stek29/2f237323214ba766a579ef85a32cdda7 (or maybe you can just create /Library/Preferences/SystemConfiguration/com.apple.IPMonitor.control.plist with needed key/value -- Verbose: true)

Then try to see if there's something interesting in console.app (filtering by "IPMonitor" or even "configd should work).

I have compared configd 802.40.13 (10.11.6) and configd 963.30.1 (10.13.3) -- but there are way too much changes to review all of them, and I can't see anything obvious there.

The only problem I can suspect is that vpn service isn't added to services list.
something like this might fix it:

Connect to VPN
show State:/Network/Global/IPv4 -- see PrimaryService which was added by Tunnelblick
show Setup:/Network/Global/IPv4 -- would print list of services in ServiceOrder field
d.init
d.add ServiceOrder * %all services from previous step and service added by Tunnelblick%
set Setup:/Network/Global/IPv4
See if DNS works, if not, try to reconnect (this time vpn service would be in serviceorder list when it's set as primary)

jkbullard · 2018-07-09T16:51:11Z

I'd like to take a step back: what is the purpose of adding a new service?

stek29 · 2018-07-09T17:01:18Z

@jkbullard it seems to be cleaner than hijacking existing service, but I see your point -- just hijacking both ipv4 and ipv6 might work too.

jkbullard · 2018-07-11T01:43:08Z

I have cherry-picked @stek29's "Handle dhcp-option DNS6" commit into the master branch as d0326c0.

jkbullard · 2018-07-11T14:29:22Z

@stek29 - Attached are the before and after outputs from scutil --dns, dig @8.8.8.8 A tunnelblick.net,host tunnelblick.net, the scutil commands, and cat /etc/resolv.conf.

Note that /etc is a symlink to /private/etc which exists but /private/etc/resolv.conf is a symlink to /private/var/run/resolv.conf which no longer exists after connecting but is restored after disconnecting!

I think this means that the SC database entries are confusing the code that generates resolv.conf, so it is not recreated (as it usually is after network changes).

Output from client.4.tunnelblick.sh:

    Retrieved from OpenVPN: name server(s) [ 8.8.8.8 8.8.4.4 ], search domain(s) [  ] and SMB server(s) [  ] and using default domain name [ openvpn ]
      No such key
    Will allow changes to manually-set ServerAddresses '8.8.8.8 8.8.4.4'
    Setting search domains to 'openvpn' because running under OS X 10.6 or higher and the search domains were not set manually (or are allowed to be changed) and 'Prepend domain name to search domains' was not selected
    NOT Setting up IPv6 service
    Saved the DNS and SMB configurations so they can be restored
    Did not change DNS ServerAddresses setting of '8.8.8.8 8.8.4.4' (but re-set it)
    Changed DNS SearchDomains setting from '' to 'openvpn'
    Changed DNS DomainName setting from '' to 'openvpn'
    Did not change SMB NetBIOSName setting of ''
    Did not change SMB Workgroup setting of ''
    Did not change SMB WINSAddresses setting of ''
    DNS servers '8.8.8.8 8.8.4.4' will be used for DNS queries when the VPN is active
    The DNS servers include only free public DNS servers known to Tunnelblick.
    Flushed the DNS cache via dscacheutil
    /usr/sbin/discoveryutil not present. Not flushing the DNS cache via discoveryutil
    Notified mDNSResponder that the DNS cache was flushed

Here's a diff -u of the before and after:

@@ -2,10 +2,8 @@
     DNS configuration
 
     resolver #1
-      nameserver[0] : 8.8.8.8
-      nameserver[1] : 8.8.4.4
       flags    : Request A records
-    Reachable
+    Not Reachable
 
     resolver #2
       domain   : local
@@ -58,9 +56,10 @@
     DNS configuration (for scoped queries)
 
     resolver #1
+      search domain[0] : openvpn
       nameserver[0] : 8.8.8.8
       nameserver[1] : 8.8.4.4
-      if_index : 4 (en0)
+      if_index : 11 (utun0)
       flags    : Scoped, Request A records
     Reachable
 
@@ -71,7 +70,7 @@
     ; (1 server found)
     ;; global options: +cmd
     ;; Got answer:
-    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55738
+    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12399
     ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
 
     ;; QUESTION SECTION:
@@ -80,24 +79,21 @@
     ;; ANSWER SECTION:
     tunnelblick.net.	299	IN	A	205.233.73.116
 
-    ;; Query time: 60 msec
+    ;; Query time: 379 msec
     ;; SERVER: 8.8.8.8#53(8.8.8.8)
-    ;; WHEN: Wed Jul 11 09:42:55 2018
+    ;; WHEN: Wed Jul 11 09:50:14 2018
     ;; MSG SIZE  rcvd: 49
 
-
 $ host tunnelblick.net
-    tunnelblick.net has address 205.233.73.116
-    tunnelblick.net mail is handled by 10 mx01.your-site.com.
-    tunnelblick.net mail is handled by 99 mx99.your-site.com.
+    ;; connection timed out; no servers could be reached
 
 
 $ scutil
     > show State:/Network/Global/IPv4
     <dictionary> {
-      PrimaryInterface : en0
-      PrimaryService : F415D2B4-2A60-477B-965E-0EC0920E4CF3
-      Router : 192.168.125.1
+      PrimaryInterface : utun0
+      PrimaryService : Tunnelblickutun0
+      Router : 0.0.0.0
     }
     > show Setup:/Network/Global/IPv4
     <dictionary> {
@@ -116,14 +112,7 @@
     >
 
 $ cat /etc/resolv.conf
-    #
-    # Mac OS X Notice
-    #
-    # This file is not used by the host name and address resolution
-    # or the DNS query routing mechanisms used by most processes on
-    # this Mac OS X system.
-    #
-    # This file is automatically generated.
-    #
-    nameserver 8.8.8.8
-    nameserver 8.8.4.4
+    cat: /etc/resolv.conf: No such file or directory

jkbullard · 2018-07-11T14:33:44Z

Oops, here are the attachments.
beforeVPN.txt
afterVPN.txt

stek29 · 2018-07-11T21:39:38Z

Well, for some reason it doesn't see any resolvers, and that's why etc/resolv isn't created -- it'd be empty anyway.

I'll try to add ipv6 support in old way with service hijacking.

theNerdFromSiliconValley · 2018-12-17T12:50:52Z

How is the issue going on People?

stek29 · 2018-12-17T21:37:44Z

@theNerdFromSiliconValley Honestly I have lost interest in it since I've switched to IPSec. Feel free to adjust the code and get some PR fixing this merged.

theNerdFromSiliconValley · 2018-12-18T01:46:45Z

noted

theNerdFromSiliconValley · 2018-12-18T07:46:01Z

"Fix IPv6 addresses not working (macOS does resolve AAAA too now)

Fix DNS resolver for scoped queries becoming unreachable (old interface
was used with address given by vpn server, probably it could've even
caused dns leak)

Fix copy-paste error in down script

Handle dhcp-option DNS6"

You wanted to fix issues like this, After your pull request other people have committed 14 commits here,
Also, your "dhcp-option DNS6" code was merged even before you mentioned in this pull request.
"dhcp-option DNS6 "* ) aNameServers[nNameServerIndex-1]="$( trim "${vOptions[nOptionIndex-1]//dhcp-option DNS6 /}" )" let nNameServerIndex++ ;;

If you can tell how many problems exist among what you mentioned earlier, it will help me to fix those issues

jkbullard changed the title ~~Refactor scutil parts of client scripts, fix #440~~ Refactor scutil parts of client scripts, fix #379 Apr 27, 2018

stek29 force-pushed the scutilipv6 branch from fb99a0a to 7f4a55b Compare April 27, 2018 17:17

stek29 force-pushed the scutilipv6 branch from 7f4a55b to 2ff414a Compare June 30, 2018 15:47

stek29 force-pushed the scutilipv6 branch from 2ff414a to 1c086a0 Compare June 30, 2018 15:50

stek29 added 2 commits July 8, 2018 17:42

Handle dhcp-option DNS6

902f8b5

stek29 force-pushed the scutilipv6 branch from 1c086a0 to 836114d Compare July 8, 2018 14:42

johankool added a commit to eduvpn/macos that referenced this pull request Aug 30, 2018

Use script from Tunnelblick/Tunnelblick#452 to enable IPv6 support

4715c1f

johankool mentioned this pull request Aug 30, 2018

AAAA records are not resolved if the underlying connection doesn't have IPv6 eduvpn/macos#47

Closed

wido mentioned this pull request Oct 4, 2018

Name resolution not reconfigured for AAAA when IPv6 active only on VPN link #490

Open

jkbullard closed this Jun 30, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor scutil parts of client scripts, fix #379 #452

Refactor scutil parts of client scripts, fix #379 #452

stek29 commented Apr 27, 2018

stek29 commented Apr 27, 2018

jkbullard commented Apr 27, 2018

stek29 commented Apr 27, 2018

stek29 commented May 12, 2018

jkbullard commented Jun 29, 2018 •

edited

stek29 commented Jun 30, 2018

jkbullard commented Jun 30, 2018

stek29 commented Jun 30, 2018

stek29 commented Jun 30, 2018 •

edited

jkbullard commented Jul 6, 2018

jkbullard commented Jul 7, 2018

stek29 commented Jul 8, 2018

jkbullard commented Jul 8, 2018

jkbullard commented Jul 8, 2018

stek29 commented Jul 8, 2018

jkbullard commented Jul 9, 2018

stek29 commented Jul 9, 2018

jkbullard commented Jul 9, 2018

stek29 commented Jul 9, 2018

jkbullard commented Jul 9, 2018

stek29 commented Jul 9, 2018

jkbullard commented Jul 11, 2018

jkbullard commented Jul 11, 2018 •

edited

jkbullard commented Jul 11, 2018

stek29 commented Jul 11, 2018

theNerdFromSiliconValley commented Dec 17, 2018

stek29 commented Dec 17, 2018

theNerdFromSiliconValley commented Dec 18, 2018

theNerdFromSiliconValley commented Dec 18, 2018 •

edited

Refactor scutil parts of client scripts, fix #379 #452

Refactor scutil parts of client scripts, fix #379 #452

Conversation

stek29 commented Apr 27, 2018

stek29 commented Apr 27, 2018

jkbullard commented Apr 27, 2018

stek29 commented Apr 27, 2018

stek29 commented May 12, 2018

jkbullard commented Jun 29, 2018 • edited

stek29 commented Jun 30, 2018

jkbullard commented Jun 30, 2018

stek29 commented Jun 30, 2018

stek29 commented Jun 30, 2018 • edited

jkbullard commented Jul 6, 2018

jkbullard commented Jul 7, 2018

stek29 commented Jul 8, 2018

jkbullard commented Jul 8, 2018

jkbullard commented Jul 8, 2018

stek29 commented Jul 8, 2018

jkbullard commented Jul 9, 2018

stek29 commented Jul 9, 2018

jkbullard commented Jul 9, 2018

stek29 commented Jul 9, 2018

jkbullard commented Jul 9, 2018

stek29 commented Jul 9, 2018

jkbullard commented Jul 11, 2018

jkbullard commented Jul 11, 2018 • edited

jkbullard commented Jul 11, 2018

stek29 commented Jul 11, 2018

theNerdFromSiliconValley commented Dec 17, 2018

stek29 commented Dec 17, 2018

theNerdFromSiliconValley commented Dec 18, 2018

theNerdFromSiliconValley commented Dec 18, 2018 • edited

jkbullard commented Jun 29, 2018 •

edited

stek29 commented Jun 30, 2018 •

edited

jkbullard commented Jul 11, 2018 •

edited

theNerdFromSiliconValley commented Dec 18, 2018 •

edited