New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: LookupTXT regards a TXT record with multiple strings as multiple records on Windows #21472

Closed
cedar10bits opened this Issue Aug 16, 2017 · 53 comments

Comments

Projects
None yet
7 participants
@cedar10bits

cedar10bits commented Aug 16, 2017

What version of Go are you using (go version)?

go1.7.6 windows/amd64

What did you do?

"example.test.local" is a domain which has a TXT record with multiple strings as below.

"v=spf1 ip4:1.2.3.4 " "~all"

I resolve TXT record of "example.test.local".

txts, err := net.LookupTXT("example.test.local")

What did you expect to see?

On Linux, behave like below(go1.7.6 linux/amd64).

  • len(txts) == 1
  • txts[0] == "v=spf1 ip4:1.2.3.4 ~all"

What did you see instead?

On Windows, behave like below.

  • len(txts) == 2
  • txts[0] == "v=spf1 ip4:1.2.3.4 "
  • txts[1] == "~all"
@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 16, 2017

Contributor
Contributor

davecheney commented Aug 16, 2017

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 16, 2017

Contributor
Contributor

davecheney commented Aug 16, 2017

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 16, 2017

Can you please include the output reported by dig.

# dig example.test.local txt
;; ANSWER SECTION:
example.test.local. 0 IN TXT "v=spf1 ip4:1.2.3.4 " "~all"

cedar10bits commented Aug 16, 2017

Can you please include the output reported by dig.

# dig example.test.local txt
;; ANSWER SECTION:
example.test.local. 0 IN TXT "v=spf1 ip4:1.2.3.4 " "~all"

@ianlancetaylor ianlancetaylor changed the title from LookupTXT regards a TXT record with multiple strings as multiple records on Windows. to net: LookupTXT regards a TXT record with multiple strings as multiple records on Windows Aug 16, 2017

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Aug 16, 2017

Contributor

Looks like on Windows DnsQuery_W returns a DNSTXTData record in which the different strings are split up in some way. Perhaps Resolver.lookupTXT in net/lookup_windows.go should be concatenating all the strings in a single TXT record.

Contributor

ianlancetaylor commented Aug 16, 2017

Looks like on Windows DnsQuery_W returns a DNSTXTData record in which the different strings are split up in some way. Perhaps Resolver.lookupTXT in net/lookup_windows.go should be concatenating all the strings in a single TXT record.

@ianlancetaylor ianlancetaylor added this to the Go1.10 milestone Aug 16, 2017

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits commented Aug 16, 2017

This patch works well.
lookup_windows.go.patch.txt

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Aug 16, 2017

Contributor

@cedar10bits Thanks. To avoid any licensing confusion, we will only look at patches that go through the regular contribution process described at https://golang.org/doc/contribute. Would you be willing to send your patch through that?

Contributor

ianlancetaylor commented Aug 16, 2017

@cedar10bits Thanks. To avoid any licensing confusion, we will only look at patches that go through the regular contribution process described at https://golang.org/doc/contribute. Would you be willing to send your patch through that?

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 16, 2017

Contributor

I've been researching this this morning and I'm not sure this txt record is well formed. Paging @miekg to issue triage. Stat.

Contributor

davecheney commented Aug 16, 2017

I've been researching this this morning and I'm not sure this txt record is well formed. Paging @miekg to issue triage. Stat.

@miekg

This comment has been minimized.

Show comment
Hide comment
@miekg

miekg Aug 17, 2017

Contributor

Relevant RFC is 1035, 1464 is EXPERIMENTAL.
Text describing the rdata of the txt record:

.  <character-string> is a single
length octet followed by that number of characters.  <character-string>
is treated as binary information, and can be up to 256 characters in
length (including the length octet).

what windows does is completely legal.
concatenating the txt segments into one won't work (and I would almost say is illegal), because this will break DNSSEC.

Sorry it took 8 hours, I was asleep

Contributor

miekg commented Aug 17, 2017

Relevant RFC is 1035, 1464 is EXPERIMENTAL.
Text describing the rdata of the txt record:

.  <character-string> is a single
length octet followed by that number of characters.  <character-string>
is treated as binary information, and can be up to 256 characters in
length (including the length octet).

what windows does is completely legal.
concatenating the txt segments into one won't work (and I would almost say is illegal), because this will break DNSSEC.

Sorry it took 8 hours, I was asleep

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 17, 2017

@miekg Thanks.
But I think this is not the reason whether what Windows does is legal or not.

. <character-string> is a single
length octet followed by that number of characters.
is treated as binary information, and can be up to 256 characters in
length (including the length octet).

The above describes only <character-string> format.
TXT record format is described as below in RFC1035.

3.3.14. TXT RDATA format

    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    /                   TXT-DATA                    /
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+

where:

TXT-DATA        One or more <character-string>s.

TXT RRs are used to hold descriptive text.  The semantics of the text
depends on the domain where it is found.

what windows does is completely legal.
concatenating the txt segments into one won't work (and I would almost say is illegal), because this will break DNSSEC.

And if what Windows does is completely legal, what Linux does will be illegal.

cedar10bits commented Aug 17, 2017

@miekg Thanks.
But I think this is not the reason whether what Windows does is legal or not.

. <character-string> is a single
length octet followed by that number of characters.
is treated as binary information, and can be up to 256 characters in
length (including the length octet).

The above describes only <character-string> format.
TXT record format is described as below in RFC1035.

3.3.14. TXT RDATA format

    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    /                   TXT-DATA                    /
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+

where:

TXT-DATA        One or more <character-string>s.

TXT RRs are used to hold descriptive text.  The semantics of the text
depends on the domain where it is found.

what windows does is completely legal.
concatenating the txt segments into one won't work (and I would almost say is illegal), because this will break DNSSEC.

And if what Windows does is completely legal, what Linux does will be illegal.

@miekg

This comment has been minimized.

Show comment
Hide comment
@miekg

miekg Aug 17, 2017

Contributor

I'm sorry, I'm lost here. This bug is about behavior on windows.

Contributor

miekg commented Aug 17, 2017

I'm sorry, I'm lost here. This bug is about behavior on windows.

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 18, 2017

I think this bug is about different behavior between on Windows and on Linux.
And I think behavior on Linux is legal.

cedar10bits commented Aug 18, 2017

I think this bug is about different behavior between on Windows and on Linux.
And I think behavior on Linux is legal.

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 21, 2017

For your reference, nslookup on Windows returns these results below.

[multiple TXT records]

> nslookup -type=TXT multi.test.local
Server:  UnKnown
Address:  XXX.XXX.XXX.XXX
multi.test.local        text =
        "v=spf1 ip4:2.3.4.5 ~all"
multi.test.local        text =
        "v=spf1 ip4:1.2.3.4 ~all"
  • regards as multiple TXT records.

[single TXT record with multiple strings]

>nslookup -type=TXT example.test.local
Server:  UnKnown
Address:  192.168.203.175
example.test.local      text =
        "v=spf1 ip4:1.2.3.4 "
        "~all"
  • regards as single TXT record with multiple strings, not as multiple TXT records.

cedar10bits commented Aug 21, 2017

For your reference, nslookup on Windows returns these results below.

[multiple TXT records]

> nslookup -type=TXT multi.test.local
Server:  UnKnown
Address:  XXX.XXX.XXX.XXX
multi.test.local        text =
        "v=spf1 ip4:2.3.4.5 ~all"
multi.test.local        text =
        "v=spf1 ip4:1.2.3.4 ~all"
  • regards as multiple TXT records.

[single TXT record with multiple strings]

>nslookup -type=TXT example.test.local
Server:  UnKnown
Address:  192.168.203.175
example.test.local      text =
        "v=spf1 ip4:1.2.3.4 "
        "~all"
  • regards as single TXT record with multiple strings, not as multiple TXT records.
@alexbrainman

This comment has been minimized.

Show comment
Hide comment
@alexbrainman

alexbrainman Aug 21, 2017

Member

I am happy to change code, but I would like to see the problem for myself first. @cedar10bits do you have an example DNS server that I can use to reproduce this problem? Or alternatively maybe someone can suggest a way to easily setup / configure this server on Windows or Linux. Thank you.

Alex

Member

alexbrainman commented Aug 21, 2017

I am happy to change code, but I would like to see the problem for myself first. @cedar10bits do you have an example DNS server that I can use to reproduce this problem? Or alternatively maybe someone can suggest a way to easily setup / configure this server on Windows or Linux. Thank you.

Alex

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 21, 2017

@cedar10bits do you have an example DNS server that I can use to reproduce this problem?

@alexbrainman I don't have my own DNS server which can access globally.
But I find reprodusable domains and I list them below. You can reproduce this problem.

  • bbc.com
  • cisco.com
  • dell.com
  • ed.gov
  • nature.com
  • opera.com
  • stanford.edu

cedar10bits commented Aug 21, 2017

@cedar10bits do you have an example DNS server that I can use to reproduce this problem?

@alexbrainman I don't have my own DNS server which can access globally.
But I find reprodusable domains and I list them below. You can reproduce this problem.

  • bbc.com
  • cisco.com
  • dell.com
  • ed.gov
  • nature.com
  • opera.com
  • stanford.edu
@alexbrainman

This comment has been minimized.

Show comment
Hide comment
@alexbrainman

alexbrainman Aug 21, 2017

Member

But I find reprodusable domains and I list them below. You can reproduce this problem.

I will try when I have some time. Thank you @cedar10bits

Alex

Member

alexbrainman commented Aug 21, 2017

But I find reprodusable domains and I list them below. You can reproduce this problem.

I will try when I have some time. Thank you @cedar10bits

Alex

@miekg

This comment has been minimized.

Show comment
Hide comment
@miekg

miekg Aug 22, 2017

Contributor

Ah yes, if the zone file has this:
"v=spf1 ip4:1.2.3.4 " "~all"

Then what happens on Windows is correct, what happen under Linux is not correct. I.e. you should receive 2 txts elements.

Do you want them to be concatenated when returned from the API? Probably not, because the splitting might be done on purpose (by the zone administrator).
Also in your example you've added a space at the end of the first piece (conveniently), if removed:
``"v=spf1 ip4:1.2.3.4" "~all"`
What are you going to do? Insert spaces while concatenating?

Contributor

miekg commented Aug 22, 2017

Ah yes, if the zone file has this:
"v=spf1 ip4:1.2.3.4 " "~all"

Then what happens on Windows is correct, what happen under Linux is not correct. I.e. you should receive 2 txts elements.

Do you want them to be concatenated when returned from the API? Probably not, because the splitting might be done on purpose (by the zone administrator).
Also in your example you've added a space at the end of the first piece (conveniently), if removed:
``"v=spf1 ip4:1.2.3.4" "~all"`
What are you going to do? Insert spaces while concatenating?

@alexbrainman

This comment has been minimized.

Show comment
Hide comment
@alexbrainman

alexbrainman Aug 23, 2017

Member

I run this program:

package main

import (
	"fmt"
	"net"
	"sort"
)

func main() {
	names := []string{
		"bbc.com",
		"cisco.com",
		"dell.com",
		"ed.gov",
		"nature.com",
		"opera.com",
		"stanford.edu",
	}
	for _, name := range names {
		value, err := net.LookupTXT(name)
		if err != nil {
			panic(err)
		}
		sort.Strings(value)
		fmt.Printf("%q -> %q\n", name, value)
	}
}

on Linux:

"bbc.com" -> ["Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg==" "MS=ms25863558" "dropbox-domain-verification=mtgv0f2pudoz" "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99 include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"]
"cisco.com" -> ["926723159-3188410" "MS=ms65960035" "docusign=5e18de8e-36d0-4a8e-8e88-b7803423fa2f" "docusign=95052c5f-a421-4594-9227-02ad2d86dfbe" "google-site-verification=K2w--6oeqrFjHfYtTsYyd2tFw7OQd6g5HJDC9UAI8Jk" "v=spf1 ip4:173.37.147.224/27 ip4:173.37.142.64/26 ip4:173.38.212.128/27 ip4:173.38.203.0/24 ip4:64.100.0.0/14 ip4:72.163.7.160/27 ip4:72.163.197.0/24 ip4:144.254.0.0/16 ip4:66.187.208.0/20 ip4:173.37.86.0/24 ip4:64.104.206.0/24 ip4:64.104.15.96/27 ip4:64.102.19.192/26 ip4:144.254.15.96/27 ip4:173.36.137.128/26 ip4:173.36.130.0/24 mx:res.cisco.com mx:sco.cisco.com ~all"]
"dell.com" -> ["google-site-verification=YogqtnoKF39HPLDsBmWJ1MEDE61FlW9mxcv8pj0YnHs" "v=spf1 ip4:143.166.85.0/24 ip4:143.166.148.0/24 ip4:211.130.110.88 ip4:143.166.82.0/24 ip4:143.166.224.0/24 ip4:202.188.162.48/26 ip4:125.252.73.0/24 ip4:211.25.230.0/24 ip4:132.237.17.0/24 ip4:68.232.153.90 ip4:68.232.149.220 ip4:68.232.153.94 ip4:68.232.149.214 ip4:68.232.153.95 ip4:68.232.149.229 ip4:68.232.153.96 ip4:68.232.149.218 ~all" "zpJgV1vrEp/xXNL+QgrMsBQZ4b4teTyu+7GveMS/VpCGC/f4UAUgBVqsp4vsdoOlTW/abanUeQZLcGV6mktRig=="]
"ed.gov" -> ["MS=ms39866724" "v=spf1 include:spf.protection.outlook.com a:crdc2011.appiancloud.com include:sendgrid.net include:spf-a.rnmk.com include:spf-b.rnmk.com include:_spf.salesforce.com ip4:165.224.212.0/23 ip4:165.224.214.0/24 ip4:165.224.20.0/23 ip4:165.224.22.0/23 ip4:165.224.210.0/23 ip4:96.127.34.41 ip6:2610:e8::/32 ip4:12.150.182.154 ip4:204.94.67.74 ip4:63.145.228.10 ip6:2600:1:0:1B:0:0:0:74/32 ip4:165.224.250.0/24 ip4:165.224.249.0/24 ~all" "xbJsf3kVj8DeH8o01d2Iq4SNhOtmJpE7ZjCYDQ0Vrv0rSAvaEuVMhQrbEL2BEm5207Vh5o3ZnxPbepaFAo35QQ=="]
"nature.com" -> ["Hello GlobalSign CEOS1602043687" "google-site-verification=ieCNIjjta99aFWzeD9Mhze-lTbXHZo6GAc-MgXlclL4" "v=spf1 ip4:31.221.90.56/32 ip4:195.128.10.18/32 ip4:31.221.90.43/32 ip4:195.128.10.15/32 ip4:195.128.10.69/32 ip4:203.200.192.105/32 ip4:203.200.192.109/32 ip4:167.89.16.99/32 ip4:31.221.90.34/32 ip4:31.221.90.41/32 ip4:208.85.55.170/32 ip4:208.85.55.173/32 include:madgexjb.com include:_spf.google.com include:spf.mandrillapp.com include:_spf.sparkpostmail.com -all"]
"opera.com" -> ["google-site-verification=zsu8s2znTOAuZ0dkksfMdQE3HmkoNNyuijNib1xkiQo" "v=spf1 ip4:185.26.183.134 ip6:2001:4c28:4000:727:185:26:183:134 ip4:185.26.183.145 ip4:185.26.183.176 ip4:185.26.183.153  ip6:2001:4c28:4000:727:185:26:183:176 ip6:2001:4c28:4000:727:185:26:183:153 ip4:91.203.97.254 ip4:213.236.208.188 ip4:50.97.60.135 ip4:184.173.169.88 ip4:107.167.119.206 ip4:91.123.56.240 ip4:91.123.56.241 ip4:193.75.92.155 ip4:193.75.92.156 mx include:_spf.google.com -all"]
"stanford.edu" -> ["blitz=mu-0e24b17b-23dce65a-c0b14b75-e742cd66" "bwZknJMLNV1XuaJAyUmKlAFrdZo+p5yDlTNACmDUWhgtyihfJc8oWMnK7hWLreN+ozU3mX91yHHzZx0adJPPPg==" "firebase=cartalab" "globalsign-domain-verification=IKP42b_14E6c1gUHTYKgXqj3o5aasL2hd9ECM1tSJP" "htBOpqv5il135Xfa09GRiVHY09Xb9qXNj6o5lAfj8gNoBvyyGu5rSb4gvSj9lCUwY1NF0+wL2BO1ZDIcGIpADQ==" "v=spf1 ip4:171.67.219.64/27 ip4:171.67.224.0/28 ip4:171.67.214.160/27 ip4:171.67.216.240/28 ip4:171.67.43.137 ip4:171.67.43.138 ip4:171.67.43.139 ip4:171.67.43.140 ip4:171.67.43.141 ip4:204.63.229.192/28 include:_spf.google.com include:_spf.qualtrics.com include:icpbounce.com ip4:148.163.149.245 ip4:148.163.153.235 ?all"]

and on Windows:

"bbc.com" -> [" include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all" "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg==" "MS=ms25863558" "dropbox-domain-verification=mtgv0f2pudoz" "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99"]
"cisco.com" -> [" ip4:64.104.206.0/24 ip4:64.104.15.96/27 ip4:64.102.19.192/26 ip4:144.254.15.96/27 ip4:173.36.137.128/26 ip4:173.36.130.0/24 mx:res.cisco.com mx:sco.cisco.com ~all" "926723159-3188410" "MS=ms65960035" "docusign=5e18de8e-36d0-4a8e-8e88-b7803423fa2f" "docusign=95052c5f-a421-4594-9227-02ad2d86dfbe" "google-site-verification=K2w--6oeqrFjHfYtTsYyd2tFw7OQd6g5HJDC9UAI8Jk" "v=spf1 ip4:173.37.147.224/27 ip4:173.37.142.64/26 ip4:173.38.212.128/27 ip4:173.38.203.0/24 ip4:64.100.0.0/14 ip4:72.163.7.160/27 ip4:72.163.197.0/24 ip4:144.254.0.0/16 ip4:66.187.208.0/20 ip4:173.37.86.0/24"]
"dell.com" -> [".149.214 ip4:68.232.153.95 ip4:68.232.149.229 ip4:68.232.153.96 ip4:68.232.149.218 ~all" "google-site-verification=YogqtnoKF39HPLDsBmWJ1MEDE61FlW9mxcv8pj0YnHs" "v=spf1 ip4:143.166.85.0/24 ip4:143.166.148.0/24 ip4:211.130.110.88 ip4:143.166.82.0/24 ip4:143.166.224.0/24 ip4:202.188.162.48/26 ip4:125.252.73.0/24 ip4:211.25.230.0/24 ip4:132.237.17.0/24 ip4:68.232.153.90 ip4:68.232.149.220 ip4:68.232.153.94 ip4:68.232" "zpJgV1vrEp/xXNL+QgrMsBQZ4b4teTyu+7GveMS/VpCGC/f4UAUgBVqsp4vsdoOlTW/abanUeQZLcGV6mktRig=="]
"ed.gov" -> ["24.210.0/23 ip4:96.127.34.41 ip6:2610:e8::/32 ip4:12.150.182.154 ip4:204.94.67.74 ip4:63.145.228.10 ip6:2600:1:0:1B:0:0:0:74/32 ip4:165.224.250.0/24 ip4:165.224.249.0/24 ~all" "MS=ms39866724" "v=spf1 include:spf.protection.outlook.com a:crdc2011.appiancloud.com include:sendgrid.net include:spf-a.rnmk.com include:spf-b.rnmk.com include:_spf.salesforce.com ip4:165.224.212.0/23 ip4:165.224.214.0/24 ip4:165.224.20.0/23 ip4:165.224.22.0/23 ip4:165.2" "xbJsf3kVj8DeH8o01d2Iq4SNhOtmJpE7ZjCYDQ0Vrv0rSAvaEuVMhQrbEL2BEm5207Vh5o3ZnxPbepaFAo35QQ=="]
"nature.com" -> ["Hello GlobalSign CEOS1602043687" "google-site-verification=ieCNIjjta99aFWzeD9Mhze-lTbXHZo6GAc-MgXlclL4" "ip4:31.221.90.34/32 ip4:31.221.90.41/32 ip4:208.85.55.170/32 ip4:208.85.55.173/32 include:madgexjb.com include:_spf.google.com include:spf.mandrillapp.com include:_spf.sparkpostmail.com -all" "v=spf1 ip4:31.221.90.56/32 ip4:195.128.10.18/32 ip4:31.221.90.43/32 ip4:195.128.10.15/32 ip4:195.128.10.69/32 ip4:203.200.192.105/32 ip4:203.200.192.109/32 ip4:167.89.16.99/32 "]
"opera.com" -> ["4:184.173.169.88 ip4:107.167.119.206 ip4:91.123.56.240 ip4:91.123.56.241 ip4:193.75.92.155 ip4:193.75.92.156 mx include:_spf.google.com -all" "google-site-verification=zsu8s2znTOAuZ0dkksfMdQE3HmkoNNyuijNib1xkiQo" "v=spf1 ip4:185.26.183.134 ip6:2001:4c28:4000:727:185:26:183:134 ip4:185.26.183.145 ip4:185.26.183.176 ip4:185.26.183.153  ip6:2001:4c28:4000:727:185:26:183:176 ip6:2001:4c28:4000:727:185:26:183:153 ip4:91.203.97.254 ip4:213.236.208.188 ip4:50.97.60.135 ip"]
"stanford.edu" -> ["blitz=mu-0e24b17b-23dce65a-c0b14b75-e742cd66" "bwZknJMLNV1XuaJAyUmKlAFrdZo+p5yDlTNACmDUWhgtyihfJc8oWMnK7hWLreN+ozU3mX91yHHzZx0adJPPPg==" "firebase=cartalab" "globalsign-domain-verification=IKP42b_14E6c1gUHTYKgXqj3o5aasL2hd9ECM1tSJP" "htBOpqv5il135Xfa09GRiVHY09Xb9qXNj6o5lAfj8gNoBvyyGu5rSb4gvSj9lCUwY1NF0+wL2BO1ZDIcGIpADQ==" "include:_spf.google.com include:_spf.qualtrics.com include:icpbounce.com ip4:148.163.149.245 ip4:148.163.153.235 ?all" "v=spf1 ip4:171.67.219.64/27 ip4:171.67.224.0/28 ip4:171.67.214.160/27 ip4:171.67.216.240/28 ip4:171.67.43.137 ip4:171.67.43.138 ip4:171.67.43.139 ip4:171.67.43.140 ip4:171.67.43.141 ip4:204.63.229.192/28 "]

@miekg which version needs to be fixed? I suspect Windows is wrong here. Thank you.

Alex

Member

alexbrainman commented Aug 23, 2017

I run this program:

package main

import (
	"fmt"
	"net"
	"sort"
)

func main() {
	names := []string{
		"bbc.com",
		"cisco.com",
		"dell.com",
		"ed.gov",
		"nature.com",
		"opera.com",
		"stanford.edu",
	}
	for _, name := range names {
		value, err := net.LookupTXT(name)
		if err != nil {
			panic(err)
		}
		sort.Strings(value)
		fmt.Printf("%q -> %q\n", name, value)
	}
}

on Linux:

"bbc.com" -> ["Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg==" "MS=ms25863558" "dropbox-domain-verification=mtgv0f2pudoz" "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99 include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"]
"cisco.com" -> ["926723159-3188410" "MS=ms65960035" "docusign=5e18de8e-36d0-4a8e-8e88-b7803423fa2f" "docusign=95052c5f-a421-4594-9227-02ad2d86dfbe" "google-site-verification=K2w--6oeqrFjHfYtTsYyd2tFw7OQd6g5HJDC9UAI8Jk" "v=spf1 ip4:173.37.147.224/27 ip4:173.37.142.64/26 ip4:173.38.212.128/27 ip4:173.38.203.0/24 ip4:64.100.0.0/14 ip4:72.163.7.160/27 ip4:72.163.197.0/24 ip4:144.254.0.0/16 ip4:66.187.208.0/20 ip4:173.37.86.0/24 ip4:64.104.206.0/24 ip4:64.104.15.96/27 ip4:64.102.19.192/26 ip4:144.254.15.96/27 ip4:173.36.137.128/26 ip4:173.36.130.0/24 mx:res.cisco.com mx:sco.cisco.com ~all"]
"dell.com" -> ["google-site-verification=YogqtnoKF39HPLDsBmWJ1MEDE61FlW9mxcv8pj0YnHs" "v=spf1 ip4:143.166.85.0/24 ip4:143.166.148.0/24 ip4:211.130.110.88 ip4:143.166.82.0/24 ip4:143.166.224.0/24 ip4:202.188.162.48/26 ip4:125.252.73.0/24 ip4:211.25.230.0/24 ip4:132.237.17.0/24 ip4:68.232.153.90 ip4:68.232.149.220 ip4:68.232.153.94 ip4:68.232.149.214 ip4:68.232.153.95 ip4:68.232.149.229 ip4:68.232.153.96 ip4:68.232.149.218 ~all" "zpJgV1vrEp/xXNL+QgrMsBQZ4b4teTyu+7GveMS/VpCGC/f4UAUgBVqsp4vsdoOlTW/abanUeQZLcGV6mktRig=="]
"ed.gov" -> ["MS=ms39866724" "v=spf1 include:spf.protection.outlook.com a:crdc2011.appiancloud.com include:sendgrid.net include:spf-a.rnmk.com include:spf-b.rnmk.com include:_spf.salesforce.com ip4:165.224.212.0/23 ip4:165.224.214.0/24 ip4:165.224.20.0/23 ip4:165.224.22.0/23 ip4:165.224.210.0/23 ip4:96.127.34.41 ip6:2610:e8::/32 ip4:12.150.182.154 ip4:204.94.67.74 ip4:63.145.228.10 ip6:2600:1:0:1B:0:0:0:74/32 ip4:165.224.250.0/24 ip4:165.224.249.0/24 ~all" "xbJsf3kVj8DeH8o01d2Iq4SNhOtmJpE7ZjCYDQ0Vrv0rSAvaEuVMhQrbEL2BEm5207Vh5o3ZnxPbepaFAo35QQ=="]
"nature.com" -> ["Hello GlobalSign CEOS1602043687" "google-site-verification=ieCNIjjta99aFWzeD9Mhze-lTbXHZo6GAc-MgXlclL4" "v=spf1 ip4:31.221.90.56/32 ip4:195.128.10.18/32 ip4:31.221.90.43/32 ip4:195.128.10.15/32 ip4:195.128.10.69/32 ip4:203.200.192.105/32 ip4:203.200.192.109/32 ip4:167.89.16.99/32 ip4:31.221.90.34/32 ip4:31.221.90.41/32 ip4:208.85.55.170/32 ip4:208.85.55.173/32 include:madgexjb.com include:_spf.google.com include:spf.mandrillapp.com include:_spf.sparkpostmail.com -all"]
"opera.com" -> ["google-site-verification=zsu8s2znTOAuZ0dkksfMdQE3HmkoNNyuijNib1xkiQo" "v=spf1 ip4:185.26.183.134 ip6:2001:4c28:4000:727:185:26:183:134 ip4:185.26.183.145 ip4:185.26.183.176 ip4:185.26.183.153  ip6:2001:4c28:4000:727:185:26:183:176 ip6:2001:4c28:4000:727:185:26:183:153 ip4:91.203.97.254 ip4:213.236.208.188 ip4:50.97.60.135 ip4:184.173.169.88 ip4:107.167.119.206 ip4:91.123.56.240 ip4:91.123.56.241 ip4:193.75.92.155 ip4:193.75.92.156 mx include:_spf.google.com -all"]
"stanford.edu" -> ["blitz=mu-0e24b17b-23dce65a-c0b14b75-e742cd66" "bwZknJMLNV1XuaJAyUmKlAFrdZo+p5yDlTNACmDUWhgtyihfJc8oWMnK7hWLreN+ozU3mX91yHHzZx0adJPPPg==" "firebase=cartalab" "globalsign-domain-verification=IKP42b_14E6c1gUHTYKgXqj3o5aasL2hd9ECM1tSJP" "htBOpqv5il135Xfa09GRiVHY09Xb9qXNj6o5lAfj8gNoBvyyGu5rSb4gvSj9lCUwY1NF0+wL2BO1ZDIcGIpADQ==" "v=spf1 ip4:171.67.219.64/27 ip4:171.67.224.0/28 ip4:171.67.214.160/27 ip4:171.67.216.240/28 ip4:171.67.43.137 ip4:171.67.43.138 ip4:171.67.43.139 ip4:171.67.43.140 ip4:171.67.43.141 ip4:204.63.229.192/28 include:_spf.google.com include:_spf.qualtrics.com include:icpbounce.com ip4:148.163.149.245 ip4:148.163.153.235 ?all"]

and on Windows:

"bbc.com" -> [" include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all" "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg==" "MS=ms25863558" "dropbox-domain-verification=mtgv0f2pudoz" "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99"]
"cisco.com" -> [" ip4:64.104.206.0/24 ip4:64.104.15.96/27 ip4:64.102.19.192/26 ip4:144.254.15.96/27 ip4:173.36.137.128/26 ip4:173.36.130.0/24 mx:res.cisco.com mx:sco.cisco.com ~all" "926723159-3188410" "MS=ms65960035" "docusign=5e18de8e-36d0-4a8e-8e88-b7803423fa2f" "docusign=95052c5f-a421-4594-9227-02ad2d86dfbe" "google-site-verification=K2w--6oeqrFjHfYtTsYyd2tFw7OQd6g5HJDC9UAI8Jk" "v=spf1 ip4:173.37.147.224/27 ip4:173.37.142.64/26 ip4:173.38.212.128/27 ip4:173.38.203.0/24 ip4:64.100.0.0/14 ip4:72.163.7.160/27 ip4:72.163.197.0/24 ip4:144.254.0.0/16 ip4:66.187.208.0/20 ip4:173.37.86.0/24"]
"dell.com" -> [".149.214 ip4:68.232.153.95 ip4:68.232.149.229 ip4:68.232.153.96 ip4:68.232.149.218 ~all" "google-site-verification=YogqtnoKF39HPLDsBmWJ1MEDE61FlW9mxcv8pj0YnHs" "v=spf1 ip4:143.166.85.0/24 ip4:143.166.148.0/24 ip4:211.130.110.88 ip4:143.166.82.0/24 ip4:143.166.224.0/24 ip4:202.188.162.48/26 ip4:125.252.73.0/24 ip4:211.25.230.0/24 ip4:132.237.17.0/24 ip4:68.232.153.90 ip4:68.232.149.220 ip4:68.232.153.94 ip4:68.232" "zpJgV1vrEp/xXNL+QgrMsBQZ4b4teTyu+7GveMS/VpCGC/f4UAUgBVqsp4vsdoOlTW/abanUeQZLcGV6mktRig=="]
"ed.gov" -> ["24.210.0/23 ip4:96.127.34.41 ip6:2610:e8::/32 ip4:12.150.182.154 ip4:204.94.67.74 ip4:63.145.228.10 ip6:2600:1:0:1B:0:0:0:74/32 ip4:165.224.250.0/24 ip4:165.224.249.0/24 ~all" "MS=ms39866724" "v=spf1 include:spf.protection.outlook.com a:crdc2011.appiancloud.com include:sendgrid.net include:spf-a.rnmk.com include:spf-b.rnmk.com include:_spf.salesforce.com ip4:165.224.212.0/23 ip4:165.224.214.0/24 ip4:165.224.20.0/23 ip4:165.224.22.0/23 ip4:165.2" "xbJsf3kVj8DeH8o01d2Iq4SNhOtmJpE7ZjCYDQ0Vrv0rSAvaEuVMhQrbEL2BEm5207Vh5o3ZnxPbepaFAo35QQ=="]
"nature.com" -> ["Hello GlobalSign CEOS1602043687" "google-site-verification=ieCNIjjta99aFWzeD9Mhze-lTbXHZo6GAc-MgXlclL4" "ip4:31.221.90.34/32 ip4:31.221.90.41/32 ip4:208.85.55.170/32 ip4:208.85.55.173/32 include:madgexjb.com include:_spf.google.com include:spf.mandrillapp.com include:_spf.sparkpostmail.com -all" "v=spf1 ip4:31.221.90.56/32 ip4:195.128.10.18/32 ip4:31.221.90.43/32 ip4:195.128.10.15/32 ip4:195.128.10.69/32 ip4:203.200.192.105/32 ip4:203.200.192.109/32 ip4:167.89.16.99/32 "]
"opera.com" -> ["4:184.173.169.88 ip4:107.167.119.206 ip4:91.123.56.240 ip4:91.123.56.241 ip4:193.75.92.155 ip4:193.75.92.156 mx include:_spf.google.com -all" "google-site-verification=zsu8s2znTOAuZ0dkksfMdQE3HmkoNNyuijNib1xkiQo" "v=spf1 ip4:185.26.183.134 ip6:2001:4c28:4000:727:185:26:183:134 ip4:185.26.183.145 ip4:185.26.183.176 ip4:185.26.183.153  ip6:2001:4c28:4000:727:185:26:183:176 ip6:2001:4c28:4000:727:185:26:183:153 ip4:91.203.97.254 ip4:213.236.208.188 ip4:50.97.60.135 ip"]
"stanford.edu" -> ["blitz=mu-0e24b17b-23dce65a-c0b14b75-e742cd66" "bwZknJMLNV1XuaJAyUmKlAFrdZo+p5yDlTNACmDUWhgtyihfJc8oWMnK7hWLreN+ozU3mX91yHHzZx0adJPPPg==" "firebase=cartalab" "globalsign-domain-verification=IKP42b_14E6c1gUHTYKgXqj3o5aasL2hd9ECM1tSJP" "htBOpqv5il135Xfa09GRiVHY09Xb9qXNj6o5lAfj8gNoBvyyGu5rSb4gvSj9lCUwY1NF0+wL2BO1ZDIcGIpADQ==" "include:_spf.google.com include:_spf.qualtrics.com include:icpbounce.com ip4:148.163.149.245 ip4:148.163.153.235 ?all" "v=spf1 ip4:171.67.219.64/27 ip4:171.67.224.0/28 ip4:171.67.214.160/27 ip4:171.67.216.240/28 ip4:171.67.43.137 ip4:171.67.43.138 ip4:171.67.43.139 ip4:171.67.43.140 ip4:171.67.43.141 ip4:204.63.229.192/28 "]

@miekg which version needs to be fixed? I suspect Windows is wrong here. Thank you.

Alex

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 23, 2017

Do you want them to be concatenated when returned from the API? Probably not, because the splitting might be done on purpose (by the zone administrator).

@miekg I think so, too. I think this API should be defined as below, ideally.

type TXTRecord struct {
    Txt []string
}
func LookupTXT(name string) ([]TXTRecord, error)

But this API is defined as a function that returns multiple TXT records of a string array (https://golang.org/pkg/net/#LookupTXT).
@alexbrainman shows difference between Windows and Linux.
On Windows, a record with multiple strings is treated as multiple records with single string.
I think this behavior is not followed by the definition of this API.

cedar10bits commented Aug 23, 2017

Do you want them to be concatenated when returned from the API? Probably not, because the splitting might be done on purpose (by the zone administrator).

@miekg I think so, too. I think this API should be defined as below, ideally.

type TXTRecord struct {
    Txt []string
}
func LookupTXT(name string) ([]TXTRecord, error)

But this API is defined as a function that returns multiple TXT records of a string array (https://golang.org/pkg/net/#LookupTXT).
@alexbrainman shows difference between Windows and Linux.
On Windows, a record with multiple strings is treated as multiple records with single string.
I think this behavior is not followed by the definition of this API.

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 23, 2017

Also in your example you've added a space at the end of the first piece (conveniently), if removed:
``"v=spf1 ip4:1.2.3.4" "~all"`
What are you going to do? Insert spaces while concatenating?

@miekg If removed, just concatenate them without adding spaces.
In this case, I expect "v=spf1 ip4:1.2.3.4~all" (no space between "1.2.3.4" and "~all").
It is useful for some cases, for example, SPF (https://tools.ietf.org/html/rfc7208).

3.3.  Multiple Strings in a Single DNS Record

   As defined in [RFC1035], Sections 3.3 and 3.3.14, a single text DNS
   record can be composed of more than one string.  If a published
   record contains multiple character-strings, then the record MUST be
   treated as if those strings are concatenated together without adding
   spaces.  For example:

      IN TXT "v=spf1 .... first" "second string..."

   is equivalent to:

      IN TXT "v=spf1 .... firstsecond string..."

cedar10bits commented Aug 23, 2017

Also in your example you've added a space at the end of the first piece (conveniently), if removed:
``"v=spf1 ip4:1.2.3.4" "~all"`
What are you going to do? Insert spaces while concatenating?

@miekg If removed, just concatenate them without adding spaces.
In this case, I expect "v=spf1 ip4:1.2.3.4~all" (no space between "1.2.3.4" and "~all").
It is useful for some cases, for example, SPF (https://tools.ietf.org/html/rfc7208).

3.3.  Multiple Strings in a Single DNS Record

   As defined in [RFC1035], Sections 3.3 and 3.3.14, a single text DNS
   record can be composed of more than one string.  If a published
   record contains multiple character-strings, then the record MUST be
   treated as if those strings are concatenated together without adding
   spaces.  For example:

      IN TXT "v=spf1 .... first" "second string..."

   is equivalent to:

      IN TXT "v=spf1 .... firstsecond string..."
@miekg

This comment has been minimized.

Show comment
Hide comment
@miekg

miekg Aug 23, 2017

Contributor

But LookupTXT has no idea it deals with SPF records (oh if only they had used a new record type...), so following those rules in 7208 doesn't work for all cases.

@alexbrainman using

package main

import (
	"fmt"
	"net"
	"sort"
)

func main() {
	names := []string{
		"bbc.com",
		"dell.com",
	}
	for _, name := range names {
		value, err := net.LookupTXT(name)
		if err != nil {
			panic(err)
		}
		sort.Strings(value)

		fmt.Println(name)
		for _, v := range value {
			fmt.Printf("  * %q\n", v)
		}
	}
}

Building this with CGO_ENABLED=0 go build lookup.go as we don't want to involve glibc here (assuming)
And checking what this gives (on linux) does have:

bbc.com
  * "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
  * "MS=ms25863558"
  * "dropbox-domain-verification=mtgv0f2pudoz"
  * "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99 include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"

Checking with dig:

;bbc.com.			IN	TXT

;; ANSWER SECTION:
bbc.com.		497	IN	TXT	"Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
bbc.com.		497	IN	TXT	"MS=ms25863558"
bbc.com.		497	IN	TXT	"dropbox-domain-verification=mtgv0f2pudoz"
bbc.com.		497	IN	TXT	"v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99" " include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"

So indeed on Linux this look OK (Don't own Windows - so can't check there).

What's wrong with Windows? The order of the different txt segments? (At least that is what I can make out).
There is no inherent order of these TXT records, the can be shuffled around as you please. If you want to insists there is only 1 ordering: the canonical DNS ordering (RFC 4034).

It could be that on the windows tests the TXT order is different (unlikely). What does dig on windows show? Is Go doing some sorting (on Windows)?

Contributor

miekg commented Aug 23, 2017

But LookupTXT has no idea it deals with SPF records (oh if only they had used a new record type...), so following those rules in 7208 doesn't work for all cases.

@alexbrainman using

package main

import (
	"fmt"
	"net"
	"sort"
)

func main() {
	names := []string{
		"bbc.com",
		"dell.com",
	}
	for _, name := range names {
		value, err := net.LookupTXT(name)
		if err != nil {
			panic(err)
		}
		sort.Strings(value)

		fmt.Println(name)
		for _, v := range value {
			fmt.Printf("  * %q\n", v)
		}
	}
}

Building this with CGO_ENABLED=0 go build lookup.go as we don't want to involve glibc here (assuming)
And checking what this gives (on linux) does have:

bbc.com
  * "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
  * "MS=ms25863558"
  * "dropbox-domain-verification=mtgv0f2pudoz"
  * "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99 include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"

Checking with dig:

;bbc.com.			IN	TXT

;; ANSWER SECTION:
bbc.com.		497	IN	TXT	"Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
bbc.com.		497	IN	TXT	"MS=ms25863558"
bbc.com.		497	IN	TXT	"dropbox-domain-verification=mtgv0f2pudoz"
bbc.com.		497	IN	TXT	"v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99" " include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"

So indeed on Linux this look OK (Don't own Windows - so can't check there).

What's wrong with Windows? The order of the different txt segments? (At least that is what I can make out).
There is no inherent order of these TXT records, the can be shuffled around as you please. If you want to insists there is only 1 ordering: the canonical DNS ordering (RFC 4034).

It could be that on the windows tests the TXT order is different (unlikely). What does dig on windows show? Is Go doing some sorting (on Windows)?

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 23, 2017

But LookupTXT has no idea it deals with SPF records (oh if only they had used a new record type...), so following those rules in 7208 doesn't work for all cases.

I know those rules are only for SPF (multiple strings), not for TXT records. That's OK.

Here is what nslookup gives on Windows (instead of dig).

>nslookup -type=TXT bbc.com
Server:  UnKnown
Address:  XXX.XXX.XXX.XXX

Non-authoritative answer:
bbc.com text =

        "MS=ms25863558"
bbc.com text =

        "dropbox-domain-verification=mtgv0f2pudoz"
bbc.com text =

        "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
bbc.com text =

        "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99"
        " include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"

@alexbrainman using

package main

import (
	"fmt"
	"net"
	// "sort"
)

func main() {
	names := []string{
		"bbc.com",
		"dell.com",
	}
	for _, name := range names {
		value, err := net.LookupTXT(name)
		if err != nil {
			panic(err)
		}
		// sort.Strings(value)

		fmt.Println(name)
		for _, v := range value {
			fmt.Printf("  * %q\n", v)
		}
	}
}

What this gives on Linux (CGO_ENABLED=0) is below:

bbc.com
  * "MS=ms25863558"
  * "dropbox-domain-verification=mtgv0f2pudoz"
  * "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
  * "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99 include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"

And what this gives on Windows is below:

bbc.com
  * "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
  * "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99"
  * " include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"
  * "MS=ms25863558"
  * "dropbox-domain-verification=mtgv0f2pudoz"

On Windows, "v=spf1 ... " and " include: ... " are regarded as two other TXT records (not single TXT record).
Order of TXT records is not stable (on both OS), but " include: ... " always follows "v=spf1 ... " on Windows.

cedar10bits commented Aug 23, 2017

But LookupTXT has no idea it deals with SPF records (oh if only they had used a new record type...), so following those rules in 7208 doesn't work for all cases.

I know those rules are only for SPF (multiple strings), not for TXT records. That's OK.

Here is what nslookup gives on Windows (instead of dig).

>nslookup -type=TXT bbc.com
Server:  UnKnown
Address:  XXX.XXX.XXX.XXX

Non-authoritative answer:
bbc.com text =

        "MS=ms25863558"
bbc.com text =

        "dropbox-domain-verification=mtgv0f2pudoz"
bbc.com text =

        "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
bbc.com text =

        "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99"
        " include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"

@alexbrainman using

package main

import (
	"fmt"
	"net"
	// "sort"
)

func main() {
	names := []string{
		"bbc.com",
		"dell.com",
	}
	for _, name := range names {
		value, err := net.LookupTXT(name)
		if err != nil {
			panic(err)
		}
		// sort.Strings(value)

		fmt.Println(name)
		for _, v := range value {
			fmt.Printf("  * %q\n", v)
		}
	}
}

What this gives on Linux (CGO_ENABLED=0) is below:

bbc.com
  * "MS=ms25863558"
  * "dropbox-domain-verification=mtgv0f2pudoz"
  * "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
  * "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99 include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"

And what this gives on Windows is below:

bbc.com
  * "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
  * "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99"
  * " include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"
  * "MS=ms25863558"
  * "dropbox-domain-verification=mtgv0f2pudoz"

On Windows, "v=spf1 ... " and " include: ... " are regarded as two other TXT records (not single TXT record).
Order of TXT records is not stable (on both OS), but " include: ... " always follows "v=spf1 ... " on Windows.

@miekg

This comment has been minimized.

Show comment
Hide comment
@miekg

miekg Aug 23, 2017

Contributor

ah thanks @cedar10bits now I see (these strings are too long).
On linux the returned segment has a length > 255 (which can't possibly be right if you follow the letter of the RFC). In windows the 2 segments of that TXT record v=spf1 are correctly put into two text elements.

Again Windows code seems correct to me, what happens on Linux is somewhat fishy.

Contributor

miekg commented Aug 23, 2017

ah thanks @cedar10bits now I see (these strings are too long).
On linux the returned segment has a length > 255 (which can't possibly be right if you follow the letter of the RFC). In windows the 2 segments of that TXT record v=spf1 are correctly put into two text elements.

Again Windows code seems correct to me, what happens on Linux is somewhat fishy.

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 23, 2017

This API is defined as below (https://golang.org/pkg/net/#LookupTXT).

LookupTXT returns the DNS TXT records for the given domain name. 

This API returns TXT records as a string array.
I think that one element of this string array must be single whole TXT record, not a part of TXT record.
v=spf1 is a part of TXT record, but not single whole TXT record.
If Windows code is correct, behavior on Windows is not followed by this API's definition.

cedar10bits commented Aug 23, 2017

This API is defined as below (https://golang.org/pkg/net/#LookupTXT).

LookupTXT returns the DNS TXT records for the given domain name. 

This API returns TXT records as a string array.
I think that one element of this string array must be single whole TXT record, not a part of TXT record.
v=spf1 is a part of TXT record, but not single whole TXT record.
If Windows code is correct, behavior on Windows is not followed by this API's definition.

@miekg

This comment has been minimized.

Show comment
Hide comment
@miekg

miekg Aug 23, 2017

Contributor
Contributor

miekg commented Aug 23, 2017

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 23, 2017

If what Windows does is correct, I will have no idea to distinguish between single whole TXT record and a part of TXT record.
To use this API correctly, I need to distinguish.
@miekg do you have any idea ?

cedar10bits commented Aug 23, 2017

If what Windows does is correct, I will have no idea to distinguish between single whole TXT record and a part of TXT record.
To use this API correctly, I need to distinguish.
@miekg do you have any idea ?

@miekg

This comment has been minimized.

Show comment
Hide comment
@miekg

miekg Aug 23, 2017

Contributor
Contributor

miekg commented Aug 23, 2017

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 23, 2017

It changes what the zone administrator originally put in the zone.

I think both behavior on Windows and Linux change what the zone administrator originally put in the zone.
@miekg what do you think about it ?

cedar10bits commented Aug 23, 2017

It changes what the zone administrator originally put in the zone.

I think both behavior on Windows and Linux change what the zone administrator originally put in the zone.
@miekg what do you think about it ?

@miekg

This comment has been minimized.

Show comment
Hide comment
@miekg

miekg Aug 23, 2017

Contributor
Contributor

miekg commented Aug 23, 2017

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 24, 2017

To understand DNS standpoint, I study DNS zone file.
I want to check whether my understandings is correct or not.

Zone file of bbc.com may be written like below (TXT record only):

@       IN  TXT     "MS=ms25863558"
@       IN  TXT     "dropbox-domain-verification=mtgv0f2pudoz"
@       IN  TXT     "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
@       IN  TXT     (   "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99"
                        " include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all")

On Windows, this API regards zone file as below:

@       IN  TXT     "MS=ms25863558"
@       IN  TXT     "dropbox-domain-verification=mtgv0f2pudoz"
@       IN  TXT     "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
@       IN  TXT     "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99"
@       IN  TXT     " include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all" 

And on Linux, this API regards as below (syntax error occurs on BIND because of too long strings):

@       IN  TXT     "MS=ms25863558"
@       IN  TXT     "dropbox-domain-verification=mtgv0f2pudoz"
@       IN  TXT     "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
@       IN  TXT     "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99 include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"

Is the above correct ?
If correct, both versions change what the zone administrator originally put in the zone.

cedar10bits commented Aug 24, 2017

To understand DNS standpoint, I study DNS zone file.
I want to check whether my understandings is correct or not.

Zone file of bbc.com may be written like below (TXT record only):

@       IN  TXT     "MS=ms25863558"
@       IN  TXT     "dropbox-domain-verification=mtgv0f2pudoz"
@       IN  TXT     "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
@       IN  TXT     (   "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99"
                        " include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all")

On Windows, this API regards zone file as below:

@       IN  TXT     "MS=ms25863558"
@       IN  TXT     "dropbox-domain-verification=mtgv0f2pudoz"
@       IN  TXT     "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
@       IN  TXT     "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99"
@       IN  TXT     " include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all" 

And on Linux, this API regards as below (syntax error occurs on BIND because of too long strings):

@       IN  TXT     "MS=ms25863558"
@       IN  TXT     "dropbox-domain-verification=mtgv0f2pudoz"
@       IN  TXT     "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="
@       IN  TXT     "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99 include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"

Is the above correct ?
If correct, both versions change what the zone administrator originally put in the zone.

@miekg

This comment has been minimized.

Show comment
Hide comment
@miekg

miekg Aug 24, 2017

Contributor

I've given my answers in previous comments

Contributor

miekg commented Aug 24, 2017

I've given my answers in previous comments

@alexbrainman

This comment has been minimized.

Show comment
Hide comment
@alexbrainman

alexbrainman Aug 24, 2017

Member

I do not have any suggestions. But I decided to implement LookupTXT that returns [][]string instead of []string on windows.

diff --git a/src/net/lookup_windows.go b/src/net/lookup_windows.go
index 0036d89..6994985 100644
--- a/src/net/lookup_windows.go
+++ b/src/net/lookup_windows.go
@@ -287,6 +287,29 @@ func (*Resolver) lookupTXT(ctx context.Context, name string) ([]string, error) {
 	return txts, nil
 }
 
+func WindowsLookupTXT(name string) ([][]string, error) {
+	acquireThread()
+	defer releaseThread()
+	var r *syscall.DNSRecord
+	e := syscall.DnsQuery(name, syscall.DNS_TYPE_TEXT, 0, nil, &r, nil)
+	if e != nil {
+		return nil, &DNSError{Err: winError("dnsquery", e).Error(), Name: name}
+	}
+	defer syscall.DnsRecordListFree(r, 1)
+
+	txts := make([][]string, 0, 10)
+	for _, p := range validRecs(r, syscall.DNS_TYPE_TEXT, name) {
+		d := (*syscall.DNSTXTData)(unsafe.Pointer(&p.Data[0]))
+		txt := make([]string, 0)
+		for _, v := range (*[1 << 10]*uint16)(unsafe.Pointer(&(d.StringArray[0])))[:d.StringCount] {
+			s := syscall.UTF16ToString((*[1 << 20]uint16)(unsafe.Pointer(v))[:])
+			txt = append(txt, s)
+		}
+		txts = append(txts, txt)
+	}
+	return txts, nil
+}
+
 func (*Resolver) lookupAddr(ctx context.Context, addr string) ([]string, error) {
 	// TODO(bradfitz): finish ctx plumbing. Nothing currently depends on this.
 	acquireThread()

Then this program

package main

import (
	"fmt"
	"net"
)

func main() {
	names := []string{
		"bbc.com",
	}
	for _, name := range names {
		value, err := net.WindowsLookupTXT(name)
		if err != nil {
			panic(err)
		}

		fmt.Println(name)
		for _, v := range value {
			fmt.Printf("  * %q\n", v)
		}
	}
}

outputs

bbc.com
  * ["MS=ms25863558"]
  * ["dropbox-domain-verification=mtgv0f2pudoz"]
  * ["v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99" " include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"]
  * ["Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="]

So that would have been perfect. No?

Alex

Member

alexbrainman commented Aug 24, 2017

I do not have any suggestions. But I decided to implement LookupTXT that returns [][]string instead of []string on windows.

diff --git a/src/net/lookup_windows.go b/src/net/lookup_windows.go
index 0036d89..6994985 100644
--- a/src/net/lookup_windows.go
+++ b/src/net/lookup_windows.go
@@ -287,6 +287,29 @@ func (*Resolver) lookupTXT(ctx context.Context, name string) ([]string, error) {
 	return txts, nil
 }
 
+func WindowsLookupTXT(name string) ([][]string, error) {
+	acquireThread()
+	defer releaseThread()
+	var r *syscall.DNSRecord
+	e := syscall.DnsQuery(name, syscall.DNS_TYPE_TEXT, 0, nil, &r, nil)
+	if e != nil {
+		return nil, &DNSError{Err: winError("dnsquery", e).Error(), Name: name}
+	}
+	defer syscall.DnsRecordListFree(r, 1)
+
+	txts := make([][]string, 0, 10)
+	for _, p := range validRecs(r, syscall.DNS_TYPE_TEXT, name) {
+		d := (*syscall.DNSTXTData)(unsafe.Pointer(&p.Data[0]))
+		txt := make([]string, 0)
+		for _, v := range (*[1 << 10]*uint16)(unsafe.Pointer(&(d.StringArray[0])))[:d.StringCount] {
+			s := syscall.UTF16ToString((*[1 << 20]uint16)(unsafe.Pointer(v))[:])
+			txt = append(txt, s)
+		}
+		txts = append(txts, txt)
+	}
+	return txts, nil
+}
+
 func (*Resolver) lookupAddr(ctx context.Context, addr string) ([]string, error) {
 	// TODO(bradfitz): finish ctx plumbing. Nothing currently depends on this.
 	acquireThread()

Then this program

package main

import (
	"fmt"
	"net"
)

func main() {
	names := []string{
		"bbc.com",
	}
	for _, name := range names {
		value, err := net.WindowsLookupTXT(name)
		if err != nil {
			panic(err)
		}

		fmt.Println(name)
		for _, v := range value {
			fmt.Printf("  * %q\n", v)
		}
	}
}

outputs

bbc.com
  * ["MS=ms25863558"]
  * ["dropbox-domain-verification=mtgv0f2pudoz"]
  * ["v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99" " include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"]
  * ["Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg=="]

So that would have been perfect. No?

Alex

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 24, 2017

@miekg I don't think behavior on Windows is legal on DNS standpoint.
Because single TXT record may be regarded as multiple TXT records.
Isn't it a problem on DNS standpoint?

@alexbrainman Thank you. Maybe I can use it.
I wish net.LookupTXT were like this.

cedar10bits commented Aug 24, 2017

@miekg I don't think behavior on Windows is legal on DNS standpoint.
Because single TXT record may be regarded as multiple TXT records.
Isn't it a problem on DNS standpoint?

@alexbrainman Thank you. Maybe I can use it.
I wish net.LookupTXT were like this.

@alexbrainman

This comment has been minimized.

Show comment
Hide comment
@alexbrainman

alexbrainman Aug 24, 2017

Member

I wish net.LookupTXT were like this.

If net.LookupTXT is important enough from security point of view or something, maybe we could mark net.LookupTXT as depreciated and add new net.ZZZLookupTXT function. I do not know enough about this to decide.

Alex

Member

alexbrainman commented Aug 24, 2017

I wish net.LookupTXT were like this.

If net.LookupTXT is important enough from security point of view or something, maybe we could mark net.LookupTXT as depreciated and add new net.ZZZLookupTXT function. I do not know enough about this to decide.

Alex

@miekg

This comment has been minimized.

Show comment
Hide comment
@miekg

miekg Aug 24, 2017

Contributor
Contributor

miekg commented Aug 24, 2017

@alexbrainman

This comment has been minimized.

Show comment
Hide comment
@alexbrainman

alexbrainman Aug 24, 2017

Member

I would consider LookupTXT good enough at this point.

Sounds good to me.
@cedar10bits should we close this as unfortunate?

Alex

Member

alexbrainman commented Aug 24, 2017

I would consider LookupTXT good enough at this point.

Sounds good to me.
@cedar10bits should we close this as unfortunate?

Alex

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 24, 2017

Sounds good to me.
@cedar10bits should we close this as unfortunate?

Sorry, I do not close this.

Package net is portable standard library (see overview).
https://golang.org/pkg/net/

net.LookupTXT is not portable because of different behavior between on Windows and on Linux.

cedar10bits commented Aug 24, 2017

Sounds good to me.
@cedar10bits should we close this as unfortunate?

Sorry, I do not close this.

Package net is portable standard library (see overview).
https://golang.org/pkg/net/

net.LookupTXT is not portable because of different behavior between on Windows and on Linux.

@alexbrainman

This comment has been minimized.

Show comment
Hide comment
@alexbrainman

alexbrainman Aug 24, 2017

Member

Closing as unfortunate.

Member

alexbrainman commented Aug 24, 2017

Closing as unfortunate.

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 24, 2017

Why close ? To support legacy code portability, net.LookupTXT should be fixed.

cedar10bits commented Aug 24, 2017

Why close ? To support legacy code portability, net.LookupTXT should be fixed.

@alexbrainman

This comment has been minimized.

Show comment
Hide comment
@alexbrainman

alexbrainman Aug 24, 2017

Member

To support legacy code portability, net.LookupTXT should be fixed.

What do you propose we do?

Alex

Member

alexbrainman commented Aug 24, 2017

To support legacy code portability, net.LookupTXT should be fixed.

What do you propose we do?

Alex

@alexbrainman alexbrainman reopened this Aug 24, 2017

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 24, 2017

What do you propose we do?

If we leave this API what it is, this API is not portable.
Some codes already use this API and these must be portable.
To avoid this, we have two options.

  1. Fix this API to equalize both behavior on Windows and on Linux.
  2. Add new API instead of this API in the future (deprecate this API).

I would like to propose first one (and fix Windows version).
But now, option 2 seems to be adopted.
If adopt 2, we need to decide when to add and announce it to users.

cedar10bits commented Aug 24, 2017

What do you propose we do?

If we leave this API what it is, this API is not portable.
Some codes already use this API and these must be portable.
To avoid this, we have two options.

  1. Fix this API to equalize both behavior on Windows and on Linux.
  2. Add new API instead of this API in the future (deprecate this API).

I would like to propose first one (and fix Windows version).
But now, option 2 seems to be adopted.
If adopt 2, we need to decide when to add and announce it to users.

@miekg

This comment has been minimized.

Show comment
Hide comment
@miekg

miekg Aug 24, 2017

Contributor
Contributor

miekg commented Aug 24, 2017

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Aug 24, 2017

Other than the difference between Windows and Linux, what is the actually problem here?

Almost nothing. If fix the behavior, we cannot decide how to fix.
Because both behavior on Windows and Linux is not legal on DNS standpoint.
(I think behavior on Linux is better for API users)

cedar10bits commented Aug 24, 2017

Other than the difference between Windows and Linux, what is the actually problem here?

Almost nothing. If fix the behavior, we cannot decide how to fix.
Because both behavior on Windows and Linux is not legal on DNS standpoint.
(I think behavior on Linux is better for API users)

@alexbrainman

This comment has been minimized.

Show comment
Hide comment
@alexbrainman

alexbrainman Aug 25, 2017

Member

@cedar10bits

Fix this API to equalize both behavior on Windows and on Linux.

Since I know nothing about DNS, I won't be able to implement this. So if you want this implemented you have to do it yourself. And I am not sure your change will be accepted - your change might be considered as "changing behavior too much". But you have to try to find out.

Add new API instead of this API in the future (deprecate this API).

I called this option, but I don't see this accepted, unless we can demonstrate that current behavior is very bad (security wise or something). I don't think "the difference between Windows and Linux" is good enough reason to introduce new API.

Alex

Member

alexbrainman commented Aug 25, 2017

@cedar10bits

Fix this API to equalize both behavior on Windows and on Linux.

Since I know nothing about DNS, I won't be able to implement this. So if you want this implemented you have to do it yourself. And I am not sure your change will be accepted - your change might be considered as "changing behavior too much". But you have to try to find out.

Add new API instead of this API in the future (deprecate this API).

I called this option, but I don't see this accepted, unless we can demonstrate that current behavior is very bad (security wise or something). I don't think "the difference between Windows and Linux" is good enough reason to introduce new API.

Alex

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Sep 24, 2017

@alexbrainman
Sorry for replying so late, I had got into some trouble in private.

So if you want this implemented you have to do it yourself.

I will implement this if accepted.

I don't think "the difference between Windows and Linux" is good enough reason to introduce new API.

Having thought about this, I think so, too.
So I won't deprecate this API and I want change this API like below.

  1. Change behavior of this API on Windows
  2. Add func LookupTXTWithSep(name, sep string) ([]string, error) (TBD: function name)
    This function returns multiple TXT records as a string array.
    Multiple strings in single TXT record are concatenated and the separator string sep is placed between the strings.
    To use this function, the user can get what the zone administrator originally put in the zone.
    For example, LookupTXTWithSep("bbc.com", "\n") returns TXT records as below:
{
    "MS=ms25863558",
    "dropbox-domain-verification=mtgv0f2pudoz",
    "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg==",
    "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99\n include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"
    // "\n" is placed after "ip4:207.159.133.99".
}

FYI, LookupTXT(name) is the same as LookupTXTWithSep(name, "").

cedar10bits commented Sep 24, 2017

@alexbrainman
Sorry for replying so late, I had got into some trouble in private.

So if you want this implemented you have to do it yourself.

I will implement this if accepted.

I don't think "the difference between Windows and Linux" is good enough reason to introduce new API.

Having thought about this, I think so, too.
So I won't deprecate this API and I want change this API like below.

  1. Change behavior of this API on Windows
  2. Add func LookupTXTWithSep(name, sep string) ([]string, error) (TBD: function name)
    This function returns multiple TXT records as a string array.
    Multiple strings in single TXT record are concatenated and the separator string sep is placed between the strings.
    To use this function, the user can get what the zone administrator originally put in the zone.
    For example, LookupTXTWithSep("bbc.com", "\n") returns TXT records as below:
{
    "MS=ms25863558",
    "dropbox-domain-verification=mtgv0f2pudoz",
    "Fzj91DPhHcxL3FxKMiBraJ9CajRin4nqr8AxflyEQLI+dM+xdOt5/I8F4xGMWelgP2SwFda7w8U2KZFjDR6Ocg==",
    "v=spf1 ip4:212.58.224.0/19 ip4:132.185.0.0/16 ip4:78.136.53.80/28 ip4:78.136.14.192/27 ip4:78.136.19.8/29 ip4:89.234.10.72/29 ip4:74.112.66.33 ip4:208.251.80.51 ip4:89.202.185.0/24 ip4:207.159.133.98 ip4:207.159.133.99\n include:msgfocus.com include:cmail1.com include:mktomail.com include:servers.mcsv.net include:redsnapper.net ?all"
    // "\n" is placed after "ip4:207.159.133.99".
}

FYI, LookupTXT(name) is the same as LookupTXTWithSep(name, "").

@alexbrainman

This comment has been minimized.

Show comment
Hide comment
@alexbrainman

alexbrainman Sep 25, 2017

Member

So I won't deprecate this API and I want change this API like below.

We cannot change existing API because existing programs that use net.LookupTXT will fail to build, so this change is not allowed as per https://golang.org/doc/go1compat

Alex

Member

alexbrainman commented Sep 25, 2017

So I won't deprecate this API and I want change this API like below.

We cannot change existing API because existing programs that use net.LookupTXT will fail to build, so this change is not allowed as per https://golang.org/doc/go1compat

Alex

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Oct 5, 2017

So I won't deprecate this API and I want change this API like below.

It was said in an inappropriate way.
I want to change behavior of LookupTXT(name string) on Windows (concatenating all the strings in a single TXT record).
And I want to add extra API, LookupTXTWithSep(name, sep string) to get single TXT record with multiple strings properly.
I think these ideas will solve discussed problems.

cedar10bits commented Oct 5, 2017

So I won't deprecate this API and I want change this API like below.

It was said in an inappropriate way.
I want to change behavior of LookupTXT(name string) on Windows (concatenating all the strings in a single TXT record).
And I want to add extra API, LookupTXTWithSep(name, sep string) to get single TXT record with multiple strings properly.
I think these ideas will solve discussed problems.

@alexbrainman

This comment has been minimized.

Show comment
Hide comment
@alexbrainman

alexbrainman Oct 5, 2017

Member

I want to change behavior of LookupTXT(name string)
And I want to add extra API, LookupTXTWithSep(name, sep string)

I am not the person who decides here. If you are OK with your changes to be rejected, feel free to try and send your changes for review.

Alex

Member

alexbrainman commented Oct 5, 2017

I want to change behavior of LookupTXT(name string)
And I want to add extra API, LookupTXTWithSep(name, sep string)

I am not the person who decides here. If you are OK with your changes to be rejected, feel free to try and send your changes for review.

Alex

@miekg

This comment has been minimized.

Show comment
Hide comment
@miekg

miekg Oct 5, 2017

Contributor
Contributor

miekg commented Oct 5, 2017

@cedar10bits

This comment has been minimized.

Show comment
Hide comment
@cedar10bits

cedar10bits Oct 26, 2017

  1. was discussed at length in this thread, so I'm not sure why it is being brought back.

A function in std lib must be consistent behavior even if called on any OS.
Both behavior on Windows and on Linux must be same.
Is it a wrong idea?

cedar10bits commented Oct 26, 2017

  1. was discussed at length in this thread, so I'm not sure why it is being brought back.

A function in std lib must be consistent behavior even if called on any OS.
Both behavior on Windows and on Linux must be same.
Is it a wrong idea?

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Nov 22, 2017

Contributor

This is really very simple. I'm sorry we wasted 50 comments on it.

The actual DNS response is a sequence of RRs each containing a sequence of string fragments. The correct handling of the response is to do:

for each rr {
    list = append(list, strings.Join(rr.fragments, ""))
}

(look at dnsRR_TXT.Walk, used on most platforms). The Windows code incorrectly does:

for each rr {
    list = append(list, rr.fragments...)
}

(look at Resolver.lookupTXT in lookup_windows.go).

Fix at https://golang.org/cl/79555

Contributor

rsc commented Nov 22, 2017

This is really very simple. I'm sorry we wasted 50 comments on it.

The actual DNS response is a sequence of RRs each containing a sequence of string fragments. The correct handling of the response is to do:

for each rr {
    list = append(list, strings.Join(rr.fragments, ""))
}

(look at dnsRR_TXT.Walk, used on most platforms). The Windows code incorrectly does:

for each rr {
    list = append(list, rr.fragments...)
}

(look at Resolver.lookupTXT in lookup_windows.go).

Fix at https://golang.org/cl/79555

@gopherbot

This comment has been minimized.

Show comment
Hide comment
@gopherbot

gopherbot Nov 22, 2017

Change https://golang.org/cl/79555 mentions this issue: net: fix LookupTXT of long records on Windows

gopherbot commented Nov 22, 2017

Change https://golang.org/cl/79555 mentions this issue: net: fix LookupTXT of long records on Windows

@gopherbot gopherbot closed this in b6cf58d Nov 23, 2017

billybanfield added a commit to mongodb/mongo-go-driver that referenced this issue Nov 30, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment