Federation fails when using IPv6 DNS servers #1002

benhylau · 2016-08-11T05:29:44Z

Federation fails with:

2016-08-10 22:27:45,916 - synapse.http.matrixfederationclient - 183 - WARNING - POST-4- {GET-O-2} Sending request failed to matrix.org: GET matrix://matrix.org/_matrix/federation/v1/query/profile?field=avatar_url&user_id=%40benhylau%3Amatrix.org: InvalidA
ddressError - InvalidAddressError: IPv4 port write() called with IPv6 address

Removing my IPv6 DNS servers in /etc/resolv.conf fix federation:

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# nameserver 2001:4860:4860::8844 <-- comment out
# nameserver 2001:4860:4860::8888 <-- comment out
nameserver 8.8.8.8

Seems to be a known issue that has come up in #matrix:matrix.org many times.

The text was updated successfully, but these errors were encountered:

ara4n · 2016-08-27T20:14:44Z

Yup. the official bug for this is https://matrix.org/jira/browse/SYN-254

ara4n · 2016-12-20T10:18:50Z

...and now fixed on develop by #1696, we believe :)

ara4n · 2016-12-20T10:26:15Z

...except apparently it's still broken on develop :(

kyrias · 2016-12-20T10:38:17Z

@glyph Do you know the state of using IPv6 resolvers in twisted, if there's any work going on regarding it? Currently twisted's trunk still doesn't work with them.

glyph · 2016-12-20T22:05:20Z

@kyrias I'm afraid I don't understand the question. Twisted's trunk only recently added support for them. What do you consider an "IPv6 resolver"?

kyrias · 2016-12-20T22:51:10Z

@glyph An IPv6 resolver is a resolver queried over IPv6.

glyph · 2016-12-21T00:06:35Z

@kyrias Oh, so just an IPv6 address for looking up A records, not a resolver that can resolve AAAA records? Huh. In what sense does that not work? Are you talking about something in twisted.names?

kyrias · 2016-12-21T08:02:21Z

@glyph Yes.

>>> from twisted.names import client
>>> client.getHostByName('google.com')
<Deferred at 0x7f1818a1f290 current result: <twisted.python.failure.Failure twisted.internet.error.InvalidAddressError: >>

glyph · 2016-12-22T00:49:01Z

@kyrias This is definitely a serious bug in Twisted. It's been filed here - https://twistedmatrix.com/trac/ticket/8877 - unfortunately that's still fairly recent. Sorry about that!

swift11 · 2016-12-29T06:41:46Z

@glyph Weird: it's a bug with "normal" priority in Twisted...

glyph · 2016-12-29T14:45:16Z

@swift11: Priority is not really used in our bugtracker. You can tell if something is really high priority by whether it gets fixed or not :)

reactormonk · 2016-12-29T19:23:57Z

twisted/twisted#649

reactormonk · 2017-01-18T09:01:47Z

Time to update dependencies, boys.

kyrias · 2017-01-18T09:35:29Z

The dependency won't be updated until there's a new twisted release. ;P

But awesome work fixing it!

ara4n · 2017-01-18T10:19:57Z

@reactormonk Awesome!! huge thanks for sorting it out :) Will close the bug once we've bumped the dep.

glyph · 2017-01-19T09:21:39Z

Since 16.7 was full of ghosts, the next release will be 17.1 or 17.2.

alejzeis · 2017-02-12T16:06:19Z

It looks like 17.1 was just released yesterday, going to try and test it out.

EDIT: Doesn't seem to work, getting a 401 UNAUTHORIZED in homeserver.log when I try to join #matrix:matrix.org through my homeserver:

2017-02-12 10:38:10,226 - synapse.access.http.8008 - 59 - INFO - GET-25- 127.0.0.1 - 8008 - Received request: GET /_matrix/client/r0/directory/room/%23matrix%3Amatrix.org?access_token=<redacted>
2017-02-12 10:38:10,227 - synapse.http.outbound - 124 - INFO - GET-25- {GET-O-1} [matrix.org] Sending request: GET matrix://matrix.org/_matrix/federation/v1/query/directory?room_alias=%23matrix%3Amatrix.org
2017-02-12 10:38:10,229 - twisted - 131 - INFO - - /etc/resolv.conf changed, reparsing
2017-02-12 10:38:10,230 - twisted - 131 - INFO - - Resolver added ('2001:4860:4860::8844', 53) to server list
2017-02-12 10:38:10,230 - twisted - 131 - INFO - - Resolver added ('2001:4860:4860::8888', 53) to server list
2017-02-12 10:38:10,232 - twisted - 131 - INFO - - DNSDatagramProtocol starting on 43979
2017-02-12 10:38:10,232 - twisted - 131 - INFO - - Starting protocol <twisted.names.dns.DNSDatagramProtocol object at 0x41b7ad0>
2017-02-12 10:38:10,233 - synapse.metrics - 162 - INFO - GET-25- Collecting gc 0
2017-02-12 10:38:10,252 - twisted - 131 - INFO - GET-25- DNSDatagramProtocol starting on 39985
2017-02-12 10:38:10,252 - twisted - 131 - INFO - GET-25- Starting protocol <twisted.names.dns.DNSDatagramProtocol object at 0x41aadd0>
2017-02-12 10:38:10,253 - twisted - 131 - INFO - GET-25- (UDP Port 43979 Closed)
2017-02-12 10:38:10,253 - twisted - 131 - INFO - GET-25- Stopping protocol <twisted.names.dns.DNSDatagramProtocol object at 0x41b7ad0>
2017-02-12 10:38:10,271 - synapse.http.endpoint - 248 - INFO - GET-25- Connecting to 83.166.64.33:8448
2017-02-12 10:38:10,274 - twisted - 131 - INFO - GET-25- Starting factory <twisted.web.client._HTTP11ClientFactory instance at 0x41bbf38>
2017-02-12 10:38:10,275 - twisted - 131 - INFO - GET-25- (UDP Port 39985 Closed)
2017-02-12 10:38:10,275 - twisted - 131 - INFO - GET-25- Stopping protocol <twisted.names.dns.DNSDatagramProtocol object at 0x41aadd0>
2017-02-12 10:38:10,296 - synapse.storage.TIME - 207 - INFO - GET-25- Total database time: 0.000% {update_presence(0): 0.000%, update_cached_last_access_time(0): 0.000%, store_device(0): 0.000%} {}
2017-02-12 10:38:13,494 - synapse.http.outbound - 209 - INFO - GET-25- {GET-O-1} [matrix.org] Result: 401 Unauthorized
2017-02-12 10:38:13,495 - root - 180 - WARNING - GET-25- Error retrieving alias
2017-02-12 10:38:13,496 - synapse.http.server - 126 - ERROR - GET-25- 401: Unauthorized
Traceback (most recent call last):
  File "/home/jython234/synapse/lib/python2.7/site-packages/synapse/http/server.py", line 116, in wrapped_request_handler
    yield request_handler(self, request, request_metrics)
  File "/home/jython234/synapse/lib/python2.7/site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/jython234/synapse/lib/python2.7/site-packages/twisted/python/failure.py", line 393, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/jython234/synapse/lib/python2.7/site-packages/synapse/http/server.py", line 255, in _async_render
    callback_return = yield callback(request, **kwargs)
  File "/home/jython234/synapse/lib/python2.7/site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/jython234/synapse/lib/python2.7/site-packages/twisted/python/failure.py", line 393, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/jython234/synapse/lib/python2.7/site-packages/synapse/rest/client/v1/directory.py", line 49, in on_GET
    res = yield dir_handler.get_association(room_alias)
  File "/home/jython234/synapse/lib/python2.7/site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/jython234/synapse/lib/python2.7/site-packages/twisted/python/failure.py", line 393, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/jython234/synapse/lib/python2.7/site-packages/synapse/handlers/directory.py", line 177, in get_association
    retry_on_dns_fail=False,
  File "/home/jython234/synapse/lib/python2.7/site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/jython234/synapse/lib/python2.7/site-packages/twisted/python/failure.py", line 393, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/jython234/synapse/lib/python2.7/site-packages/synapse/federation/transport/client.py", line 185, in make_query
    timeout=10000,
  File "/home/jython234/synapse/lib/python2.7/site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/jython234/synapse/lib/python2.7/site-packages/twisted/python/failure.py", line 393, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/jython234/synapse/lib/python2.7/site-packages/synapse/http/matrixfederationclient.py", line 390, in get_json
    timeout=timeout,
  File "/home/jython234/synapse/lib/python2.7/site-packages/twisted/internet/defer.py", line 1301, in _inlineCallbacks
    result = g.send(result)
  File "/home/jython234/synapse/lib/python2.7/site-packages/synapse/http/matrixfederationclient.py", line 219, in _create_request
    response.code, response.phrase, body
HttpResponseException: 401: Unauthorized

ara4n · 2017-02-12T16:56:16Z

@jython234 let us know how you get on :)

alejzeis · 2017-02-12T17:04:46Z

It turns out my error was because of incorrect reverse proxy setup, not IPv6. I fixed it and all is well now! This should be fixed, I submitted a pull #1909 which simply bumps the dependency.

richvdh · 2017-03-30T23:03:01Z

so this is fixed by upgrading Twisted to 17.1.0. We should recommend that in the README, but this is fixed.

glyph · 2017-03-31T06:18:49Z

🙌 🎉 The system works!!!🎉🙌

ara4n · 2017-03-31T18:29:47Z

readme-recommendation PR is #2088 ftr. and \o/ for the system working

PMaynard mentioned this issue Dec 7, 2016

Federation doesn't play nice with IPv6 (SYN-254) #1279

Closed

ara4n closed this as completed Dec 20, 2016

ara4n reopened this Dec 20, 2016

ara4n changed the title ~~InvalidAddressError: IPv4 port write() called with IPv6 address~~ Federation fails when using IPv6 DNS servers Dec 20, 2016

alejzeis mentioned this issue Feb 12, 2017

Bump Twisted Dependency to 17.1.0 #1909

Closed

richvdh closed this as completed Mar 30, 2017

LouisMT mentioned this issue Jul 17, 2018

Add IPv6 support for Docker images #3551

Closed

aryasenna mentioned this issue Aug 8, 2019

Federation DNS Lookup fails for healthy domain #5831

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Federation fails when using IPv6 DNS servers #1002

Federation fails when using IPv6 DNS servers #1002

benhylau commented Aug 11, 2016

ara4n commented Aug 27, 2016

ara4n commented Dec 20, 2016

ara4n commented Dec 20, 2016

kyrias commented Dec 20, 2016

glyph commented Dec 20, 2016

kyrias commented Dec 20, 2016

glyph commented Dec 21, 2016

kyrias commented Dec 21, 2016

glyph commented Dec 22, 2016

swift11 commented Dec 29, 2016

glyph commented Dec 29, 2016

reactormonk commented Dec 29, 2016

reactormonk commented Jan 18, 2017 •

edited

Loading

kyrias commented Jan 18, 2017 •

edited

Loading

ara4n commented Jan 18, 2017

glyph commented Jan 19, 2017

alejzeis commented Feb 12, 2017 •

edited

Loading

ara4n commented Feb 12, 2017

alejzeis commented Feb 12, 2017

richvdh commented Mar 30, 2017

glyph commented Mar 31, 2017

ara4n commented Mar 31, 2017

Federation fails when using IPv6 DNS servers #1002

Federation fails when using IPv6 DNS servers #1002

Comments

benhylau commented Aug 11, 2016

ara4n commented Aug 27, 2016

ara4n commented Dec 20, 2016

ara4n commented Dec 20, 2016

kyrias commented Dec 20, 2016

glyph commented Dec 20, 2016

kyrias commented Dec 20, 2016

glyph commented Dec 21, 2016

kyrias commented Dec 21, 2016

glyph commented Dec 22, 2016

swift11 commented Dec 29, 2016

glyph commented Dec 29, 2016

reactormonk commented Dec 29, 2016

reactormonk commented Jan 18, 2017 • edited Loading

kyrias commented Jan 18, 2017 • edited Loading

ara4n commented Jan 18, 2017

glyph commented Jan 19, 2017

alejzeis commented Feb 12, 2017 • edited Loading

ara4n commented Feb 12, 2017

alejzeis commented Feb 12, 2017

richvdh commented Mar 30, 2017

glyph commented Mar 31, 2017

ara4n commented Mar 31, 2017

reactormonk commented Jan 18, 2017 •

edited

Loading

kyrias commented Jan 18, 2017 •

edited

Loading

alejzeis commented Feb 12, 2017 •

edited

Loading