-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
asynchronously resolve a name to multiple address objects #548
Merged
Merged
Changes from all commits
Commits
Show all changes
42 commits
Select commit
Hold shift + click to select a range
b5503bb
interface proposal
glyph adb0b51
Merge remote-tracking branch 'origin/trunk' into 4362-multiresolve
glyph f493f55
wtb $NEXTVER
glyph 4d4330b
document return type
glyph a28268e
enough ceremony to deterministically call a function
glyph f8e87ab
hrm. should this be a native string?
glyph ab7db3c
AF_* sensitivity
glyph 5900339
topfile
glyph eef928f
Merge remote-tracking branch 'origin/trunk' into 4362-multiresolve
glyph 3ef18ce
new style
glyph 0b7919a
test for error case
glyph 1a02f09
assertIs
glyph 6f65252
implement the error case
glyph 7f8fa2b
Merge branch 'trunk' into 4362-multiresolve
glyph 3c1a225
write a test for code that codecov's complaining about
glyph 53c10ab
oops
glyph 8da4f9c
interface doc updates
glyph c2b1359
Merge remote-tracking branch 'origin/trunk' into 4362-multiresolve
glyph 8037bf3
Merge branch 'trunk' into 4362-multiresolve
glyph ccf377f
address family selection
glyph d9d48ec
docstring adjustment
glyph 6795570
pacify twistedchecker
glyph 25d2451
Merge branch 'trunk' into 4362-multiresolve
glyph 983ea33
Merge remote-tracking branch 'origin/4362-multiresolve' into 4362-mul…
glyph bcdab97
vws
glyph b59681e
twistedchecker clean
glyph 4bf5601
tests & fix for various AF_ types
glyph 819eed6
hoist constant
glyph 537fabb
fewer lines
glyph 86949be
less code more data
glyph 402778d
None first
glyph 8b0718f
no, Nones are bad
glyph efb07fe
socket types
glyph bd740c7
socktype
glyph 0c9d6cc
compatibility layer
glyph 9094566
distinct names since these need to live on the same object
glyph 9cdc20b
twistedchecker
glyph 06afa1e
forward and backward compatibility
glyph 7555e09
defer acquisition of the threadpool
glyph 65eebe9
twistedchecker
glyph 7d3a61c
Merge branch 'trunk' into 4362-multiresolve
hawkowl 6625d4b
too many newlines as per review
glyph File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,263 @@ | ||
# -*- test-case-name: twisted.internet.test.test_resolver -*- | ||
# Copyright (c) Twisted Matrix Laboratories. | ||
# See LICENSE for details. | ||
|
||
""" | ||
IPv6-aware hostname resolution. | ||
|
||
@see: L{IHostnameResolver} | ||
""" | ||
|
||
from __future__ import division, absolute_import | ||
|
||
__metaclass__ = type | ||
|
||
from socket import (getaddrinfo, AF_INET, AF_INET6, AF_UNSPEC, SOCK_STREAM, | ||
SOCK_DGRAM, gaierror) | ||
|
||
from zope.interface import implementer | ||
|
||
from twisted.internet.interfaces import (IHostnameResolver, IHostResolution, | ||
IResolverSimple, IResolutionReceiver) | ||
from twisted.internet.error import DNSLookupError | ||
from twisted.internet.defer import Deferred | ||
from twisted.internet.threads import deferToThreadPool | ||
from twisted.internet.address import IPv4Address, IPv6Address | ||
from twisted.logger import Logger | ||
|
||
|
||
@implementer(IHostResolution) | ||
class HostResolution(object): | ||
""" | ||
The in-progress resolution of a given hostname. | ||
""" | ||
|
||
def __init__(self, name): | ||
""" | ||
Create a L{HostResolution} with the given name. | ||
""" | ||
self.name = name | ||
|
||
|
||
|
||
_any = frozenset([IPv4Address, IPv6Address]) | ||
|
||
_typesToAF = { | ||
frozenset([IPv4Address]): AF_INET, | ||
frozenset([IPv6Address]): AF_INET6, | ||
_any: AF_UNSPEC, | ||
} | ||
|
||
_afToType = { | ||
AF_INET: IPv4Address, | ||
AF_INET6: IPv6Address, | ||
} | ||
|
||
_transportToSocket = { | ||
'TCP': SOCK_STREAM, | ||
'UDP': SOCK_DGRAM, | ||
} | ||
|
||
_socktypeToType = { | ||
SOCK_STREAM: 'TCP', | ||
SOCK_DGRAM: 'UDP', | ||
} | ||
|
||
|
||
|
||
@implementer(IHostnameResolver) | ||
class GAIResolver(object): | ||
""" | ||
L{IHostnameResolver} implementation that resolves hostnames by calling | ||
L{getaddrinfo} in a thread. | ||
""" | ||
|
||
def __init__(self, reactor, getThreadPool=None, getaddrinfo=getaddrinfo): | ||
""" | ||
Create a L{GAIResolver}. | ||
|
||
@param reactor: the reactor to schedule result-delivery on | ||
@type reactor: L{IReactorThreads} | ||
|
||
@param getThreadPool: a function to retrieve the thread pool to use for | ||
scheduling name resolutions. If not supplied, the use the given | ||
C{reactor}'s thread pool. | ||
@type getThreadPool: 0-argument callable returning a | ||
L{twisted.python.threadpool.ThreadPool} | ||
|
||
@param getaddrinfo: a reference to the L{getaddrinfo} to use - mainly | ||
parameterized for testing. | ||
@type getaddrinfo: callable with the same signature as L{getaddrinfo} | ||
""" | ||
self._reactor = reactor | ||
self._getThreadPool = (reactor.getThreadPool if getThreadPool is None | ||
else getThreadPool) | ||
self._getaddrinfo = getaddrinfo | ||
|
||
|
||
def resolveHostName(self, resolutionReceiver, hostName, portNumber=0, | ||
addressTypes=None, transportSemantics='TCP'): | ||
""" | ||
See L{IHostnameResolver.resolveHostName} | ||
|
||
@param resolutionReceiver: see interface | ||
|
||
@param hostName: see interface | ||
|
||
@param portNumber: see interface | ||
|
||
@param addressTypes: see interface | ||
|
||
@param transportSemantics: see interface | ||
|
||
@return: see interface | ||
""" | ||
pool = self._getThreadPool() | ||
addressFamily = _typesToAF[_any if addressTypes is None | ||
else frozenset(addressTypes)] | ||
socketType = _transportToSocket[transportSemantics] | ||
def get(): | ||
try: | ||
return self._getaddrinfo(hostName, portNumber, addressFamily, | ||
socketType) | ||
except gaierror: | ||
return [] | ||
d = deferToThreadPool(self._reactor, pool, get) | ||
resolution = HostResolution(hostName) | ||
resolutionReceiver.resolutionBegan(resolution) | ||
@d.addCallback | ||
def deliverResults(result): | ||
for family, socktype, proto, cannoname, sockaddr in result: | ||
addrType = _afToType[family] | ||
resolutionReceiver.addressResolved( | ||
addrType(_socktypeToType.get(socktype, 'TCP'), *sockaddr) | ||
) | ||
resolutionReceiver.resolutionComplete() | ||
return resolution | ||
|
||
|
||
|
||
@implementer(IHostnameResolver) | ||
class SimpleResolverComplexifier(object): | ||
""" | ||
A converter from L{IResolverSimple} to L{IHostnameResolver}. | ||
""" | ||
|
||
_log = Logger() | ||
|
||
def __init__(self, simpleResolver): | ||
""" | ||
Construct a L{SimpleResolverComplexifier} with an L{IResolverSimple}. | ||
""" | ||
self._simpleResolver = simpleResolver | ||
|
||
|
||
def resolveHostName(self, resolutionReceiver, hostName, portNumber=0, | ||
addressTypes=None, transportSemantics='TCP'): | ||
""" | ||
See L{IHostnameResolver.resolveHostName} | ||
|
||
@param resolutionReceiver: see interface | ||
|
||
@param hostName: see interface | ||
|
||
@param portNumber: see interface | ||
|
||
@param addressTypes: see interface | ||
|
||
@param transportSemantics: see interface | ||
|
||
@return: see interface | ||
""" | ||
resolution = HostResolution(hostName) | ||
resolutionReceiver.resolutionBegan(resolution) | ||
onAddress = self._simpleResolver.getHostByName(hostName) | ||
def addressReceived(address): | ||
resolutionReceiver.addressResolved(IPv4Address('TCP', address, 0)) | ||
def errorReceived(error): | ||
if not error.check(DNSLookupError): | ||
self._log.failure("while looking up {name} with {resolver}", | ||
error, name=hostName, | ||
resolver=self._simpleResolver) | ||
onAddress.addCallbacks(addressReceived, errorReceived) | ||
def finish(result): | ||
resolutionReceiver.resolutionComplete() | ||
onAddress.addCallback(finish) | ||
return resolution | ||
|
||
|
||
|
||
@implementer(IResolutionReceiver) | ||
class FirstOneWins(object): | ||
""" | ||
An L{IResolutionReceiver} which fires a L{Deferred} with its first result. | ||
""" | ||
|
||
def __init__(self, deferred): | ||
""" | ||
@param deferred: The L{Deferred} to fire when the first resolution | ||
result arrives. | ||
""" | ||
self._deferred = deferred | ||
self._resolved = False | ||
|
||
|
||
def resolutionBegan(self, resolution): | ||
""" | ||
See L{IResolutionReceiver.resolutionBegan} | ||
|
||
@param resolution: See L{IResolutionReceiver.resolutionBegan} | ||
""" | ||
self._resolution = resolution | ||
|
||
|
||
def addressResolved(self, address): | ||
""" | ||
See L{IResolutionReceiver.addressResolved} | ||
|
||
@param address: See L{IResolutionReceiver.addressResolved} | ||
""" | ||
if self._resolved: | ||
return | ||
self._resolved = True | ||
self._deferred.callback(address.host) | ||
|
||
|
||
def resolutionComplete(self): | ||
""" | ||
See L{IResolutionReceiver.resolutionComplete} | ||
""" | ||
if self._resolved: | ||
return | ||
self._deferred.errback(DNSLookupError(self._resolution.name)) | ||
|
||
|
||
|
||
@implementer(IResolverSimple) | ||
class ComplexResolverSimplifier(object): | ||
""" | ||
A converter from L{IHostnameResolver} to L{IResolverSimple} | ||
""" | ||
def __init__(self, nameResolver): | ||
""" | ||
Create a L{ComplexResolverSimplifier} with an L{IHostnameResolver}. | ||
|
||
@param nameResolver: The L{IHostnameResolver} to use. | ||
""" | ||
self._nameResolver = nameResolver | ||
|
||
|
||
def getHostByName(self, name, timeouts=()): | ||
""" | ||
See L{IResolverSimple.getHostByName} | ||
|
||
@param name: see L{IResolverSimple.getHostByName} | ||
|
||
@param timeouts: see L{IResolverSimple.getHostByName} | ||
|
||
@return: see L{IResolverSimple.getHostByName} | ||
""" | ||
result = Deferred() | ||
self._nameResolver.resolveHostName(FirstOneWins(result), name, 0, | ||
[IPv4Address]) | ||
return result |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why do we need this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To avoid accidentally declaring an old-style class.