-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gethostbyname slowing things down on OS X #96
Comments
Note this line in the traceback:
Also note that this is code that happens in requests prior to betamax taking control over the request/response flow. Short of monkey-patching the standard library on a very specific operating system only when a cassette has already been recorded seems like overkill and out of the scope of this project to me. Do you agree? |
I hadn't realized that. From my perspective, I'm calling Rather than monkey patching the standard library, perhaps betamax could augment requests to bypass the check for proxies when a cassette is present. With that said, I understand if you feel it's out-of-scope. Perhaps it's worth mentioning the issue in the docs because it's fairly significant. Usually my test suite runs in fewer than half a second, but in the cases, for whatever reason, when it starts checking this proxy setting it takes > 30 seconds because for every request there is a DNS lookup. Maybe there's a separate issue here which is preventing those DNS responses from being cached; they're all for the same host. If I wanted to do the latter, do you have understanding of what is actually happening here? I'd be happy to write something up for the documentation if I knew why these DNS requests were always happening. |
So the point of Betamax working the way it does is to avoid monkey-patching anything. That said, if we want to prevent requests from using this, we need to monkey-patch requests (but, again, only when the cassette is already recorded). I understand the frustration with this.
That would be a requests specific question IMO. I can look into it tonight maybe. |
Hm, so this only ever happens when handling redirects. The
And both In both cases, they are guarded by the |
Yes, I think explaining why this occurs and a potential work around (my fix below) would be suitable. # Temporarily work around issue with gethostbyname on OS X
import socket
socket.gethostbyname = lambda x: '127.0.0.1' I do, however, like your solution of forcing |
We don't change anything else about the Session besides its adapters. I can't think of a case where someone might be subclassing a Session and looking at with Betamax(session) as recorder:
recorder.use_cassette('cassette_name') So we'll have to track extra state on the |
#107 added documentation about this for the 0.7.0 milestone. I'm going to remove that milestone now. When there is a good solution, I'll re-milestone this. |
Awesome. Thanks for the update. |
Closing this issue as I feel the documentation addition is sufficient. |
I finally got annoyed that my integration tests would periodically slow down. After running a profiler I saw that
socket.gethostbyname
was the bottleneck. I adapted the betamax example to show the stacktrace that shows thatsocket.gethostbyname
is indeed being called. I'm guessing that it probably shouldn't be.Here's the example
--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/31935842-gethostbyname-slowing-things-down-on-os-x?utm_campaign=plugin&utm_content=tracker%2F198445&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F198445&utm_medium=issues&utm_source=github).socket.gethostbyname
does not get called when I tested from a linux machine.The text was updated successfully, but these errors were encountered: