Next Page in Iteration fails

[rsalesas]

Hi, I'm running the following code and it fails consistently both in users and leads when it needs to page.

This happens in Python 2.7 or 3.0

for lead in intercom.leads.all(): print(lead.email)

for user in intercom.users.all(): print(user.email)

Error is:

Traceback

(most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/python_intercom-3.0b2-py3.5.egg/intercom/collection_proxy.py", line 52, in next
resource = six.next(self.resources)
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 142, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/util/connection.py", line 67, in create_connection
for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 732, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 578, in url open
chunked=chunked)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 351, in _make_request
self._validate_conn(conn)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 814, in _validate_conn
conn.connect()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 254, in connect
conn = self._new_conn()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 151, in _new_conn
self, "Failed to establish a new connection: %s" % e)
requests.packages.urllib3.exceptions.NewConnectionError: <requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x1033d75c0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/adapters.py", line 403, in send
timeout=timeout
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 623, in url open
_stacktrace=sys.exc_info()[2])
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/util/retry.py", line 281, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
requests.packages.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.intercom.iohttps', port=443): Max retries exceeded with url: //api.intercom.io/users?per_page=50&page=2 (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x1033d75c0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "CleanIntercomLeads.py", line 9, in
for lead in intercom.users.all():
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/python_intercom-3.0b2-py3.5.egg/intercom/collection_proxy.py", line 54, in next
self.get_next_page()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/python_intercom-3.0b2-py3.5.egg/intercom/collection_proxy.py", line 71, in get_next_page
return self.get_page(self.next_page)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/python_intercom-3.0b2-py3.5.egg/intercom/collection_proxy.py", line 81, in get_page
response = self.client.get(url, params)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/python_intercom-3.0b2-py3.5.egg/intercom/client.py", line 89, in get
return self._execute_request(req, params)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/python_intercom-3.0b2-py3.5.egg/intercom/client.py", line 82, in _execute_request
result = request.execute(self.base_url, self._auth, params)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/python_intercom-3.0b2-py3.5.egg/intercom/request.py", line 23, in execute
return self.send_request_to_path(base_url, auth, params)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/python_intercom-3.0b2-py3.5.egg/intercom/request.py", line 57, in send_request_to_path
auth=auth, verify=certifi.where(), *_req_params)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/api.py", line 57, in request
return session.request(method=method, url=url, *_kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py", line 475, in request
resp = self.send(prep, *_send_kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py", line 585, in send
r = adapter.send(request, *_kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/adapters.py", line 467, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.intercom.iohttps', port=443): Max retries exceeded with url: //api.intercom.io/users?per_page=50&page=2 (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x1033d75c0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',))

[teeberg]

This looks like a DNS resolution error. Can you wget/curl it to see if it gets resolved properly? Do any other intercom API requests work as expected?

[rsalesas]

Not likely as the first set come through just fine. This is failing on getting the next page. I ran the same query with curl, then got page 2, and it worked fine.

[jkeyes]

I haven't looked into this yet @rsalesas, is it still an issue for you? I'm going to do some work around getting v3 released soon, so I should get to this while doing that.

[jt-money]

+1 on this; I have hit this same exact issue as well with the 3.0b2 branch (this issue is NOT PRESENT in version 2.1.1)
For users.all() it prints 50 users every time then errors out, which I eventually figured out as being caused by a lack of pagination...and then I found this page :(

I had to write a hacky manual pagination loop (sending page='2', etc, after a for loop of 20 elements) in the meantime, but any chance this issue will get some attention soon?

Same goes for conversations and leads; it only prints the first page for each (as such):

>>> for convo in intercom.conversations.find_all(type='admin', id='nobody'): ... print convo ... <intercom.conversation.Conversation object at 0x10696bed0> <intercom.conversation.Conversation object at 0x10696ba50> <intercom.conversation.Conversation object at 0x10696bed0> <intercom.conversation.Conversation object at 0x10696ba50> <intercom.conversation.Conversation object at 0x10696bed0> <intercom.conversation.Conversation object at 0x10696ba50> <intercom.conversation.Conversation object at 0x10696bed0> <intercom.conversation.Conversation object at 0x10696ba50> <intercom.conversation.Conversation object at 0x10696bed0> <intercom.conversation.Conversation object at 0x10696ba50> <intercom.conversation.Conversation object at 0x10696bed0> <intercom.conversation.Conversation object at 0x10696ba50> <intercom.conversation.Conversation object at 0x10696bed0> <intercom.conversation.Conversation object at 0x10696ba50> <intercom.conversation.Conversation object at 0x10696bed0> <intercom.conversation.Conversation object at 0x10696ba50> <intercom.conversation.Conversation object at 0x10696bed0> <intercom.conversation.Conversation object at 0x10696ba50> <intercom.conversation.Conversation object at 0x10696bed0> <intercom.conversation.Conversation object at 0x10696ba50> Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/site-packages/six.py", line 558, in next return type(self).__next__(self) File "/usr/local/lib/python2.7/site-packages/intercom/collection_proxy.py", line 54, in __next__ self.get_next_page() File "/usr/local/lib/python2.7/site-packages/intercom/collection_proxy.py", line 71, in get_next_page return self.get_page(self.next_page) File "/usr/local/lib/python2.7/site-packages/intercom/collection_proxy.py", line 81, in get_page response = self.client.get(url, params) File "/usr/local/lib/python2.7/site-packages/intercom/client.py", line 89, in get return self._execute_request(req, params) File "/usr/local/lib/python2.7/site-packages/intercom/client.py", line 82, in _execute_request result = request.execute(self.base_url, self._auth, params) File "/usr/local/lib/python2.7/site-packages/intercom/request.py", line 23, in execute return self.send_request_to_path(base_url, auth, params) File "/usr/local/lib/python2.7/site-packages/intercom/request.py", line 57, in send_request_to_path auth=auth, verify=certifi.where(), **req_params) File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 56, in request return session.request(method=method, url=url, **kwargs) File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 475, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 596, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 487, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.intercom.iohttps', port=443): Max retries exceeded with url: //api.intercom.io/conversations?type=admin&id=nobody&per_page=20&page=2 (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x1069e5a90>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',))

[jkeyes]

I just merged #125 which is related to next_link in collection_proxy. I think this may have resolved this issue.

[rsalesas]

How come you can't see the failure when you test?

• Robert Salesas

On 16 Nov 2016, at 03:51, John Keyes notifications@github.com wrote:

I just merged #125 which is related to next_link in collection_proxy. I think this may have resolved this issue.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[jkeyes]

I wasn't able to reproduce it last time I looked at it (don't know if that was a fault with the test or not). I haven't tested it this time around, I'm just letting you know that the PR might have resolved it as it was to do with pagination. The issue is still open.

[rsalesas]

OK I’ll give it another shot. Stopped using it, as it didn’t work.

On 16 Nov 2016, at 17:14, John Keyes notifications@github.com wrote:

I wasn't able to reproduce it last time I looked at it (don't know if that was a fault with the test or not). I haven't tested it this time around, I'm just letting you know that the PR might have resolved it as it was to do with pagination. The issue is still open.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub #114 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/AE4ezMhCIBHCCKP78EcWhdYL32Bqh0Chks5q-smBgaJpZM4Ie0e0.

[jkeyes]

Sorry about that @rsalesas. I see a window of time opening up a bit for me so we'll get the version 3 release out the door soonish anyway. This should be fixed in that release.

[jkeyes]

@rsalesas This is fixed in 3.0.