chats endpoint returning 400 Status - Bad Request Errors

[dsprayberry]

We noticed a bad request against the chats endpoint.

---

400 Client Error: BAD REQUEST for url: https://www.zopim.com/api/v2/chats/search?q=type%3Achat+AND+end_timestamp%3A%5B2017-11-19T14%3A47%3A52%2B00%3A00+TO+2017-12-03T14%3A47%3A52%2B00%3A00%5D&page=251

Full traceback below:

2018-11-20 18:24:03,353Z    tap - Traceback (most recent call last):
2018-11-20 18:24:03,353Z    tap -   File "tap-env/bin/tap-zendesk-chat", line 11, in <module>
2018-11-20 18:24:03,354Z    tap -     sys.exit(main())
2018-11-20 18:24:03,354Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zendesk_chat/__init__.py", line 112, in main
2018-11-20 18:24:03,354Z    tap -     raise exc
2018-11-20 18:24:03,355Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zendesk_chat/__init__.py", line 109, in main
2018-11-20 18:24:03,355Z    tap -     main_impl()
2018-11-20 18:24:03,355Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zendesk_chat/__init__.py", line 105, in main_impl
2018-11-20 18:24:03,355Z    tap -     sync(ctx)
2018-11-20 18:24:03,356Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zendesk_chat/__init__.py", line 91, in sync
2018-11-20 18:24:03,356Z    tap -     stream.sync(ctx)
2018-11-20 18:24:03,356Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zendesk_chat/streams.py", line 150, in sync
2018-11-20 18:24:03,357Z    tap -     self._pull(ctx, "chat", "end_timestamp", full_sync=full_sync),
2018-11-20 18:24:03,357Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zendesk_chat/streams.py", line 117, in _pull
2018-11-20 18:24:03,357Z    tap -     search_resp = ctx.client.request(self.tap_stream_id, url=next_url)
2018-11-20 18:24:03,358Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/backoff.py", line 286, in retry
2018-11-20 18:24:03,358Z    tap -     ret = target(*args, **kwargs)
2018-11-20 18:24:03,358Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zendesk_chat/http.py", line 34, in request
2018-11-20 18:24:03,358Z    tap -     response.raise_for_status()
2018-11-20 18:24:03,359Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/requests/models.py", line 940, in raise_for_status
2018-11-20 18:24:03,359Z    tap -     raise HTTPError(http_error_msg, response=self)
2018-11-20 18:24:03,359Z    tap - requests.exceptions.HTTPError: 400 Client Error: BAD REQUEST for url: https://www.zopim.com/api/v2/chats/search?q=type%3Achat+AND+end_timestamp%3A%5B2017-11-19T14%3A47%3A52%2B00%3A00+TO+2017-12-03T14%3A47%3A52%2B00%3A00%5D&page=251
2018-11-20 18:24:03,436Z   main - INFO State update: adding bookmarks.agents.offset.id = null
2018-11-20 18:24:03,436Z target - INFO Exiting normally
2018-11-20 18:24:03,437Z   main - INFO State update: adding bookmarks.chats.chat.end_timestamp = "2017-11-19T14:47:52Z"
2018-11-20 18:24:03,437Z   main - INFO State update: adding bookmarks.chats.offset.chat.next_url = "https://www.zopim.com/api/v2/chats/search?q=type%3Achat+AND+end_timestamp%3A%5B2017-11-19T14%3A47%3A52%2B00%3A00+TO+2017-12-03T14%3A47%3A52%2B00%3A00%5D&page=251"
2018-11-20 18:24:03,437Z   main - INFO State update: adding currently_syncing = "chats"
2018-11-20 18:24:03,527Z   main - INFO Target exited normally with status 0
2018-11-20 18:24:03,529Z   main - INFO Exit status is: Discovery succeeded. Tap failed with code 1 and error message: "400 Client Error: BAD REQUEST for url: https://www.zopim.com/api/v2/chats/search?q=type%3Achat+AND+end_timestamp%3A%5B2017-11-19T14%3A47%3A52%2B00%3A00+TO+2017-12-03T14%3A47%3A52%2B00%3A00%5D&page=251". Target succeeded.

[luandy64]

I'm labeling this as help wanted for anyone who wants to explore this more.

[nicklupu]

+1 on getting this error when trying to use stitch. Any ideas how to get around?

[jamiedeveney]

+1 on this as well. Any updates on getting around this? I've tried removing replication from account and agents, along with ban, but still getting the same error.

[sarandeeps]

Getting the same error now. It will be great help if anyone can guide on how to resolve the same. Thanks.

[brethaueter]

Getting the same error. Seems related to a previous update that limited the start and end time pulled to 14 days in order to limit the total chats retrieved from the endpoint to 10k or less. If you exceed 10k (40 per page x 250 pages) on the 251st page you will always get this error. Seems enough people are having it that we may want to reduce this start and end time window to 7 days.

[brethaueter]

Alternately, if this has been parameterized in the code to accept a custom start and end time, we need an update in Stitch to allow this to be set in the Integration.