Random "Unable to find guest token" with TwitterUserScraper

[palevell]

In the last few days, I have been getting Unable to find guest token exceptions with TwitterUserScraper.

I am running my code locally. The code retrieves user information for less than two dozen Twitter users. The error usually appears after 5 or 11 calls to TwitterUserScraper.

As a workaround, I applied a simple retry loop to my code, as follows:

import snscrape.modules.twitter as sntwitter
...
retries = 5
while retries > 0:
	try:
		result = sntwitter.TwitterUserScraper(twit, isUserId=False)
		...
		retries = -1
	except snscrape.base.ScraperException:
		if retries == 0:
			raise
		retries -= 1
		sleep(30)

Hopefully, this will help other users experiencing this issue.

[JustAnotherArchivist]

Yeah, I have also observed that occasionally since yesterday. I haven't spotted any patterns yet, and it's definitely happening more rarely for me than what you saw. Will investigate further on the weekend.

[JustAnotherArchivist]

Hopefully fixed by retrying the guest token retrieval in 525cd71.

[JustAnotherArchivist]

Post-mortem: Twitter started shifting the primary way they're handing out guest tokens to a different method in early December. The retries on the commit above helped as a crude workaround, but I have since added that other token retrieval method in commit 0336ce1, which should solve it properly, especially in combination with the token caching implemented in response to #326.