List support

[codilau]

Hey there, thanks for the good work.
For my use case i'd need to be able to scrape multiple lists.
Is there a way to ask the GraphQL endpoint instead of passing through the list:id search query?
thanks

[vladkens]

Hi, @codilau.

I haven't looked into it, but I've seen something like that in the API calls.

Do you have an example of a list in the twitter interface (web app) so we can see what requests the tweeter makes to its API and add this to the library?

I personally don't use twitter enough, so I'm not very familiar with all its functions.

[codilau]

Hi, thanks for looking into this subject. It's helpful to have this feature because it allows curated content with very few api calls.
The example uri looks like this : "https://twitter.com/i/lists/1494877848087187461" A list can have up to 5k accounts.
There is the possibility to use the search query "list:1494877848087187461", but it would make more sense not to use that endpoint if a dedicated one exists in their graphQL implementation.
Of course, I'm unsure if any of this is relevant now since Elon has a hard-on on scraping.
Again, thanks for your work!

[codilau]

Digging in a bit i found this:
endpoint = "/graphql/ZBbXrl37E6za5ml-DIpmgg/ListLatestTweetsTimeline"

[vladkens]

Hi, @codilau.

Thanks for the explanation. I will try to add this functionality as soon as possible.

So, API should be like:

async for tweet in api.list_timeline(LIST_ID, limit=100):
    print(tweet.id)

Its right?

[codilau]

@vladkens absolutely, thanks!

[vladkens]

Hi, @codilau.

Added lists support to v0.5.0, update please:

pip install --upgrade twscrape

Then:

twscrape list_timeline 1494877848087187461
twscrape list_timeline 1494877848087187461 --raw

Or from code:

async for tweet in api.list_timeline(1494877848087187461):
    print(tweet.id)

[codilau]

wonderful, thank you @vladkens!