Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Twitterscraper was working fine till tuesday but now it shows 0 tweets #299

Closed
lubhaniagarwal opened this issue Jun 4, 2020 · 12 comments
Closed

Comments

@lubhaniagarwal
Copy link

I am facing problem while retrieving tweets with twitterscraper as it was working fine till tuesday but now it shows 0 tweets.
I even tried to clone the git repository and change header list, but still it is not working.
Is there anyway to fix this issue or anything wrong on twitterscraper side.
please help me out !!:((

@hakanyusufoglu
Copy link

Yes me too. Problem is back. Please help !

@lapp0
Copy link
Collaborator

lapp0 commented Jun 4, 2020

Please share the command you're running, and any input files so I can try to reproduce.

@lubhaniagarwal
Copy link
Author

lubhaniagarwal commented Jun 4, 2020

first installed twitterscraper using pip install twitterscraper. It was working fine till tuesday.
then i uninstalled it
cloned repo-https://github.com/taspinar/twitterscraper.git
changed the header list in query.py to :HEADERS_LIST = ['Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.93 Safari/537.36']
then run - python setup.py install

but its not working.

command that i am working on is:

twitterscraper "amazon delivery" -bd 2010-01-01 -ed 2020-05-26 --limit 100000 --lang en --output xyz.json

thanks a lot.

@lapp0
Copy link
Collaborator

lapp0 commented Jun 4, 2020

I ran it from origin/master without making any changes to the headers and got 653 results. Could you share the programs full output?

@lubhaniagarwal
Copy link
Author

error.txt
here is the commands I'm running on command prompt and results that i get .
thank you

@lapp0
Copy link
Collaborator

lapp0 commented Jun 4, 2020

I'm not sure why I was able to retrieve tweets. It appears other users are suggesting twitter doesn't allow JS disabled.

#296 (comment)

I will try to hack together a selenium solution tonight.

@lubhaniagarwal
Copy link
Author

lubhaniagarwal commented Jun 4, 2020 via email

@lapp0
Copy link
Collaborator

lapp0 commented Jun 5, 2020

Please try this and let me know how it works for you #302

@abhisheksaxena1998
Copy link

HEADER = {'User-Agent': random.choice(HEADERS_LIST), 'X-Requested-With': 'XMLHttpRequest'}

This line in query.py will surely fix this issue.

@lubhaniagarwal
Copy link
Author

@abhisheksaxena1998 thanks a lot .It started working now.
@lapp0 I will try this #302 and let you know. Thanks you so much! :)

@lapp0
Copy link
Collaborator

lapp0 commented Jun 5, 2020

Duplicate of #296

@lapp0 lapp0 marked this as a duplicate of #296 Jun 5, 2020
@lapp0 lapp0 closed this as completed Jun 5, 2020
@lapp0
Copy link
Collaborator

lapp0 commented Jun 5, 2020

@lubhaniagarwal other users are suggesting that legacy now works, give origin/master another try

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants