no results #90
Comments
|
@bartengine27 I could not reproduce this? Is this still an issue? |
|
Having the same issue Yesterday it worked without any problems, today the scraper does not show any results. I tested it on 2 Windows computers and one MacOS in two different networks, the output is basically the same: twitterscraper trump -l 100 -d |
|
I noticed the results are consistent (not sure if this is the case on every platform/OS) if you set the number of processes to 1, could you try to run the query with the command line argument |
|
@taspinar twitterscraper trump -l 1000 -p 1 -d If the number of processes is set to default, one of the processes (the last one) returns a small number of tweets, while others return 0: twitterscraper trump -l 1000 -d INFO: Got 0 tweets for trump%20since%3A2017-10-04%20until%3A2017-10-27. 5 processes: twitterscraper trump -l 1000 -p 5 -d INFO: Got 0 tweets for trump%20since%3A2017-04-06%20until%3A2017-07-10. 2 processes twitterscraper trump -p 2 -o trump2.json -l 1000 INFO: Got 0 tweets for trump%20since%3A2017-01-01%20until%3A2017-08-26. when repeated in a minute, the number of tweets is 511 |
|
Good morning, I´m having the same issues. Im working with Ubuntu in 16.04 version and two days ago was perfectly working and today is not extracting any data. twitterscraper %23Executiveprotection%20since%3A2018-01-01%20until%3A2018-04-30 --lang en -o Executive.json INFO: Querying %23Executiveprotection%20since%3A2018-01-01%20until%3A2018-04-30 since:2006-03-21 until:2006-10-29 |
|
Not working on official site either @taspinar |
|
Just ran into this problem, think it has something to do with different Twitter responses depending on user agent. If I force the User Agent as: |
|
@mattcyu1 The forced user agent worked yesterday, however, today returned no results. I went back to the random user agent today and it's working great. |
|
I noticed that in some of the cases, BeautifulSoup does not find any I'm trying to implement an alternative approach with Regular Expressions. It looks like the result is more consist but the RegExp still needs some work. |
|
@mattcyu1 How do you change/force the User Agent when running the command? |
|
Good morning @taspinar, continue without extracting anything and how do you force the User Agent? |
|
Change the HEADERS_LIST in query.py such that it only contains the elements you want. |
|
Changing HEADERS_LIST as you suggested still returns zero tweets... |
|
It does not work and still returns zero tweets |
|
Make sure to run |
|
ok, thank you, so that partially worked...
Here's the new error message |
|
it works, thanks |
|
Good morning @taspinar , the extraction problem persists, there are day when it works. During May 4th I made an extraction that dropped 50.825 tweets. Today with all the changes I’ve made hasn’t extract any tweets, besides yesterday it was only extracted 7,000 tweets with the same query which was as follows: twitterscraper %40camachocastro%20since%3A2013-02-01%20until%3A2014-05-31 -o 26991572.json INFO: Got 0 tweets for %40camachocastro%20since%3A2013-02-01%20until%3A2014-05-31%20since%3A2009-11-20%20until%3A2010-07-01. try this other way: twitterscraper %40camachocastro%20since%3A2013-02-01%20until%3A2014-05-31 -p 5 -o prueba001.json INFO: Got 0 tweets for %40camachocastro%20since%3A2013-02-01%20until%3A2014-05-31%20since%3A2008-08-30%20until%3A2009-04-10. |
|
@CamiloVeloz your query I specify dates, but I use the You need to set the begin date with |
|
I believe that most of the 'not being able to get all tweets' issues are caused by the useragent provided by fake_useragent. Once this PR is merged, the newest versions should no longer have these issues. Can you guys have a look at the PR? |
* query.py : remove fake_useragent, move separate try / except - useragents will no longer be generated with the fake_useragent package. At the moment seven of the most popular useragent strings are written hardcoded in query.py In later stages this should be done via a separate module. -By removing the separate try / except for the JsonDecodeError, each process now also continues with retrying to get the data instead of exiting. This fixes #118, #115, #90
|
Hi there, Using the latest version of Twitterscraper in python 3.7.3, I'm getting the same got 0 tweets issue: INFO: Got 0 tweets for xxx%20since%3A2019-10-12%20until%3A2019-10-13. In fact, it's generating different numbers of tweets each time I run it. Using the same query a few days ago, there was no problem. Any ideas? And thanks for providing this library! |
|
I have got a very similar problems as everything mentioned before. It appears as if tweets from today are appearing, but the tweets from further back cannot be collected (even though the visible links do show tweets). It also appears as if it works on some days or some networks and not on others. Any explanation for this? INFO: queries: ['Trump since:2019-05-07 until:2019-05-08'] |
|
Apparently, this issue is still present. After scraping for few hours, i face this problem. Anyone has a solution? Changing the User-Agent was only a temporary solution. Introducing new a User-Agent does also not help. |
|
I am still having this issue. |
|
Me too. |
|
I need it immediately and got the same thing. can anyone help me? |
|
Same problem here |
|
how about this you can easily get twitter information without api |
|
Please try Duplicate of #296 please move discussion there |
|
INFO: queries: ['corona since:2020-05-07 until:2020-05-08'] |
|
I'm also having this problem. Are there any solutions? |
|
not yet
…On Wed, Jun 24, 2020 at 5:49 AM Malcolm Grossman ***@***.***> wrote:
I'm also having this problem. Are there any solutions?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#90 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANIWFLE7TFFDKAAKI2ET44TRYFBATANCNFSM4EQFSARA>
.
--
chhinder kaur
|
|
plz suggest another way to download historical tweets |
|
This may work as an alternative https://pypi.org/project/GetOldTweets3/ |
* query.py : remove fake_useragent, move separate try / except - useragents will no longer be generated with the fake_useragent package. At the moment seven of the most popular useragent strings are written hardcoded in query.py In later stages this should be done via a separate module. -By removing the separate try / except for the JsonDecodeError, each process now also continues with retrying to get the data instead of exiting. This fixes taspinar/twitterscraper#118, taspinar/twitterscraper#115, taspinar/twitterscraper#90
The following example does not return any tweet, expected?
twitterscraper Trump -l 100 -bd 2017-01-01 -ed 2017-06-01 -o tweets.json
INFO: Querying Trump since:2017-01-01 until:2017-01-09
INFO: Querying Trump since:2017-01-25 until:2017-02-02
INFO: Querying Trump since:2017-01-17 until:2017-01-25
INFO: Querying Trump since:2017-02-02 until:2017-02-10
INFO: Querying Trump since:2017-02-10 until:2017-02-18
INFO: Querying Trump since:2017-02-18 until:2017-02-26
INFO: Querying Trump since:2017-02-26 until:2017-03-06
INFO: Querying Trump since:2017-01-09 until:2017-01-17
INFO: Querying Trump since:2017-03-06 until:2017-03-14
INFO: Querying Trump since:2017-03-14 until:2017-03-22
INFO: Querying Trump since:2017-03-22 until:2017-03-30
INFO: Querying Trump since:2017-03-30 until:2017-04-07
INFO: Querying Trump since:2017-04-07 until:2017-04-15
INFO: Querying Trump since:2017-04-15 until:2017-04-23
INFO: Querying Trump since:2017-04-23 until:2017-05-01
INFO: Querying Trump since:2017-05-01 until:2017-05-09
INFO: Querying Trump since:2017-05-09 until:2017-05-17
INFO: Querying Trump since:2017-05-17 until:2017-05-25
INFO: Querying Trump since:2017-05-25 until:2017-06-01
INFO: Got 0 tweets for Trump%20since%3A2017-02-10%20until%3A2017-02-18.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-03-30%20until%3A2017-04-07.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-02-18%20until%3A2017-02-26.
INFO: Got 0 tweets for Trump%20since%3A2017-01-09%20until%3A2017-01-17.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-04-07%20until%3A2017-04-15.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-04-15%20until%3A2017-04-23.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-03-22%20until%3A2017-03-30.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-03-06%20until%3A2017-03-14.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-05-25%20until%3A2017-06-01.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-01-01%20until%3A2017-01-09.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-04-23%20until%3A2017-05-01.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-01-17%20until%3A2017-01-25.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-05-01%20until%3A2017-05-09.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-03-14%20until%3A2017-03-22.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-01-25%20until%3A2017-02-02.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-02-26%20until%3A2017-03-06.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-05-17%20until%3A2017-05-25.
INFO: Got 0 tweets for Trump%20since%3A2017-02-02%20until%3A2017-02-10.
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets (0 new).
INFO: Got 0 tweets for Trump%20since%3A2017-05-09%20until%3A2017-05-17.
INFO: Got 0 tweets (0 new).
The text was updated successfully, but these errors were encountered: