Can't find any tweet

[G-Pavel]

Hi guys!
Can't find any tweets from user @BPharmCatalyst
[!] No more data! Scraping will stop now. found 0 deleted tweets in this search.

Could you check please:

twint -u BPharmCatalyst

Maybe it's just my mistake ((

[minamotorin]

Same for me.

twint uses Twitter search for scrape user's Tweets.
But @BPharmCatalyst's Tweets are not shown.
The cause may be Twitter's search rules.

Alternatively, you can scrape user's Tweets and RTs via twint -u BPharmCatalyst --retweets.
Only recent Tweets can be displayed.

This option will cause JSONDecodeError.
About the error, see #1335.

[G-Pavel]

Same for me.

twint uses Twitter search for scrape user's Tweets. But @BPharmCatalyst's Tweets are not shown. The cause may be Twitter's search rules.

Alternatively, you can scrape user's Tweets and RTs via twint -u BPharmCatalyst --retweets. Only recent Tweets can be displayed.

This option will cause JSONDecodeError. About the error, see #1335.

Many thanks!
I fixed the error JSONDecodeError as you said.
And now instead
twint.run.Search(c)
I use something like this to search for a tweets:

tweets = []

c = twint.Config()
c.Limit=100
c.Store_object_tweets_list = tweets

twint.run.Profile(c)
    
tweets = c.Store_object_tweets_list

p.s. Damn, twint needs a major update ((

[lyhtllw]

@G-Pavel Hi,I failed to get tweets from user @BPharmCatalyst by your code ,Is there anything wrong with my use?

tweets = []

c = twint.Config()
c.Username = "BPharmCatalyst"
c.Limit=100
c.Store_object_tweets_list = tweets

twint.run.Profile(c)
    
tweets = c.Store_object_tweets_list```

[G-Pavel]

@G-Pavel Hi,I failed to get tweets from user @BPharmCatalyst by your code ,Is there anything wrong with my use?

tweets = []

c = twint.Config()
c.Username = "BPharmCatalyst"
c.Limit=100
c.Store_object_tweets_list = tweets

twint.run.Profile(c)
    
tweets = c.Store_object_tweets_list```

Hi,
It twint -u BPharmCatalyst --retweets doesn't work anymore:

[!] Twitter does not return more data, scrape stops here.

I don't know what to do, it's complete nonsense ((
@minamotorin do you or anyone else have any ideas?

[minamotorin]

@lyhtllw
You forgot c.Store_object = True.

# This patch is in the public domain.
  tweets = []

  c = twint.Config()
  c.Username = "BPharmCatalyst"
  c.Limit=100
  c.Store_object_tweets_list = tweets
+ c.Store_object = True

  twint.run.Profile(c)
    
  tweets = c.Store_object_tweets_list

@lyhtllw @G-Pavel
The cause is Twitter's changes.
I updated my code. minamotorin/twint minamotorin/twint

This code looks working. But I haven't tested the code seriously.
This means there might be omissions in the scraping results of Tweets (reports are welcome).

[G-Pavel]

@lyhtllw You forgot c.Store_object = True.

# This patch is in the public domain.
  tweets = []

  c = twint.Config()
  c.Username = "BPharmCatalyst"
  c.Limit=100
  c.Store_object_tweets_list = tweets
+ c.Store_object = True

  twint.run.Profile(c)
    
  tweets = c.Store_object_tweets_list

@lyhtllw @G-Pavel The cause is Twitter's changes. I updated my code. minamotorin/twint

This code looks working. But I haven't tested the code seriously. This means there might be omissions in the scraping results of Tweets (reports are welcome).

Seems to work, thanks a lot!

[Tegrisco-ZSQ]

@lyhtllw You forgot c.Store_object = True.

# This patch is in the public domain.
  tweets = []

  c = twint.Config()
  c.Username = "BPharmCatalyst"
  c.Limit=100
  c.Store_object_tweets_list = tweets
+ c.Store_object = True

  twint.run.Profile(c)
    
  tweets = c.Store_object_tweets_list

@lyhtllw @G-Pavel The cause is Twitter's changes. I updated my code. minamotorin/twint

This code looks working. But I haven't tested the code seriously. This means there might be omissions in the scraping results of Tweets (reports are welcome).

It's working, super helpful!!

[minamotorin]

This case is different, but it seems that the same problem may occur due to the account id changes.

E.g., if the id is changed from @before to @after, twint -u after return the tweets only after that point.
You can scrape the tweets until that point via twint -u before or twint -u after --retweets.

[Tegrisco-ZSQ]

@lyhtllw You forgot c.Store_object = True.

# This patch is in the public domain.
  tweets = []

  c = twint.Config()
  c.Username = "BPharmCatalyst"
  c.Limit=100
  c.Store_object_tweets_list = tweets
+ c.Store_object = True

  twint.run.Profile(c)
    
  tweets = c.Store_object_tweets_list

@lyhtllw @G-Pavel The cause is Twitter's changes. I updated my code. minamotorin/twint

This code looks working. But I haven't tested the code seriously. This means there might be omissions in the scraping results of Tweets (reports are welcome).

I tested the code. There are omissions in the searching results with c.Since and c.Until option. The number reported from the code mismatches with the number of tweets stored in csv.
@minamotorin do you have any idea about this issue? Thanks.

[minamotorin]

@Tegrisco-ZSQ
Are there any example?
In my environment, the problem doesn't seems to be happening.

How can I see this number?

The number reported from the code

[Tegrisco-ZSQ]

@Tegrisco-ZSQ Are there any example? In my environment, the problem doesn't seems to be happening.

How can I see this number?

The number reported from the code

@minamotorin
The code I used:

def get_tweets(start_date, end_date, keyword, filename):
    c = twint.Config()
    c.Search = keyword
    c.Since = start_date
    c.Until = end_date
    c.Hide_output = False
    c.Lang = 'en'
    c.Count = True
    c.Store_csv = True
    c.Output = filename
    twint.run.Search(c)

get_tweets('2022-01-01', '2022-01-05', '$V','V.csv')

The keyword is a cashtag '$V'

The screenshot is the search result from code:

But I only got 317 tweets in the csv file.
V.csv

This issue happens when I use other keywords like '$GME', '$MSFT', etc.

[minamotorin]

@Tegrisco-ZSQ

This can happen with also original twint and this has nothing to do with my code.
Moreover, this is jut display bug and the mismatches doesn't mean there are omissions.

By the way, there is a another bug which causes omissions with since or until. #1136
Perhaps I fixed this in minamotorin@ef04733, and the display bug may also be fixed as a result.

About bug of this update, please report to minamotorin/twint.