New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Status doesn't properly load extended tweets from streaming APIs #506

robmathers opened this Issue Sep 16, 2017 · 2 comments


None yet
3 participants

robmathers commented Sep 16, 2017


The Status.NewFromJsonDict method doesn't properly load extended tweets with an extended_tweet payload, which is how the streaming APIs provide extended tweets

Steps to Reproduce

  1. Load a JSON dict of an extended tweet in Twitter's "Compatibility with additional extended_tweet in payload" mode. You can find a sample here, or use GetStreamSample() and check for tweets with truncated = True.
  2. Use Status.NewFromJsonDict to convert the JSON dict to a Status.
  3. The status will just have a text property, without a full_text, even when the API instance has tweet_mode set to extended. Other extended entities are also not available.


By copying the keys from the JSON's extended_tweet object to the root object, NewFromJsonDict properly parses the tweets as an extended tweet, with full_text available.


# Where json_dict is a dictionary parsed from the JSON of a tweet in this mode
if json_dict.has_key('extended_tweet'):
    for key, value in json_dict['extended_tweet'].items():
        json_dict[key] = value
converted_tweet = Status.NewFromJsonDict(json_dict)
# converted_tweet.full_text will be populated and not truncated

This comment has been minimized.

zefoo commented Dec 26, 2017

I'm not able to get this to work. Any updates on this? I have a search result and I try something like this:


And it seems to be empty. Any ideas?

Thank you!

jeremylow added a commit that referenced this issue Mar 10, 2018

fix issue with streaming tweets loading extended_tweet data
assigns each node in the extended_tweets dict
to the top-level node
tweet[extended_tweet][entities] > tweet[entities]
for example. this fixes an issue where tweets
were not being fully hydrated from the streaming

close issue #506; close issue #491

This comment has been minimized.


jeremylow commented Mar 10, 2018

Fix lands in 564e6bb

@jeremylow jeremylow closed this Mar 10, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment