New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Status doesn't properly load extended tweets from streaming APIs #506

Closed
robmathers opened this Issue Sep 16, 2017 · 2 comments

Comments

Projects
None yet
3 participants
@robmathers

robmathers commented Sep 16, 2017

Summary

The Status.NewFromJsonDict method doesn't properly load extended tweets with an extended_tweet payload, which is how the streaming APIs provide extended tweets

Steps to Reproduce

  1. Load a JSON dict of an extended tweet in Twitter's "Compatibility with additional extended_tweet in payload" mode. You can find a sample here, or use GetStreamSample() and check for tweets with truncated = True.
  2. Use Status.NewFromJsonDict to convert the JSON dict to a Status.
  3. The status will just have a text property, without a full_text, even when the API instance has tweet_mode set to extended. Other extended entities are also not available.

Workaround

By copying the keys from the JSON's extended_tweet object to the root object, NewFromJsonDict properly parses the tweets as an extended tweet, with full_text available.

Example:

# Where json_dict is a dictionary parsed from the JSON of a tweet in this mode
if json_dict.has_key('extended_tweet'):
    for key, value in json_dict['extended_tweet'].items():
        json_dict[key] = value
converted_tweet = Status.NewFromJsonDict(json_dict)
# converted_tweet.full_text will be populated and not truncated
@zefoo

This comment has been minimized.

zefoo commented Dec 26, 2017

I'm not able to get this to work. Any updates on this? I have a search result and I try something like this:

result.NewFromJsonDict(result.AsDict()).full_text

And it seems to be empty. Any ideas?

Thank you!

jeremylow added a commit that referenced this issue Mar 10, 2018

fix issue with streaming tweets loading extended_tweet data
assigns each node in the extended_tweets dict
to the top-level node
tweet[extended_tweet][entities] > tweet[entities]
for example. this fixes an issue where tweets
were not being fully hydrated from the streaming
apis.

close issue #506; close issue #491
@jeremylow

This comment has been minimized.

Collaborator

jeremylow commented Mar 10, 2018

Fix lands in 564e6bb

@jeremylow jeremylow closed this Mar 10, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment