Support for OAuth 1.0a (needed for protected accounts) #36
Comments
|
Hi! I think I have a hunch of what part of the code is misbehaving, but I'd have to debug it to make sure. And paste here the output you get, that way I can analyze Twitter's API response and see the JSON myself. |
|
@robertoszek I've added that logger line to |
|
Is the account private by any chance? Do you have a bearer token able to consult their tweets? You can try doing the request directly and see if you get the same result: curl -H "Authorization: Bearer <your-token-here>" 'https://api.twitter.com/2/tweets/search/recent?max_results=40&query=from%3A<twitter-account-here>&poll.fields=duration_minutes%2Cend_datetime%2Cid%2Coptions%2Cvoting_status&media.fields=duration_ms%2Cheight%2Cmedia_key%2Cpreview_image_url%2Ctype%2Curl%2Cwidth%2Cpublic_metrics&expansions=attachments.poll_ids%2Cattachments.media_keys%2Cauthor_id%2Centities.mentions.username%2Cgeo.place_id%2Cin_reply_to_user_id%2Creferenced_tweets.id%2Creferenced_tweets.id.author_id&tweet.fields=attachments%2Cauthor_id%2Ccontext_annotations%2Cconversation_id%2Ccreated_at%2Centities%2Cgeo%2Cid%2Cin_reply_to_user_id%2Clang%2Cpublic_metrics%2Cpossibly_sensitive%2Creferenced_tweets%2Csource%2Ctext%2Cwithheld'Replacing |
|
The account isn't private (Twitter username is |
|
Actually, I forgot, I changed it recently and this is the actual request now: curl -H "Authorization: Bearer <your-token-here>" 'https://api.twitter.com/2/users/<twitter-id>/tweets'If you don't know the Twitter's account ID, you can get it here using their Twitter handle: Either way, I have no problem getting the Twitter account tweets with my token and I can't reproduce the issue while debugging pointing to that Twitter account. This is very interesting. |
|
Here ya go! twitter_base_url: https://api.twitter.com/1.1
# Change this to your Fediverse instance
pleroma_base_url: https://mastodon.linuxbox.ninja
# How many tweets to get in every execution
# Twitter's API hard limit is 3,200
max_tweets: 40
# Twitter bearer token
twitter_token: **************
# List of users and their attributes
users:
- twitter_username: cryo
pleroma_username: 45792
pleroma_token: ***************
media_upload: true
nitter: false
visibility: "public"
sensitive: false
include_rts: false
include_replies: false
support_account: jimmyb
bio_text: "\U0001F916 BEEP BOOP \U0001F916 \n I'm a bot that mirrors {{ twitter_username }}'s Twitter's\
\ account. \n \n " |
|
Huh, is this the full config? I tried it with your config to get the tweets from January and ran it multiple times but I wasn't able to reproduce the issue. |
|
Ok, that tipped me off. That wasn't the full config. The issue is I used the same So running the cURL command before, gave me this: The actual account having an issue IS private. If I leave the date empty, and just hit enter at the prompt, I get this: |
|
Oh, test release v0.7.3 fixed that empty date business, you could try installing it like this: pip install --index-url https://test.pypi.org/simple/ pleroma-bot --upgradeAnd then run it against the private account, see if you get the same result. |
|
Hrmm, it won't let me install the test version for whatever reason: However even if I manually copy the Python files from the site it fails. |
|
Well, looking more into it, the root of the issue is that OAuth 2.0 Bearer tokens do not have access to protected tweets. The only way to retrieve those is to obtain OAuth 1.0a Consumer API Keys and Access Tokens performing a 3-legged OAuth so you can act on behalf of a Twitter account (which has to be following or be the owner of the protected account). I'll try generating them and see if we can adapt the code to somehow support those, it's not a high priority right now to be honest. |
robertoszek
added
enhancement
robertoszek
changed the title
|
Hello again, pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pleroma-bot --upgradeIf you have trouble getting it installed, maybe try uninstalling it first: pip uninstall pleroma-botAfter upgrading, you'll need to add to the protected user section in the config the following: consumer_key: xxxxxxxxxxxxxxxxxxxxxxxxx
consumer_secret: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
access_token_key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
access_token_secret: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxTo obtain the I was able to reproduce the issue with a Twitter account with protected tweets and this fixed it for me. |
|
Hey @robertoszek! It looks like it's working! The first time it errored out because of a 404 error (may want to consider some further error trapping/catching. But that didn't surprise me since I selected to get all tweets and the account hasn't had much activity since like 2017 and before (one of the images had a 404). It looks like a lot of the users I have setup hadn't ever been synced which is odd because I thought I ran an initial sync when I got this all setup at the beginning of the month. They all seem to be working now though so no big deal! I really appreciate the help and work to get this going. For what it's worth it looks like you can get all the necessary tokens/keys from the Twitter developer portal UI:
|
|
Excellent, good to hear it worked! This issue will be automatically closed when a pull request merges the changes into master, don't hesitate to add additional comments if you find any issues with the implementation. |
|
The only other issue I seemed to have come across is accounts with no tweets cause an error. Perhaps catching that and just moving on would be good there. It seems silly to "sync" an account with no tweets, but for the sake of having "good" code, might as well as try to avoid dying and just showing a warning or something and moving on to the next account. 😁 |
|
I've not come across an error with Twitter accounts that have no tweets. It didn't fail for me when the account had no new tweets (since last Fediverse post) nor when an account had no tweets whatsoever (tried both with protected and public). And I wasn't able to make it happen either no matter if it was a first run or not. What error gets raised? And did you happen to save the log? |
|
Ok, it seems to only happen during the cronjob, but not when running the script manually. Perhaps because when the cron runs it doesn't get any input from the user for the date? |
Yeah, that looks like it's exactly what's happening. The first time you add a new user you should run it manually at least once, so it doesn't require your input anymore in following executions. |
So I did it manually and gave it a date and it was fine. But now during cron runs it still has an issue. Would it be best to just add the |
That's what I'd try next, yes. Did the Fediverse account had any posts/toots published after the manual run or did it remain empty? Oh! I forgot to add earlier that there have been a couple of minor fixes since the last test release, if you're keen of giving it a spin and see if it solves any of the issues you're experiencing ( |
|
@robertoszek It remained empty. After looking at the account it actually does have some tweets, which are pretty old but I guess it should still fetch them when I tell the script to get everything. https://twitter.com/cowardswayout So there may still an issue here. I am giving 0.7.9 a shot now. |
|
Actually, Twitter's API only lets you go back to ' |
|
I think you're definitely right. It looks like the newest Tweet is from Jul 16, 2010. Alright so, I think it would be best if I just ran my cron(s) with the |
|
It's safer that way, as it should never ask for your input. |
|
@robertoszek Awesome, sound good. I really appreciate all your quick communication in getting me all setup. I think we can consider this/these issues resolved! |
|
No worries, thank you for your patience and sticking through with the troubleshooting! haha |
Hiyo!
First off apologies on not getting around to testing 0.7.0 until now (#28, #29). Things were working initially for the first run it seems, but I am now getting an error:
It appears to be getting "stuck" at the same Twitter account during each run. Is there anything else I can provide to help troubleshoot?
The text was updated successfully, but these errors were encountered: