New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

404 on /twitter_user_to_rss #59

Open
alexlamsl opened this Issue Oct 17, 2018 · 18 comments

Comments

Projects
None yet
@alexlamsl

alexlamsl commented Oct 17, 2018

Looks like the same as #52 - started about 10 hours ago.

Try https://twitrss.me/twitter_user_to_rss/?user=BillGates for example.

@anarchitectures

This comment has been minimized.

anarchitectures commented Oct 17, 2018

Noticed the same issue. Many of my twitter feeds are now either unreachable or unscrapable.

@kazux

This comment has been minimized.

kazux commented Oct 17, 2018

same issue here

@kav2k

This comment has been minimized.

kav2k commented Oct 17, 2018

The request that pulled Twitter data (through scraping with Yahoo's YQL) now returns:

"message": "html table is no longer supported. See https://policies.yahoo.com/us/en/yahoo/terms/product-atos/yql/index.htm for YQL Terms of Use"

He's dead, Jim..

@ciderpunx

This comment has been minimized.

Owner

ciderpunx commented Oct 17, 2018

Hi folks.

Yes it looks like twitter are now rate limiting TwitRSS.me's IP address. TBH I'm slightly surprised it has taken them so long!

You should be able to run your own version of the code with no problems assuming you're not looking at loads of users/searches.

I can still see the mobile version of twitter from that IP, but I'd need to rewrite the code to deal with that version of Twitter - not likely to happen until the weekend.

Sorry for the annoyance, but y'know, blame Twitter.

@kazux

This comment has been minimized.

kazux commented Oct 17, 2018

i was able to fix my issue by using the advanced search mode if you want to get feeds from a specific user it will be like this

https://twitrss.me/twitter_search_to_rss/?term=from%253ABillGates

@AlexeyVolkov

This comment has been minimized.

AlexeyVolkov commented Oct 17, 2018

Does anyone know other services (cannot get the Twitter api key)?

publicate.it

@miksh7

This comment has been minimized.

miksh7 commented Oct 17, 2018

I fixed it by changing the user name from low to upper case, but the next day the uppercase is blocked while lowcase is okay. Looks like Twitter's blocking logic is case-sensitive.

@piccolbo

This comment has been minimized.

piccolbo commented Oct 17, 2018

@piccolbo

This comment has been minimized.

piccolbo commented Oct 17, 2018

@cookieguru

This comment has been minimized.

Contributor

cookieguru commented Oct 19, 2018

@AlexeyVolkov:

publicate.it

Doesn't seem to work, and is loaded with CORS errors

@kimadagem

This comment has been minimized.

kimadagem commented Oct 19, 2018

@kazux, that worked for me. Thank you.

@kiwi810

This comment has been minimized.

kiwi810 commented Oct 23, 2018

@kazux that worked thanks, however now i think i see EVERY single post the user is posting even replies to other users rather than the ones he broadcasts on his/her own account.
Not that it bothers me but it kinda shows you what kind of person he/she really is when you see them reply rudely to other people on twitter lol wow im gobsmacked at the things ive been reading.

@garas

This comment has been minimized.

garas commented Oct 24, 2018

Until problem resolved, could you respond with single static RSS item like

<item>
  <title>Twitter are rate limiting requests from TwitRSS.me</title>
  <description>Twitter are rate limiting requests from TwitRSS.me, meaning it is effectively broken until I can think of a way round it. You can still run your own instance</description>
  <pubDate>Wed, 17 Oct 2018 00:00:00 +0000</pubDate>
  <link>https://github.com/ciderpunx/twitrssme/issues/59#issuecomment-430535935</link>
</item>

I didn't notice that feed is down, and Feedly didn't show sign until checked myself today.

ronilaukkarinen added a commit to ronilaukkarinen/khonsu that referenced this issue Oct 29, 2018

@p-ho

This comment has been minimized.

p-ho commented Oct 31, 2018

Would it be viable to fetch Twitter via proxies?
Just did a quick check: The two perl-scripts seem to run fine with torsocks for example (with increased timeout).

@jessicachinafile

This comment has been minimized.

jessicachinafile commented Oct 31, 2018

@kazux @kiwi810 to avoid all the replies and retweets you can just add the following to the end of the url: +exclude:replies+exclude:retweets
The Bill Gates feed is not a good example since he doesn't appear to reply to people :) . But I tested it with other feeds and it appears to be working.

@kiwi810

This comment has been minimized.

kiwi810 commented Nov 5, 2018

@p-ho What format for the feed did you use with tor?

@jessicachinafile thanks will try that out

@p-ho

This comment has been minimized.

p-ho commented Nov 5, 2018

@kiwi810 What exactly do you mean by format?
As for the URLs I've just used the "straightforward" ones such as https://twitter.com/billgates

@ariddell

This comment has been minimized.

ariddell commented Nov 11, 2018

Perhaps a rotating proxy service could be persuaded to donate to twitrss? For example, https://scrapinghub.com/crawlera quotes 150K monthly requests at $25/mo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment