Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vienna is making Inoreader accounts get blocked #1252

Open
dupilax opened this issue Mar 11, 2019 · 42 comments
Open

Vienna is making Inoreader accounts get blocked #1252

dupilax opened this issue Mar 11, 2019 · 42 comments
Assignees
Milestone

Comments

@dupilax
Copy link

@dupilax dupilax commented Mar 11, 2019

Hello

Today there is an issue with Inoreader.
The message from Vienna is wrong password, but the password is correct.
There was a smililar issue in 2016.

Am I alone ?

Vienna macOS version 3.5.4 :8b11d92d:

Thanks

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Mar 11, 2019

Same problem here with current git master:35181a4f: (7088). Not related to issue with bad characters in password.

@defrilitus

This comment has been minimized.

Copy link

@defrilitus defrilitus commented Mar 11, 2019

Same problem here, as well (my password characters just letters and digits). I tried reinstalling a couple of Viennna's previous versions, but that has no effect. I also tried changing my username/password on Inoreader, but that didn't work either. I've happily used Vienna for years, but am now asking myself whether I should upgrade to News Explorer.

@Smy

This comment has been minimized.

Copy link

@Smy Smy commented Mar 12, 2019

Same problem, maybe an InoReader modification : https://twitter.com/Inoreader/status/1105350573904326656

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Mar 12, 2019

Wow, I hope they didn't just roll that SSL change out without a grace period for developers to migrate.

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Mar 12, 2019

I contacted InoReader support and got the following response:

This has nothing to do with the SSL rollout.

It looks like Vienna's user agent got blocked because of multiple excessive
floods to our API. Look at the attached screenshot. Some of those requests
are literally 100s per second. Those methods could become really expensive
for some users (when they have large number of feeds) and this compromises
our backend stability. You can share the screenshot from the log with the
developers. The automatic blocks are normally for 3 days, so it will be
released soon, but as soon as there is another similar flood the block will
be imposed again.

The Inoreader Support team

image

Cc: @barijaona @Eitot

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Mar 12, 2019

The support team wrote me to indicate that they manually removed the block, but that it will be blocked again if this continues.

@Smy

This comment has been minimized.

Copy link

@Smy Smy commented Mar 12, 2019

The support team wrote me to indicate that they manually removed the block, but that it will be blocked again if this continues.

Thank you ! It works fine now.

Do they say what is a normal delay between two requests ?

@dupilax

This comment has been minimized.

Copy link
Author

@dupilax dupilax commented Mar 12, 2019

Don't work for me right now...

Only User-Agent blocked, no IP ?

@Smy

This comment has been minimized.

Copy link

@Smy Smy commented Mar 12, 2019

Don't work for me right now...
Only User-Agent blocked, no IP ?

Did you checked your password in the Vienna settings ?

@dupilax

This comment has been minimized.

Copy link
Author

@dupilax dupilax commented Mar 12, 2019

password didn't change.

change public IP and it si working now

@Jacketbg

This comment has been minimized.

Copy link

@Jacketbg Jacketbg commented Mar 13, 2019

Inoreader dev here. This issue with Vienna is still not fixed and we will be forced to block access again. It looks like it is happening for quite a while, but lately it has become quite a burden for the backend and enough to trigger sms alerts to the ops team every night. I am attaching a log file from tonight from just one IP address and there are dozens of those in our logs. Just look at the timestamps.
It looks like you are calling the /reader/api/0/stream/items/ids method for synchronizing read and starred states separately for each feed and you are doing this in parallel. That's not a good design. You can optimize this with a single request without an s parameter. Yes for heavy accounts this will probably not sync everything, but otherwise you are thrashing our backend and we can't allow this to continue.

@josh64x2

This comment has been minimized.

Copy link
Member

@josh64x2 josh64x2 commented Mar 13, 2019

@Jacketbg thanks for the feedback- we'll get this fixed by the weekend. Sorry for the trouble !

@josh64x2 josh64x2 changed the title Vienna Inoreader wrong password Vienna is making Inoreader accounts get blocked Mar 14, 2019
@josh64x2 josh64x2 pinned this issue Mar 14, 2019
@josh64x2

This comment has been minimized.

Copy link
Member

@josh64x2 josh64x2 commented Mar 20, 2019

Sorry for the delay in fixing this - I am working on it and it will take a few more days :(

@Jacketbg

This comment has been minimized.

Copy link

@Jacketbg Jacketbg commented Mar 20, 2019

No worries. We have tweaked our SMS alerts and it's not causing real outages, just additional load that we'd like to avoid. It will be good if you can provide us with a range of builds that are affected by this issue so we can identify them later.

@barijaona

This comment has been minimized.

Copy link
Member

@barijaona barijaona commented Mar 24, 2019

Hi @Jacketbg,
I am currently the main maintainer for the part of the code which is causing this. Sorry for the delay in tackling this, but I had two hectic weeks.

I examined the log file you transmitted. The user seems to have a slightly agressive configuration :

  • Vienna is set to check for new articles every 15 minutes (which is the maximum. Default is three hours).
  • the user has 170 feeds hosted on Inoreader

So, every 15 minutes, this copy of Vienna is launching approximatively 510 requests, as there are three requests per feed :

  • retrieving new articles since the last refresh
  • getting IDs of unread articles
  • getting IDs of starred articles

You suggest that we should optimize Vienna and use a single request to get all unread articles (alternatively all starred articles), by omitting the s parameter.
My preliminary tests show that this could work with Inoreader, but unfortunately, this approach seems to be incompatible with other services like Bazqux (which returns an empty array) or FeedHQ (which complains about a bad request). May be I am missing something ?

Yes, current Vienna's implementation creates a lot of network traffic, but Vienna should limit itself to a maximum of 6 simultaneous connections with your server.
So I am a bit surprised that it creates such a heavy load on your backend…

… unless we are confronted to a modified development build of Vienna. The user agent "Vienna/7068 ..." makes me suspect this. The two latest "official" versions (3.5.3 and 3.5.4) which you can download as binaries from our websites should announce respectively "Vienna/7033 …" and "Vienna/7074…".

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Mar 24, 2019

@barijaona : I have not modified my build in any way -- I'm only running a dev build for the purposes of ensuring that I have the latest improvements to your application.

15 minutes does not seem like a terribly low setting for a refresh interval, and, in fact, Vienna allows for setting it to 5 minutes. I'm choosing to be less aggressive with my automatic refresh,. Furthermore, 170 feeds is not actually that large a number for someone who would pay to use a service like InoReader -- if you look at their pricing page, you'll see that even free users are allowed to create 150 feeds, while those on the lowest paid tier can create 500.

This seems to be a case where the app needs to make more efficient use of the available APIs, not a case where the user should be blamed for simply making use of a service he's paid for with a client that claims to support that service.

@Eitot

This comment has been minimized.

Copy link
Contributor

@Eitot Eitot commented Mar 24, 2019

The screenshot above suggests that this happens with the release version too. I suppose the httpMaximumConnectionsPerHost property will only limit simultaneous connections, which could still mean that it will not stop hundreds of consecutive connections if each one only takes a split second.

Perhaps queuing needs to implemented in the OpenReader class too.

barijaona added a commit to barijaona/vienna-rss that referenced this issue Mar 24, 2019
Trying to tackle resources abuse reported by Inoreader (issue ViennaRSS#1252)
@barijaona

This comment has been minimized.

Copy link
Member

@barijaona barijaona commented Mar 24, 2019

Having only a small glimpse of Inoreader logs, I can only speculate without being able to identify a specific configuration or person 😃

I am as surprised as anybody that a single client could stress Inoreader's server. Could Inoreader be much more popular among Vienna users than TheOldReader, Bazqux, FeedHQ and FreshRSS which did not (yet) report similar problems ?

Queuing is already implemented in the OpenReader class (which uses the same networking queue than RefreshManager). The choices available in the advanced preferences seem reasonable to me.

My current suspicion is that there are parameters in the request which might complicate Inoreader's database access, while they are simply ignored by other servers.
@tonycpsu : could you try my work-1252 branch ? It simplifies requests and implements HTTP pipelining.

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Mar 24, 2019

@tonycpsu : could you try my work-1252 branch ? It simplifies requests and implements HTTP pipelining.

I could, but it seems pretty useless to do so without coordinating with the InoReader folks to see if the number of requests is alleviated on their end.

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Apr 7, 2019

@barijaona , I tried this work-1252 branch today. No idea whether it's helping the InoReader folks, but the refresh process is taking perhaps 10-20 times as long as it did before.

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Apr 8, 2019

The updates seem quicker today, for what it's worth -- maybe the first refresh took longer. Things are no worse than before on my end -- not sure what it's looking like server-side.

@pintapass

This comment has been minimized.

Copy link

@pintapass pintapass commented Apr 18, 2019

Any ETA on when this is going to be fixed? I appreciate the vast amounts of time and effort the devs put into making Vienna available, but this critical issue has been unresolved for more than month.

@josh64x2

This comment has been minimized.

Copy link
Member

@josh64x2 josh64x2 commented Apr 19, 2019

@barijaona do you have time to keep working on a solution?

@pintapass

This comment has been minimized.

Copy link

@pintapass pintapass commented Apr 26, 2019

@josh64x2 I'm guessing no?

@josh64x2

This comment has been minimized.

Copy link
Member

@josh64x2 josh64x2 commented Apr 28, 2019

Yup, I'll try and pick this up over the next week.

@josh64x2 josh64x2 self-assigned this Apr 28, 2019
@josh64x2 josh64x2 added this to the v3.5.5 milestone Apr 28, 2019
@barijaona

This comment has been minimized.

Copy link
Member

@barijaona barijaona commented Apr 29, 2019

I have been busy and often offline during last few weeks.

However, I took a look at it and I think this could be dealt by having specific treatment for "Refresh all", while keeping current logic for refreshing individual feeds.

@josh64x2

This comment has been minimized.

Copy link
Member

@josh64x2 josh64x2 commented Apr 29, 2019

Thanks @barijaona :)

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Feb 7, 2020

I'm having sync issues again, and the inoreader status page says there are no problems.
CC: @Jacketbg @barijaona @josh64x2

@bythoss

This comment was marked as spam.

Copy link

@bythoss bythoss commented Feb 7, 2020

@Jacketbg

This comment has been minimized.

Copy link

@Jacketbg Jacketbg commented Feb 7, 2020

It's been a year and this issue still isn't fixed. I am sorry but this is causing us infrastructure issues and in today's case - several outages. The screenshot from our logs says enough. This is only one client and there are hundreds. Each request is trying to fetch 1000 items. We do not have the scale of Google and Facebook. It is absurd to make so many requests per second to our backend for a simple sync operation. This needs to be fixed before we can allow requests from Vienna again.

Screenshot 2020-02-07 at 20 47 55

@bythoss

This comment was marked as spam.

Copy link

@bythoss bythoss commented Feb 7, 2020

@pintapass

This comment has been minimized.

Copy link

@pintapass pintapass commented Feb 7, 2020

It's been a year and this issue still isn't fixed. I am sorry but this is causing us infrastructure issues and in today's case - several outages. The screenshot from our logs says enough. This is only one client and there are hundreds. Each request is trying to fetch 1000 items. We do not have the scale of Google and Facebook. It is absurd to make so many requests per second to our backend for a simple sync operation. This needs to be fixed before we can allow requests from Vienna again.

Screenshot 2020-02-07 at 20 47 55

@Jacketbg, you've got one less instance of Vienna to deal with. I switched to Reeder back when it was clear this issue was never going to be dealt with.

@bythoss

This comment was marked as spam.

Copy link

@bythoss bythoss commented Feb 7, 2020

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Feb 7, 2020

I understand the situation InoReader is in here, but at the same time, I understand that Vienna is a FOSS project. As a FOSS developer myself, I don't think I'm owed any particular response time for an issue of this nature, but I also do find myself tempted to use other clients. I did my part by trying the work-1252 developer build as requested and reporting my experience, but those fixes were never merged for some reason.

Really at a loss as to how to proceed here, and Reeder doesn't check enough boxes in terms of functionality as compared to Vienna for me. Going to muddle through with the Inoreader web interface for a while and hope the Vienna devs take this issue more seriously in the future.

@pintapass

This comment has been minimized.

Copy link

@pintapass pintapass commented Feb 7, 2020

I understand the situation InoReader is in here, but at the same time, I understand that Vienna is a FOSS project.

I share your sympathy toward FOSS projects, but the ball seems to have been dropped hard on this one, and I'd have to prioritize InoReader—the cornerstone of my RSS usage—over a reader like Vienna. Also agree that Reeder isn't perfect, though it's been a pleasant stopgap. I've been waiting for the resurrected NetNewsWire (also freeware) to implement InoReader syncing.

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Feb 7, 2020

I guess I'm in a different position. If forced to choose between Vienna with a different backend and Inoreader with a different MacOS client, I think I'd be tempted to try one of the other backends. Things might be different if I could find a better Android client with Inoreader support, but the ones that exist are pretty poor ever since NewsPlus stopped working.

Really hope this bug can be addressed soon so I don't have to make the decision, though.

@josh64x2

This comment has been minimized.

Copy link
Member

@josh64x2 josh64x2 commented Feb 7, 2020

Thank you all for the understanding.

I have recently been gathering API documentation for Open Reader so I can rewrite our Open Reader code. I'm hoping inoreader hasn't deviated too much from the "standard". This issue being revived has given me a bit more motivation to continue on the Open Reader rewrite!

@bythoss

This comment was marked as spam.

Copy link

@bythoss bythoss commented Feb 7, 2020

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Feb 8, 2020

I appreciate that the team is taking this more seriously now, @josh64x2 . For now, I'm finding the newly-redesigned InoReader web interface adequate for my needs once I customized the theme / fonts to my liking, but it's definitely a stopgap solution, and not nearly as usable as a good desktop application.

@Eitot

This comment has been minimized.

Copy link
Contributor

@Eitot Eitot commented Feb 8, 2020

I have had a chance to look at this too. I have to admit that I do not understand the proposed solution. From what I can gather, stream/items/ids returns just this (just showing one itemRefs item):

{
  "items": [
  ],
  "itemRefs": [
    {
      "id": "23224933824",
      "directStreamIds": [
      ],
      "timestampUsec": "1581188915436283"
    },
…
  ],
  "continuation": "ydo75E9NzcfD"
}

That chronological list of item IDs is not at all useful without passing xt to limit the results to either read, unread or starred items. Furthermore, the number of results is limited to at most 1000. Exceeding that number will require another request with the continuation parameter (as shown above).

Assuming that a user has 5 feeds with an unknown number of items, Vienna would have to make at least 15 fetch requests to retrieve usable data; even more if a fetch result returns more than 1000 results. Not adding the s parameter might reduce the total number of requests Vienna has to make, but only if there are feeds that do not have as many items. If the proposed solution is to avoid having paginated requests as well though – what seems to be implied here – then I see this solution as undesirable, because then only some items might be updated, whereas others are not.

My tentative conclusion is that the request does not seem that useful for what it is currently used for. Going by Inoreader’s documentation, they do not seem to support the stream/items/contents request (cf. FeedHQ) which would allow clustering item IDs into a single request to retrieve their read/starred state, which is more in line with what Vienna would need to mirror the local database to Inoreader.

I wonder then if it would be better to stop using stream/items/ids altogether for Inoreader and not sync the read/starred state of items from the server at all, effectively making the fetching one-directional.

@bythoss

This comment was marked as spam.

Copy link

@bythoss bythoss commented Feb 8, 2020

@tonycpsu

This comment has been minimized.

Copy link

@tonycpsu tonycpsu commented Feb 9, 2020

To the extent that a user vote matters in this discussion, a solution that only synced read/starred state one way would defeat much of the point of sync in the first place, and would have me switching to another client and/or backend, or possibly muddling through with the InoReader web interface until something better comes along.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants
You can’t perform that action at this time.