Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Urge users to update Tampermonkey #957

Closed
GreenLunar opened this issue Apr 10, 2016 · 37 comments
Closed

Urge users to update Tampermonkey #957

GreenLunar opened this issue Apr 10, 2016 · 37 comments
Labels
mightfix Ehh maybe... convince us all... UI Pertains inclusively to the User Interface.

Comments

@GreenLunar
Copy link
Member

Use an information bar (a line that pops up from top of a page), similarly to popup blocking or accept cookies notifications, that would ask users to update to recent version of Tampermonkey.

Also, add a News entry about this occurrence, so other developers of User script engines would take this overload scenario into consideration.

@derjanb, please post your feedback on this matter.

@Martii
Copy link
Member

Martii commented Apr 10, 2016

Not liking the too busy check ay ;) What's weird is usage almost always raises around 5pm-8pm EST and 5am-8am EST... odd that it's almost symmetrical time wise.

There's no reliable form of identification though that I know of... unless something changed in Chromium/Chromes/Androids identification. e.g. someone needs to tell us how to fingerprint it reliably before we would do that.

@Martii Martii added UI Pertains inclusively to the User Interface. mightfix Ehh maybe... convince us all... labels Apr 10, 2016
@Martii
Copy link
Member

Martii commented Apr 10, 2016

@GreenLunar
I think it might be better to have a feed, either RSS or Atom for #914 (how is the search going over there btw) ... however...

@derjanb
I don't see a feed (text or icons) on tampermonkey.net for this... Does this exist? Thanks for another moment of your time.

@derjanb
Copy link

derjanb commented Apr 10, 2016

@Martii
Not at the moment, cause I went to bed 2 a.m. after our conversation (CEST) and on Sundays my wife, my daughter and I usually visit my mother-in-law. However, after lunch there is some time to work. :)
I'm focusing on new Opera and Safari releases at the moment. Do you see many requests by those users? How is the traffic doing in general?

@Martii
Copy link
Member

Martii commented Apr 10, 2016

@derjanb
Like clockwork two times a day we get hit now... I've kept some of it at bay with memory checks just recently and will do some more after some more pondering.

If you could get a RSS feed up at some point we'll definitely add you when #914 gets completed... no rush because we don't have that implemented just yet. LOL ;)

@derjanb
Copy link

derjanb commented Apr 10, 2016

There's no reliable form of identification though that I know of... unless something changed in Chromium/Chromes/Androids identification. e.g. someone needs to tell us how to fingerprint it reliably before we would do that.

Tampermonkey sends Accept: text/x-userscript-meta, */* at meta data requests (if at all ;)) and Accept: text/x-userscript, */* when it requests a script. I don't think other script handlers send the second one. So this could help you to identify TM users regardless of their user agent, but only when they're downloading a script.

@Martii
Copy link
Member

Martii commented Apr 10, 2016

text/x-userscript, */*

That's what we are being bombarded with... plus we might have an issue with text/x-userscript-meta, */* which I'll look into right now.

@derjanb
Copy link

derjanb commented Apr 10, 2016

And the user agents? It's all caused by Chrome browsers or also Opera and Safari?

@Martii
Copy link
Member

Martii commented Apr 10, 2016

Well Chrome's UA's are bit duplicative... so I see Safari for sure.

Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36

@Martii
Copy link
Member

Martii commented Apr 10, 2016

Here's an Opera

Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.155 Safari/537.36 OPR/31.0.1889.174

@derjanb
Copy link

derjanb commented Apr 10, 2016

But it's hard to say whether this are normal or "faulty" requests, right?

@Martii
Copy link
Member

Martii commented Apr 10, 2016

Unlikely that it's normal as @TimidScript reported 70K installs... that's not normal.

@derjanb
Copy link

derjanb commented Apr 10, 2016

I'm aware that Chrome users cause a lot of requests at the moment and I guess that since TM is not doing a meta data request, but is downloading the complete script source, OUJS counts every update request as installation, but I might be wrong.

However, my question was whether the "faulty" requests are caused by other browsers as well. "Faulty" means repeated downloads for no reason. But I think this is very hard to say when the log files overflow.

@Martii
Copy link
Member

Martii commented Apr 10, 2016

The snapshot I grabbed in the terminal had thousands of Chrome starting UA's... when I get done splitting out the Accept header which has been extended by TM I'll see if that improves things with the meta routine routes. As far as requesting goes the scripts aren't that popular twice a day. ;)

@Martii
Copy link
Member

Martii commented Apr 10, 2016

K... now that I have dedicated time to answer things here.

OUJS counts every update request as installation, but I might be wrong.

See https://openuserjs.org/about/Frequently-Asked-Questions#q-does-openuserjs-org-have-meta- and https://openuserjs.org/about/Frequently-Asked-Questions#q-is-there-a-way-to-not-to-count-script-updates-with-this-sites-install-counter- and quacku's Q with my A at https://openuserjs.org/discuss/Huge_Number_of_Downloads#comment-153fcb25955 . I've had zero time to update our docs to reflect quacku's question.

was whether the "faulty" requests are caused by other browsers as well.

GM and GM Port do not have this issue... part of the issue is that we at OUJS didn't know that you were extending the Accept header e.g. from the original upstream spec is just text/x-userscript and nothing else... it's okay to extend as the Accept header has this option... but it was unexpected.

Bringing @JasonBarnabe into this to ensure he knows this too as a courtesy... just a FYI in case you didn't know and need to know.

But I think this is very hard to say when the log files overflow.

We don't log traffic like that... as @JasonBarnabe found out it killed his server... we only nab when we need to.

@Martii
Copy link
Member

Martii commented Apr 10, 2016

As far as an improvement with #962 and adding in the TM extension to the Accept header it didn't help... we're still at ~50% MEM (the limiting I set in place for serving scripts)

@derjanb
Copy link

derjanb commented Apr 10, 2016

One last question. How many different IPs do you see? I'd expect it to be a low number. But those IPs request the scripts again and again.

@Martii
Copy link
Member

Martii commented Apr 10, 2016

Need to find that in the request object first... just a few moments and I'll look at another snapshot.

@Martii
Copy link
Member

Martii commented Apr 10, 2016

More than 10... they whiz by very quickly... I would have to say that 90% of them are Chrome based UA's.


Largest recurrence is an IP in France. ;) OUI OUI!

@derjanb
Copy link

derjanb commented Apr 11, 2016

If you know the IPs you could try to add them to a iptables DROP rule. This means that no response packet is sent back to the source anymore and all requests should run into the TCP timeout. This might throttle the requests a little bit and may be worth a try.

@Martii
Copy link
Member

Martii commented Apr 11, 2016

How many different IPs do you see? I'd expect it to be a low number.

There are five main ones:

  • JAPAN
  • FRANCE
  • INDONESIA
  • INDONESIA
  • HUNGARY

However there are hundreds of little tiny ones that pop in and out... I also noticed install requests from Chrome UA's picking other Accept types like image/png when attempting to get the source... which is definitely not right.

If you know the IPs you could try to add them to a iptables DROP rule.

I've asked @sizzlemctwizzle to educate me on what he's set up that I don't have knowledge of what he did... I can IP block here but that's not the the VPS. I'd also like to keep this in node for contributors understanding as well... if needed one of us can block IP's in the OS. (and yes I know the IPs now)

@Martii
Copy link
Member

Martii commented Apr 11, 2016

image/webp,image/,/;q=0.8 /install/open2chExtender/open2chAdBlock.user.js
image/png,image/
;q=0.8,/;q=0.5 /install/open2chExtender/open2chAdBlock.user.js

This isn't normal... both from chrome based agents and multiple recurrences over hours.

@derjanb
Copy link

derjanb commented Apr 11, 2016

I agree, but it's caused by the script:

@Martii
Copy link
Member

Martii commented Apr 11, 2016

Well I'll filter that out eventually. Thanks for the tip... anyhow... I'm exhausted...bbi about 8ish hours. Feel free to look over things... I usually comment pretty well. Hope you publish a working meta check soon too. ;) :)

@Martii
Copy link
Member

Martii commented Apr 11, 2016

Some more new countries today

  • RUSSIA
  • SWEDEN
  • NORWAY

This was referenced Apr 11, 2016
@derjanb
Copy link

derjanb commented Apr 18, 2016

Hope you publish a working meta check soon too. ;) :)

@Martii The current beta version requests meta data (incl. x-userscript-meta) from the source URL (.user.js) if no @updateURL is set. This is a little different from other script hosting sites - they server meta data only at .meta.js URLs.

Is this OK for you or do you think it's better to use .meta.js URLs for OUJS based scripts as well?

@Martii
Copy link
Member

Martii commented Apr 18, 2016

if no @updateURL is set.

This part is confusing me this morning... so what happens if it is set? This FAQ entry is what we currently cover and some of the /install route (URI) pitfalls.


I would prefer not having an engine using the meta.json though that's typically used in userscripts if they really want it. e.g. keep the traffic to a minimum overall.

@derjanb
Copy link

derjanb commented Apr 25, 2016

if no @updateURL is set.

This part is confusing me this morning... so what happens if it is set?

Then it's used. What else should happen?

Do you still see issues caused by TM users with that faulty version? GreasyFork is back at a normal traffic level quite a while now. If not, then you could loosen the restriction a little bit, because it seems that even regular Tampermonkey users are affected by it. #999

@Martii
Copy link
Member

Martii commented Apr 25, 2016

Do you still see issues caused by TM users with that faulty version?

... and current release as you already referenced.

GreasyFork is back at a normal traffic level quite a while now.

He's probably not doing a deeper analysis of his data stream so that's not accurate... we still capture several clients that hit with a vengeance a couple of times a day but we quash them nicely so far. e.g. we are even better than before the ordeal.

Then it's used.

Good... it might be a good idea to send the header all the time if you know your extension wants to do an update check.

If not, then you could loosen the restriction a little bit

Absolutely not. Your update checks are minimum at 6 hours and if someone continually taps the install route they will be too many requested. We won't have a repeat of this. Until I know for sure that TM has a viable meta routine check we won't even consider "loosening" this DDoS prevention. Again TM isn't the only one with an issue here but everybody needs time to absorb what happened and what was revealed.

You asked me earlier elsewhere if other engines had issues... GM / GM Port has one on busy and VM Presto has one with installation... both projects have issues created.

@derjanb
Copy link

derjanb commented Apr 25, 2016

we won't even consider "loosening" this DDoS prevention

So an attempt to install a script, but then cancel the installation for whatever reason, just to feel certain that you want to install it 3 minutes later is a DDoS?

@Martii
Copy link
Member

Martii commented Apr 25, 2016

So an attempt to install a script, but then cancel the installation for whatever reason, just to feel certain that you want to install it 3 minutes later is a DDoS?

Correct... that's what viewing the source is in all the engines is (e.g. from whatever cache they use)... e.g. review the script first and decide if you want it installed. TM unveiled this exploit which is a good thing in part and research led into a few other arenas that have some repeat requests.

It's bad enough that I found several of our users gaming the install count intentionally... the known ones have been put to a stop although I doubt we'll ever trap all of them. I've done my best to balance inept practices with good practices on script installation. An inept practice is what you just mentioned about installing twice... take the time to decide... use the tools available... then install in the engine.

@derjanb
Copy link

derjanb commented Apr 25, 2016

Ok, this means two people at let's say at a company network are also doing a DDoS when both are trying to install the same script?

@Martii
Copy link
Member

Martii commented Apr 25, 2016

Then they will have to wait a little while in between requests... it's that simple.

@Martii
Copy link
Member

Martii commented Jul 25, 2016

@derjanb
What exactly is "externals" and why isn't @externals in your documentation at this moment? TIA

Ref:

externals

@derjanb
Copy link

derjanb commented Jul 25, 2016

@require and @resource are externals.

@Martii
Copy link
Member

Martii commented Jul 25, 2016

@derjanb

@require and @resource are externals.

Thanks.

So "Always" uses the browser cache then with the update interval? If so what is the current default caching time (no additional extensions and no presumed cache disabling)?

@derjanb
Copy link

derjanb commented Aug 1, 2016

Yes it always utilizes the browser cache regardless of the update interval setting.

If so what is the current default caching time

Tampermonkey uses a default update interval of 7 days. The browser caching time depends on the headers sent along with the resource.

@Martii
Copy link
Member

Martii commented Nov 23, 2017

This is a given so we really don't need to urge them. Hopefully google will handle this.

Closing.

@Martii Martii closed this as completed Nov 23, 2017
@OpenUserJS OpenUserJS locked as resolved and limited conversation to collaborators Apr 12, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
mightfix Ehh maybe... convince us all... UI Pertains inclusively to the User Interface.
Development

No branches or pull requests

3 participants