Drop public stats

[saivann]

It turns out we're receiving massive suspicious traffic that is polluting stats to the point where they are not so useful anymore. This traffic comes from a very large number of IP addresses, making it distributed enough so it becomes hard to filter suspicious traffic without engaging in a cat and mouse game.

Although this traffic seems all directed at root-level pages (e.g. /, /en/, /nl/), probably making other individual page request counts the only value that is still accurate.

I suggest we wait until the end of May to see if this issue resolves by itself, and if it doesn't, we could drop public stats and delete logs. Perhaps eventually we could consider a javascript & cookie tracking solution like Piwik (although this would require a privacy policy).

Ping @jgarzik (as that feature was asked by him in the past)

Stats were at least useful for:

• Prioritizing work on certain pages and translations.
• Facilitating sponsorship requests.

[harding]

ACK. I'll miss the way the stats let me brag, but I don't think losing them will create any problems.

[harding]

Someone reported that this might be the cause of the suspicious traffic: https://blog.sucuri.net/2015/05/hacked-websites-redirect-to-bitcoin-org.html

[wbnns]

RE: Piwik - I spoke with them a long time ago in the past and one of the main collaborators was seemed happy to hear it was a possibility for Bitcoin.org and wanted to be of assistance in case it was needed. The main issue was not being able to track downloads of the core software "out of the box" if I recall correctly.

[harding]

I don't think Piwik or even Google Analytics could help with the current problem of receiving redirected traffic from hacked websites.

[wbnns]

@harding - agreed. do you think we should have stats?

[saivann]

As far as I'm concerned, I think stats should provide enough value to cover efforts spent on them. With this recent issue, I think this is probably not the case anymore.

[harding]

@Coderwill I agree with @saivann. Stats are a tool in our toolbox. If a tool breaks and it costs too much to replace it, we have to learn to live without it.

[gmaxwell]

ACK. (just commenting to note that I've been following along, and I support removing them; if they're being filled with junk they're not worth putting effort or thought into maintaining)

[wbnns]

@saivann @harding @gmaxwell - another thing to take into consideration with the metrics shooting up is that recently, Bitcoin.com was redirected to http://Bitcoin.org/. So perhaps we can't conclude that all of it is from hacked websites. Having worked on Bitcoin.com for a few months during the past year, it was averaging between 4000-5000K unique sessions a day, which is now funneling directly into Bitcoin.org. This also might explain why there is such a large amount of different IPs involved.

[saivann]

@Coderwill No, a good chunk of this traffic presents suspicious referer headers, from repeating domains, with different url for each request, and this list of domains is changing over time.

It may be that some of this traffic does not originate from the issue outlined on the sucuri blog. If so, that simply means we have another important source of suspicious traffic, which only makes the issue worse. I don't know if it's possible to manipulate the referer header directly in the browser using javascript, but some reading I've made suggests there is at least a way to do it in some browsers.

Edit: You can also just look at the stats and see that, whatever date bitcoin.com started redirecting to bitcoin.org, it hasn't made any significant difference.

[wbnns]

@saivann ok, roger that, then. that's a bummer... thanks for explaining in more detail.

[saivann]

Unless the issue magically resolves by itself, public stats will be dropped on June 1th.