Skip to content
This repository

Stats on usage? #405

Closed
paulirish opened this Issue August 30, 2012 · 19 comments

10 participants

Paul Irish Lachlan Collins Ryan Kirkman Thomas Davis Jon Cram Máté FARKAS Han Lin Yap Blaise Kal Niyaz Greg Franko
Paul Irish

I'm curious if you can offer any sort of data on the relative traffic of all the libraries.

Knowing which are used more and less often would be very helpful, I think, to a lot of developers.

I'll take some raw data if that's easier for you.

Ryan Kirkman
Owner

That's something we've been working on for a while now.

The data isn't ready just yet, but in the very near future that should change.

Paul Irish
Thomas Davis

We have stats, just figuring out an efficient log parsing strategy. After that's complete I will most likely serve the data on a Restful call /statistics?start=DATE&end=DATE

which will return [
{ library: 'backbone', hits: 10000},
{ library: 'jquery', hits: 24}....
]

I will expose the call with CORS for all host and throw up a simple UI to peruse the stats by date.

Thomas Davis

Hey guys, heres an update, looking for feedback.

I have stats running at http://stats.cdnjs.com by default it shows todays stats which get parsed every 6 hours.

  • CORS enabled to *

You can access historic data by using the to and from query string parameters.

The query parameters take any valid Date.js(http://www.datejs.com/) value.

to and from both default to current date if not specified

So for example
Todays - http://stats.cdnjs.com
Yesterdays - http://stats.cdnjs.com/?to=yesterday&from=yesterday
Total of last week - http://stats.cdnjs.com/?from=last%20week

etc etc

date.js takes a decent amount of options.

Just ironing out bugs and taking feedback at the moment

Note: The database is only storing, from 25th-ish onwards.

Jon Cram

Looks great so far.

I'm a little curious why each library's stats are presented as an object wrapped in an array like this:

"jquery": [
    {
        "date": "29/9/2012",
        "hits": "771236"
    }
]

instead of without an array, like this:

"jquery": {
    "date": "29/9/2012",
    "hits": "771236"
}

The array seems unnecessary, just wondering why, not that it really matters.

Thomas Davis

@webignition if you select a date range that is larger than 1 day, the dates will appear in the array. Does that help?

Jon Cram

Yes, that helps. It makes much more sense now. Thanks!

Máté FARKAS

It would be better to use the ISO 8601 date format, especially for foreigns, and it is parseable in every environment.

Thomas Davis

put stats on the homepage http://cdnjs.com

also only library specific stats will ever be shown, need to draw up a terms of service that states we will never disclose information about particular websites

Han Lin Yap

Suggestion: Show number of unique websites that are using.


Will there be statistic by version?


Make the value on "hits" to int in json string.

Blaise Kal

http://stats.cdnjs.com is timing out a lot.

Niyaz

Looks like we need an uptime stats page for the stats page :)

Han Lin Yap

How is it going with the stats? http://stats.cdnjs.com seems down.

Greg Franko

I would love to see stats for the cdn.js hosted libraries. Any updates on this?

Thomas Davis
Owner

Hey guys,

we do have access to the logs in common log format (http://en.wikipedia.org/wiki/Common_Log_Format)

But we were struggling with putting together a cost efficient solution to parse them.

So for example http://www.s3stat.com/Pricing.aspx does $300 for ~5million hits a day whereas we could potentially be getting around ~15-30 million per day.

So my thoughts is to get an EC2 instance that parses them continuously and post the aggregate values for each day to a mysql database. (library, version, hits, date)

Greg Franko

How were you parsing the data with the http://stats.cdnjs.com API?

Thomas Davis
Owner

Same as my solution in the previous comment. But it was buggy so wanted to rewrite the parser.

Greg Franko

Gotcha, that makes sense. Let me know if you want any help with the UI.

Lachlan Collins Lockyc closed this March 22, 2013
Ryan Kirkman
Owner

Discussion moved to #1078

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.