Skip to content
This repository has been archived by the owner. It is now read-only.

Track traffic to GitHub repo longer than 14 days #399

Open
cirosantilli opened this issue May 19, 2015 · 119 comments
Open

Track traffic to GitHub repo longer than 14 days #399

cirosantilli opened this issue May 19, 2015 · 119 comments

Comments

@cirosantilli
Copy link
Collaborator

@cirosantilli cirosantilli commented May 19, 2015

Possible? If not, +1 from me.

Sample reference URL: https://github.com/cirosantilli/test/graphs/traffic

On SE: http://webapps.stackexchange.com/questions/60915

@cirosantilli
Copy link
Collaborator Author

@cirosantilli cirosantilli commented May 19, 2015

Ivan replied it is not currently possible.

@Lorenzo45

This comment was marked as spam.

5 similar comments
@greckyogurt
Copy link

@greckyogurt greckyogurt commented Nov 5, 2015

+1

@Danappelxx
Copy link

@Danappelxx Danappelxx commented Nov 7, 2015

👍

@alexbudmsft
Copy link

@alexbudmsft alexbudmsft commented Nov 19, 2015

+1

@saqib-razzaq-aspose
Copy link

@saqib-razzaq-aspose saqib-razzaq-aspose commented Dec 30, 2015

+1

@xXFracXx
Copy link

@xXFracXx xXFracXx commented Dec 31, 2015

+1

@brianmc

This comment was marked as spam.

@nikhil-pandey

This comment was marked as spam.

@sanyamc-msft
Copy link

@sanyamc-msft sanyamc-msft commented Feb 20, 2016

+1 there are many people in my team who dearly want this data. Currently we manually parse the data into a csv and power our charts on a fortnightly basis. Would really appreciate api/ json data, even if its for 14 days.

@saqib-razzaq-aspose

This comment has been hidden.

@xianyi

This comment was marked as spam.

3 similar comments
@lolbot-iichan
Copy link

@lolbot-iichan lolbot-iichan commented Feb 27, 2016

+1

@nakulpathak3
Copy link

@nakulpathak3 nakulpathak3 commented Apr 15, 2016

+1

@wvangeit
Copy link

@wvangeit wvangeit commented Apr 28, 2016

+1

@rahulkrvnit

This comment was marked as spam.

@zonky2

This comment was marked as spam.

7 similar comments
@rljordan-zz
Copy link

@rljordan-zz rljordan-zz commented Jun 21, 2016

+1

@ruhbehka
Copy link

@ruhbehka ruhbehka commented Jun 24, 2016

+1

@KindKitty
Copy link

@KindKitty KindKitty commented Oct 10, 2016

+1

@JarekToro
Copy link

@JarekToro JarekToro commented Oct 27, 2016

+1

@ManoMarks
Copy link

@ManoMarks ManoMarks commented Oct 31, 2016

+1

@johngronberg-okta
Copy link

@johngronberg-okta johngronberg-okta commented Nov 1, 2016

+1

@ZahediAquino
Copy link

@ZahediAquino ZahediAquino commented Nov 23, 2016

+1

@KrisSiegel
Copy link

@KrisSiegel KrisSiegel commented Nov 27, 2016

This would be incredibly helpful especially for those maintaining open source libraries.

@burnash
Copy link

@burnash burnash commented Oct 24, 2018

@timgrossmann

I'd really like to try your tool, but allowing full read and write access to all my public repositories only to view my traffic history seems a bit too much, doesn't it?
Why do you need write access?

Good question. In short: there's no way known to me to get the traffic data without the public_repo scope in an OAuth app. GitHub asks for read/write permission to access traffic data:

For repositories that you have push access to, the traffic API provides access to the information provided in the graphs section.

Full answer: https://github.com/repotraffic/repotraffic-feedback/issues/6#issuecomment-432638356


For those new to the thread: Repotraffic is an app that stores and plots traffic data for longer than 14 days.

@timgrossmann
Copy link

@timgrossmann timgrossmann commented Oct 24, 2018

@burnash Thank you for the answer and clarifying this.

That's pretty odd of github to ask for this.
I'm not really into allowing any platform to access my all my projects with owner right permissions...
Would've loved to use your tool.

Maybe at one point there will be a solution.

Thanks

@geniusdibya
Copy link

@geniusdibya geniusdibya commented Oct 29, 2018

But doesn't have the Clone data. Can you add that too?

Maybe. It'd be a nice feature to have. And it's has been already requested (repotraffic/repotraffic-feedback#3 and repotraffic/repotraffic-feedback#7)
I've been tracking my own open source project gspread for two years, but haven't really needed to track the number of clones.

@burnash Actually, I want to track what is the real usage of my repo. And, clones/Fork data are got indicators.

Also is the code available on GIT to contribute?

Not yet. But I'm considering open sourcing it (repotraffic/repotraffic-feedback#6) if I have enough time.

@burnash Sure. Would love to contribute.

@abbyachau

This comment was marked as spam.

@manojkumarvohra

This comment was marked as spam.

@glenn-jocher
Copy link

@glenn-jocher glenn-jocher commented Jul 1, 2019

I made a little web app that stores and plots traffic data for longer than 14 days. I'll be glad if someone finds it useful.
@burnash This is cool. But doesn't have the Clone data. Can you add that too?
Also is the code available on GIT to contribute?

This works great!

@timqian
Copy link

@timqian timqian commented Oct 14, 2019

Hi all, I made another open-source web app to record repo traffic, clones, referrers and more. Welcome to try it out

Repo Analytics

@timgrossmann
Copy link

@timgrossmann timgrossmann commented Oct 14, 2019

@timqian Same problem as with the other tooling... It's a shame that GitHub doesn't change that...

Screenshot 2019-10-14 at 09 25 59

@timqian
Copy link

@timqian timqian commented Oct 14, 2019

@timgrossmann Yeah, unfortunately, this is required 😂 Only thing I can do is to take care of your token. Also when you don't need this service, you can delete the token from your setting anytime

@UbhiTS
Copy link

@UbhiTS UbhiTS commented Jan 9, 2020

it really is a shame that github has to make users create solutions just to know the extended history and analytics of their repositories :( .. I would prefer not to give anyone write access to my profile and repositories just to know extended analytics. Github, hope you're listening :)

@BradleyA
Copy link

@BradleyA BradleyA commented Feb 24, 2020

Prefer a solution that does NOT give someone else write access to your repositories
A simple shell script that uses crontab to schedule the download of GitHub traffic statistics in plain text from your GitHub repositories. Another script gives you the text lines to add to crontab and sets up the directories for GitHub traffic statistics. It takes minutes to setup but weeks to get enough data. At this point you can process the data with any tools you wish.

I created a third script that parses the GitHub traffic and creates two markdown tables (clone, views) like this clone table.

2019 GitHub clones

Date: 07-29 08-05 08-12 08-19 08-26 09-02 09-09 12-09 12-16 12-23 12-30
Clones: 34 31 2 2 4 2 2 1 2 9 4
Unique clones: 2 3 2 2 2 2 2 1 2 8 2

Total clones: 93

It solves my challenge. It may help you.

You can find the scripts: https://github.com/BradleyA/Linux-admin/tree/master/github-repository-traffic#github-repository-traffic

@MBurtsev
Copy link

@MBurtsev MBurtsev commented Mar 26, 2020

Great and useful idea, I am sure that it should be implemented.

@UbhiTS
Copy link

@UbhiTS UbhiTS commented Apr 12, 2020

this is seriously frustrating, in this age of cheap storage, GitHub would not even have a totals counter if not the entire history graph. At least give us Total Hits, Hits in last 1 year and hits in last 1 month. I'm sure having running counters would not add much to you load. For the love of god GitHub, please listen !

@bzhu5

This comment was marked as spam.

@sangonzal
Copy link

@sangonzal sangonzal commented Sep 25, 2020

For anyone still looking for a solution for this, I made a GitHub action that you might find useful: https://github.com/marketplace/actions/repository-traffic

With a GitHub action, the access token that is needed for accessing the GitHub API stays in your control.

@kevinmgates
Copy link

@kevinmgates kevinmgates commented Oct 27, 2020

For anyone still looking for a solution for this, I made a GitHub action that you might find useful: https://github.com/marketplace/actions/repository-traffic

With a GitHub action, the access token that is needed for accessing the GitHub API stays in your control.

AWESOME!

@matthansen0
Copy link

@matthansen0 matthansen0 commented Jan 9, 2021

I wrote an automated solution for this also that uses Azure tech - https://github.com/matthansen0/logicapp-githubstats

@UbhiTS
Copy link

@UbhiTS UbhiTS commented Jan 9, 2021

After my frustrations, I also wrote a solution for myself (and others) who do not wish to share any info with anyone else. No access token, no nothing. I basically trick github and found an easy way to maintain a running counter.

https://medium.com/@ubhits/how-i-beat-githubs-14-day-only-traffic-insights-using-aws-lambda-sqs-dynamodb-and-the-api-98f9216dbe4c

@InbarGazit

This comment was marked as spam.

@antonkomarev
Copy link

@antonkomarev antonkomarev commented Mar 25, 2021

I've made a pet project Ÿ HŸPE which helps me to collect and display GitHub traffic data longer than 14 days and even more. Just login and pick what repositories you want to track.

Ÿ HŸPE repo traffic

@InbarGazit
Copy link

@InbarGazit InbarGazit commented Mar 25, 2021

does it work retroactively? or only from that point forward?

@antonkomarev
Copy link

@antonkomarev antonkomarev commented Mar 25, 2021

It can only start collecting data after installation -14 days. GitHub doesn't provide data from the past periods... 😭

@glenn-jocher
Copy link

@glenn-jocher glenn-jocher commented Mar 25, 2021

@antonkomarev @timqian do you know any way (paid or otherwise) that allows for more comprehensive github analytics over extended timespans (more similar to google analytics)?

One major limitation I see with accumulating the short 14 day numbers is that it's impossible to determine the number of unique visitors over an extended time period. For example if a repo has 1000 unique visitors over 2 weeks, this does not mean that there will be 2000 unique visitors over 4 weeks, or 4000 unique visitors over 8 weeks, etc, as many visitors over the following time period will be repeat visitors. Does my question make sense?

@antonkomarev
Copy link

@antonkomarev antonkomarev commented Mar 25, 2021

@glenn-jocher in https://yhype.me we summarize daily counters of unique visitors and if there was 3 unique visitors in January and 2 in March we tend to think that it was 5 different users.

GitHub says that it is unique visitors, but there is no 100% proof that if someone will be counted as unique visitor in January will not be treated as unique visitor in March one more time.

@jgehrcke
Copy link

@jgehrcke jgehrcke commented Apr 15, 2021

I did have a rather close look at all existing solutions to 'work around this', for custom aggregation. I've built https://github.com/marketplace/actions/github-repo-stats / https://github.com/jgehrcke/github-repo-stats and you might want to have a look! Every day, it generates a nice HTML and PDF report and from my point of view the most important part is that the data stays in GitHub. You don't need a cloud service to integrate this with. No S3 or so.

The time series data is simply stored in a "data repository" (where you run this action in), with transparent evolution of history.

The HTML report can be exposed via GitHub Pages. Example (demo): https://jgehrcke.github.io/ghrs-test/jgehrcke/covid-19-germany-gae/latest-report/report.html

Would appreciate feedback! Thanks!

And: storing this actually in GitHub is a funny way to reply to @UbhiTS:

this is seriously frustrating, in this age of cheap storage, GitHub would not even have a totals counter if not the entire history graph. ... For the love of god GitHub, please listen !

@ElliotB256
Copy link

@ElliotB256 ElliotB256 commented Aug 4, 2021

To add something to this discussion, GitHub does store the access data for repositories for longer than the 14 day period. I recently filed a DMCA takedown on an infringing repository, and they were able to provide page views and clones going back a few months for the repo on request. I'm not sure why the data is not made easily available.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet