Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse Cloudfront logs for add-on user stats #4

Closed
Sancus opened this issue Jul 8, 2018 · 17 comments

Comments

Projects
None yet
5 participants
@Sancus
Copy link
Member

commented Jul 8, 2018

In front of versioncheck.addons.thunderbird.net, Cloudfront sits. This means that most http requests won't reach the actual server, so we need to parse these logs to figure out user pings for add-on stats.

@ovari

This comment has been minimized.

Copy link

commented Jul 23, 2018

Are you able to please advise when the statistics will be once again available for add-ons? Thank you

@ovari

This comment has been minimized.

Copy link

commented Sep 27, 2018

Thunderbird/StatusMeetings/2018-09-18
sancus

  • Still working on addons-server update and stats fixes, should be done this week.

Are you able to please advise when the statistics will be once again available for add-ons? Thank you

@Sancus

This comment has been minimized.

Copy link
Member Author

commented Oct 9, 2018

I don't have an ETA for you but this hasn't been forgotten and I am actively working on it.

@RealRaven2000

This comment has been minimized.

Copy link

commented Oct 17, 2018

thanks this is badly needed. We (add-on devs) have to make a lot of hard decisions for the future when we commit to a maxver for the 60.3 release. A lot of it is "guess what's going away?".I have already been cut by Postbox deciding to drop 3rd party addons in 5.52b6

@RealRaven2000

This comment has been minimized.

Copy link

commented Nov 5, 2018

do you need any info from us to help closing this issue?

@CardBook

This comment has been minimized.

Copy link

commented Nov 9, 2018

any news ?

@Sancus

This comment has been minimized.

Copy link
Member Author

commented Dec 6, 2018

https://addons.thunderbird.net/en-US/thunderbird/addon/lightning/statistics/usage/?last=90

ADU stats should now be available, including data back to July 20th, the first full day that the site was up.

This does not include downloads because those rely on a different server, which currently has Cloudflare in front of it and so we don't have access to logs. I plan to change this eventually but it will be a couple of months probably.

There's a similar situation with personas/lightweight themes, which ping a completely different server that we currently aren't running. Updates on these seemed pretty low value, since they rarely get updated and can't really "break". I do have a plan to put these on the same servers as versioncheck(which is different from the way AMO runs them), but again not super high priority.

Complete Themes like TT DeepDark use the regular addon update server and so they do have ADUs tracked.

@ovari

This comment has been minimized.

Copy link

commented Dec 6, 2018

ADU stats should now be available, including data back to July 20th, the first full day that the site was up.

Thank you

@RealRaven2000

This comment has been minimized.

Copy link

commented Dec 6, 2018

Hurray! Thanks for this valuable tool. Is it possible to run three statistics on Thunderbird itself? It would be interesting to see how add-on growth and Application correlate.

@ovari

This comment has been minimized.

Copy link

commented Jan 14, 2019

The last available stats are for Jan 5, 2019. Are stats no longer working?

@mikehardy

This comment has been minimized.

Copy link

commented Jan 14, 2019

Only a guess on my part but I've noticed that they post at irregular times, and there was a big lag over the winter holidays so I'm guessing they are not fully automated since they are newly revived and there is some sort of staffing thing going on. Curious if I am right? If so I obviously want fresh stats but I also hope everyone's alright. Cheers.

@Sancus

This comment has been minimized.

Copy link
Member Author

commented Jan 15, 2019

The cron job seems to have gotten stuck for some reason, I unstuck it and will keep an eye on it.

@mikehardy

This comment has been minimized.

Copy link

commented Jan 19, 2019

Not urgent - but if it is indeed a cronjob, I expect it's daily? And it appears it is stuck again, but only just now. Maybe since it's recent the conditions causing it to have issues are still present.

@mikehardy

This comment has been minimized.

Copy link

commented Mar 16, 2019

@Sancus I understand these are not urgent, I'm only raising a comment here in case you weren't aware and you want to be - it appears the updates stalled again this week? Thanks for the work here :-)

1 similar comment
@ovari

This comment has been minimized.

Copy link

commented Mar 20, 2019

@Sancus I understand these are not urgent, I'm only raising a comment here in case you weren't aware and you want to be - it appears the updates stalled again this week? Thanks for the work here :-)

@mikehardy

This comment has been minimized.

Copy link

commented Mar 20, 2019

I looked and I think the 2-day lag is normal, it looks like it's caught up again since I posted a few days ago and has the normal data I'd expect. And my little plugin has a very slowly-but-steadily growing audience which is why I like to watch ;-)

@Sancus

This comment has been minimized.

Copy link
Member Author

commented Mar 29, 2019

I rewrote stats stuff so that it all runs on one machine with every part triggered by one script now. It was flaky before because there were several machines in different places involved. The stats script runs at 12pm PT now for the day before(so, 1 hour before this comment, it ran for March 28th) and stats should be updated by 12:15 in the worst case.

I'm going to close this, as it's gotten a bit ridiculously long and the actual bug was solved a long time ago. I opened #80 for download stats tracking just so it isn't completely forgotten, but I'm not planning to work on it any time soon.

@Sancus Sancus closed this Mar 29, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.