Skip to content
This repository has been archived by the owner. It is now read-only.

Replace source for "### people trying experiment" with count derived from metrics #1039

Closed
lmorchard opened this issue Jul 7, 2016 · 11 comments
Closed

Comments

@lmorchard
Copy link
Member

@lmorchard lmorchard commented Jul 7, 2016

Currently, we expose experiment usage via keeping a record of installations and counting them by distinct user. Once we get rid of the user DB (#1034), that a) removes the uniqueness index and b) means we have to rework that installations table to track client IDs.

It might be better if we also remove that installations table & API altogether, and instead find a way to get a per-experiment usage count from the Telemetry metrics sent from the client. This would also help us along the way to flat files (#988) by removing probably the last thing for which we need our own server & database.

@lmorchard
Copy link
Member Author

@lmorchard lmorchard commented Jul 7, 2016

FWIW, I have no idea how to do this yet, but I'm hoping there's some way we can get a report out of Telemetry just counting daily/monthly/overall active users.

My blue sky wish would be to have a JSON resource somewhere that just offers those counts - it could be a static file periodically updated from metrics data, or a live dynamic query that we heavily cache. Tomayto, tomahto.

@lmorchard
Copy link
Member Author

@lmorchard lmorchard commented Jul 7, 2016

Actually, I'll take it out of the milestone for now, because we can actually still get the counts from the installation table after getting rid of accounts. They don't necessarily have to be tied together and this issue will take some time to figure out

@johngruen
Copy link
Member

@johngruen johngruen commented Jul 29, 2016

@johngruen to talk to @clouserw about this stuff

@johngruen johngruen added this to the TXP-30 Remove Django milestone Aug 15, 2016
@dannycoates
Copy link
Member

@dannycoates dannycoates commented Aug 16, 2016

It seems like you could do this with a Spark job that writes json to an S3 bucket... This one that creates some data for Presto might have some clues.

@wresuolc
Copy link

@wresuolc wresuolc commented Aug 24, 2016

Sunah is working on this right now, but it won't be done in time for this sprint

@lmorchard
Copy link
Member Author

@lmorchard lmorchard commented Aug 30, 2016

FWIW, I filed #1312 with a proposed usage count JSON format that we can move to right now using the existing Django API. Hopefully whatever we come up with here can match that format, and switching over is just a matter of changing a configured URL

@wresuolc
Copy link

@wresuolc wresuolc commented Aug 30, 2016

/cc @sunahsuh

@lmorchard
Copy link
Member Author

@lmorchard lmorchard commented Sep 3, 2016

Following up here, it looks like we have data from Telemetry - and that's well more than half the battle!

https://analysis-output.telemetry.mozilla.org/testpilot/data/installation-counts/latest.json

Other parts to wrap up this issue:

@lmorchard
Copy link
Member Author

@lmorchard lmorchard commented Sep 9, 2016

Marking this as blocked to signal that we're still kind of waiting on final consensus about whether we're good to switch over to the Telemetry data. The two steps in my last comment - switching to that URL & removing the Django code - are the wrap up.

@lmorchard
Copy link
Member Author

@lmorchard lmorchard commented Sep 9, 2016

Actually, sounds _un_blocked and we should be clear to switch over to the Telemetry based numbers in a few weeks - probably next sprint end?

@lmorchard lmorchard removed the blocked label Sep 9, 2016
@wresuolc
Copy link

@wresuolc wresuolc commented Sep 12, 2016

👍 flip over any time

lmorchard added a commit to lmorchard/testpilot that referenced this issue Sep 16, 2016
Issue mozilla#1039
lmorchard added a commit to lmorchard/testpilot that referenced this issue Sep 16, 2016
Fixes mozilla#1039
lmorchard added a commit to lmorchard/testpilot that referenced this issue Sep 16, 2016
Fixes mozilla#1039
lmorchard added a commit to lmorchard/testpilot that referenced this issue Sep 19, 2016
Issue mozilla#1039
lmorchard added a commit to lmorchard/testpilot that referenced this issue Sep 20, 2016
Issue mozilla#1039
lmorchard added a commit to lmorchard/testpilot that referenced this issue Sep 20, 2016
Fixes mozilla#1039
lmorchard added a commit to lmorchard/testpilot that referenced this issue Sep 20, 2016
Fixes mozilla#1039
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.