survey: adjust the survey prompt probability computation #2545
Comments
|
I propose just choosing a random day in the following 28 days. The code
would be fairly simple, and there are good statistical tests for the
uniform distribution.
The exact number 28 isn't critical. It's about a month, and it's big enough
so that if no one starts vscode on Sundays, the distribution is still
fairly flat.
…On Tue, Nov 22, 2022 at 2:53 PM Hyang-Ah Hana Kim ***@***.***> wrote:
Assigned #2545 <#2545> to
@pjweinb <https://github.com/pjweinb>.
—
Reply to this email directly, view it on GitHub
<#2545 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABJIAI5YNG62RW2WNMY74ITWJUQDDANCNFSM6AAAAAASIGHNU4>
.
You are receiving this because you were assigned.Message ID:
***@***.***>
|
|
This sounds great to me. Very elegant solution, thanks @pjweinb! |
gopherbot
pushed a commit
that referenced
this issue
Dec 12, 2022
Users are occasionally asked to take the gopls survey. The logic is a little complicated, but in the existing code, if the user is prompted, it will be for a random day in the rest of the current month. In the new code all days of the year are pretty nearly equally probable. The new code chooses a random day in the next 28 days. There's nothing special about 28, but it's large enough to smooth out day-of-the-week effects, and small enough that a chosen user will be prompted for a survey within a month. Fixes vscode-go/#2545 Change-Id: Ifaccaa5e8e4e8b1a1050768461ed38d4e144561e Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/456041 TryBot-Result: kokoro <noreply+kokoro@google.com> Run-TryBot: Peter Weinberger <pjw@google.com> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
|
Closed with go.dev/cl/457475. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
The current logic is biased and favors users who open editors for the first time near the end of the month. Our requirements are:
Prompting should be approximately uniformly distributed throughout the month and year. As much as possible, we should avoid cyclical behavior.
The sample of prompted users should approximate our 28d user base. That is to say that any 28d user is approximately equally likely to be prompted throughout the month. Obviously for users that only use VS Code once a month, this means they are more likely to be prompted in each individual session.
We should aim for 200-300 responses per month. Currently we're around 300.
cc @pjweinb @findleyr @fflewddur
The text was updated successfully, but these errors were encountered: