Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hints: add a default hint prompt attempts threshold #36507

Merged
merged 3 commits into from
Aug 31, 2020

Conversation

Erin007
Copy link
Contributor

@Erin007 Erin007 commented Aug 28, 2020

Continuation of #36488
STAR-1205

While we determine how/if we're going to invest in setting the hint prompt attempts threshold based on the logic for the number of attempts at which 75% of students completed the level +1, we're setting a default of 6.5 for the threshold. This means that levels in CSF courses that do not already have a hint prompt attempts threshold set, will display the hint prompt dialog after 7 runs.

The number selected for the default is basically arbitrary. I made it 6.5 so that it would be easily distinguishable from thresholds set programmatically. The distribution of thresholds for the levels shown below was calculated from the data in thresholds.csv which covers levels in Course 2, 3, 4, and A-F 2017.

Screen Shot 2020-08-28 at 3 31 20 PM

7 seemed like the middle-ish of the biggest left-hand half and I opted for a lower number than a higher one since showing a dismissible hint prompt when not needed felt like less of a problem than not showing one to a student who was struggling and would benefit from the prompt. Regardless, a default hint prompt attempts threshold no matter how to you figure it from the levels that currently have hint prompt attempts thresholds set has no real validity for the levels that don't have a hint prompt attempt threshold set because we don't know the relative difficulty of the latter levels nor do we know if they are intended to take more/less attempts to solve. This is a stop-gap solution that gets the hint prompt showing up more frequently than currently, but not optimally.

Reviewer Checklist:

  • Tests provide adequate coverage
  • Code is well-commented
  • New features are translatable or updates will not break translations
  • Relevant documentation has been added or updated
  • User impact is well-understood and desirable
  • Pull Request is labeled appropriately
  • Follow-up work items (including potential tech debt) are tracked and linked

@Erin007 Erin007 requested review from Hamms and a team August 28, 2020 19:50
Copy link
Contributor

@maddiedierker maddiedierker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some nit comments, but this looks great! thank you for the in-depth description around how you chose this threshold 😄

Erin007 and others added 2 commits August 31, 2020 11:19
Co-authored-by: Madelyn Kasula <maddie.kasula@code.org>
Co-authored-by: Madelyn Kasula <maddie.kasula@code.org>
@Erin007 Erin007 merged commit 67b54b0 into staging Aug 31, 2020
@Erin007 Erin007 deleted the baseline-hint-prompt-attempts-threshold branch August 31, 2020 16:56
Copy link
Contributor

@Hamms Hamms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made it 6.5 so that it would be easily distinguishable from thresholds set programmatically

I love this! But could we get some comments in the code explaining it? I can easily see how it would be hard for a future maintainer to latch on to the cleverness at work here without some guidance.

@Erin007
Copy link
Contributor Author

Erin007 commented Aug 31, 2020

I made it 6.5 so that it would be easily distinguishable from thresholds set programmatically

I love this! But could we get some comments in the code explaining it? I can easily see how it would be hard for a future maintainer to latch on to the cleverness at work here without some guidance.

Great idea - I'll add.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants