Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hints: turn on 'just-in-time' hint prompts for all csf courses #36488

Merged
merged 4 commits into from
Aug 28, 2020

Conversation

Erin007
Copy link
Contributor

@Erin007 Erin007 commented Aug 27, 2020

STAR-1205

In 2017 the CSF team introduced “Just-in-time” hint prompts that would nudge students on certain levels in Courses 2, 3 and 4 to look at hints.
Screen Shot 2020-08-27 at 4 26 46 PM

This hint prompt dialog pops up under certain conditions:

  • The user has not passed or perfected the puzzle,
  • And there are one or more hints available in the hint well,
  • And the user has not seen the hint prompt on this puzzle in the current session,
  • And the user has not used a hint on this puzzle yet,
  • And their total number of runs on this puzzle exceeds the run threshold.

Run threshold is the number of attempts at which 75% of users completed that level + 1. The run threshold is stored on levels as a property, hint_prompt_attempts_threshold. There was momentum after the initial implementation to expand just-in-time hint prompts to Courses A-F, and thresholds were calculated and uploaded via csv for those courses. However, that work fell short of turning the hint prompts "on" because the scripts were not designated as hint_prompt_enabled?. Now that many students are studying remotely, we're again interested in expanding just-in-time-hints since students don't have the benefit of in-person prompting from their teachers to use hints if needed.

In this PR, I've turned "on" just-in-time hint prompting for all CSF courses. However, not all levels in all CSF courses have hint_prompt_attempts_threshold set. In fact, most don't since versioning wasn't in full swing at the time of initial implementation. Essentially, this work will turn on hint prompting for the 2017 versions of Courses A-F that do have the thresholds set, as well and for any levels that were reused in more recent versions of the curriculum. In practice, that's only 19 levels in the 2020 version of Courses A-F.

We still need to decide if and how we'd like to set the hint_prompt_attempts_threshold for levels in recent versions of the CSF courses. There is ongoing discussion/brainstorming in this doc.

Testing story

I added some quick test coverage to ensure that hintPromptAttemptsThreshold is getting set properly - only for levels in CSF scripts and only if hint_prompt_attempts_threshold is set as a property on the level - in level_options.

Reviewer Checklist:

  • Tests provide adequate coverage
  • Code is well-commented
  • New features are translatable or updates will not break translations
  • Relevant documentation has been added or updated
  • User impact is well-understood and desirable
  • Pull Request is labeled appropriately
  • Follow-up work items (including potential tech debt) are tracked and linked

@Erin007 Erin007 requested review from Hamms and a team August 27, 2020 20:43
Copy link
Member

@breville breville left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unfamiliar with this system, but the change seems clean, tests good, and the write-up very helpful!

Copy link
Contributor

@Hamms Hamms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unfamiliar with this system, but the change seems clean, tests good, and the write-up very helpful!

I'm somewhat familiar with this system, and I concur!

@@ -348,7 +348,7 @@ def set_tts_options(level_options, app_options)
end

def set_hint_prompt_options(level_options)
if @script && @script.hint_prompt_enabled?
if @script && @script.csf? && @level.hint_prompt_attempts_threshold
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd like to keep hint_prompt_enabled? around for clarity; we can update it to just alias csf? for now, but I suspect we'll be wanting to adjust that again in the future

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants