Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

command line script to add teachers' csd sections to ai-rubrics experiment #57848

Merged
merged 6 commits into from Apr 9, 2024

Conversation

davidsbailey
Copy link
Member

@davidsbailey davidsbailey commented Apr 5, 2024

finishes https://codedotorg.atlassian.net/browse/AITT-555. I also added a similar tool to remove teachers from the experiment.

sample output:

Dave-MBP:~/src/cdo (add-csd-section-experiment)$ ./bin/oneoff/ai-rubrics-enable-teacher-sections me@code.org,me+teacher@code.org,bogus@code.org
no CSD sections found for teacher me@code.org
could not find user with email "bogus@code.org"
successfully enabled experiment for 1 teachers

Dave-MBP:~/src/cdo (add-csd-section-experiment)$ ./bin/oneoff/ai-rubrics-disable-teacher-sections me@code.org,me+teacher@code.org,bogus@code.org
no experiments found for teacher me@code.org
could not find user with email "bogus@code.org"
successfully disabled experiment for 1 teachers

Testing story

manually tested:

ai-rubrics-enable-teacher-sections:

  • works for all 4 eligible courses/units: csd, focus-on-coding, focus-on-creativity, interactive-games-animations
  • works with one or multiple teacher emails
  • bad/blank emails raise
  • num_enabled counted correctly when teacher has more than one section
  • after running the script, confirmed teacher and students have ai features enabled

ai-rubrics-disable-teacher-sections:

  • successfully removes experiments when running after each of the above scenarios

@davidsbailey davidsbailey marked this pull request as ready for review April 5, 2024 23:30
@davidsbailey davidsbailey changed the title Add csd section experiment command line script to add teachers csd sections to ai-rubrics experiment Apr 5, 2024
@davidsbailey davidsbailey changed the title command line script to add teachers csd sections to ai-rubrics experiment command line script to add teachers' csd sections to ai-rubrics experiment Apr 5, 2024
@davidsbailey davidsbailey requested review from wilkie and a team April 5, 2024 23:45
end

# this can take up to 60 seconds
require_relative '../../dashboard/config/environment'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is a trick. Add:

ENV['SKIP_I18N_INIT'] = '1'

Before this require. Cuts the time in half roughly in environments with localization enabled like production:

/code-dot-org $ time ruby script.rb
GetSecretValue: development/cdo/slack_bot_token

real    3m57.352s
user    3m37.076s
sys     0m17.138s
/code-dot-org $ vim script_with_skip.rb
/code-dot-org $ time ruby script_with_skip.rb
GetSecretValue: development/cdo/slack_bot_token

real    1m54.238s
user    1m40.048s
sys     0m13.254s

/code-dot-org $ cat script_with_skip.rb
ENV['SKIP_I18N_INIT'] = '1'
require_relative 'dashboard/config/environment'
``

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great idea! thanks for this!

updated and retested both scripts in the simple case.

Copy link
Contributor

@wilkie wilkie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good! Given those manual tests, I feel pretty confident that this works as intended. Not too bad to revert by running the other script with the same input more-or-less.

@davidsbailey davidsbailey merged commit 7d051fd into staging Apr 9, 2024
1 of 2 checks passed
@davidsbailey davidsbailey deleted the add-csd-section-experiment branch April 9, 2024 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants