Skip to content

Conversation

@mbertrand
Copy link
Member

@mbertrand mbertrand commented Sep 20, 2024

What are the relevant tickets?

Closes https://github.com/mitodl/hq/issues/5127
Closes https://github.com/mitodl/hq/issues/5128

Description (What does it do?)

Adds the format and pace fields and populates them via ETL pipelines

How can this be tested?

  • Copy all the .env values needed for ETL pipelines (see settings_course_etl.py) from RC, except contentfile-related settings (buckets for contentfiles).

  • Run docker compose up or docker compose run --rm web ./manage.py migrate. The new migration should complete successfully.

  • Run all the following pipelines, they should complete successfully:

    ./manage.py backpopulate_mitx_edx_data
    ./manage.py backpopulate_micromasters_data
    ./manage.py backpopulate_mitxonline_data
    ./backpopulate_ocw_data --course-name 21w-758-genre-fiction-workshop-fantasy-fall-2016 --skip-contentfiles
    ./manage.py backpopulate_oll_data
    ./manage.py backpopulate_prolearn_data
    ./manage.py backpopulate_sloan_data
    ./manage.py backpopulate_xpro_data
    
  • Check that the program "Supply Chain Management" has a pace value of ["self_paced", "instructor_paced"]

  • Verify that the calculate of pace and format values follows the guidelines set per source in this spreadsheet. - - Run this code in a shell, you should get the same output assuming you have resources from each source:

    from learning_resources.etl.constants import ETLSource
    from learning_resources.models import *
    lrs = LearningResource.objects.filter(published=True)
    for source in ETLSource.names():
      print(
          source, 
          list(lrs.filter(etl_source=source).values_list("pace",flat=True).distinct()), 
          list(lrs.filter(etl_source=source).values_list("format", flat=True).distinct())
      )
    micromasters [['self_paced']] [['asynchronous']]
    mit_edx [['self_paced'], ['instructor_paced']] [['asynchronous']]
    mitxonline [['instructor_paced'], ['self_paced']] [['asynchronous']]
    oll [['self_paced']] [['asynchronous']]
    xpro [['self_paced']] [['asynchronous']]
    ocw [['self_paced']] [['asynchronous']]
    prolearn [['instructor_paced']] [['asynchronous']]
    podcast [['self_paced']] [['asynchronous']]
    see [['instructor_paced'], ['self_paced']] [['asynchronous'], ['asynchronous', 'synchronous'], ['synchronous']]
    youtube [['self_paced']] [['asynchronous']]

Additional Context

Pace and format were modeled as lists at the resource level because it's possible that a program might have multiple values depending on the courses that make up the program (some might be a mix of sync and async, instructor-paced and self-paced).
Also, at one point there was a Sloan course with runs that were a mix of instructor-paced and self-paced, sync and async (though that is no longer the case).

@mbertrand mbertrand force-pushed the mb/pace_format branch 2 times, most recently from 441c0d0 to 2067454 Compare September 20, 2024 18:17
@mbertrand mbertrand added Needs Review An open Pull Request that is ready for review and removed Work in Progress labels Sep 23, 2024
@shanbady shanbady self-requested a review September 23, 2024 15:00
@shanbady shanbady self-assigned this Sep 23, 2024
Copy link
Contributor

@shanbady shanbady left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@shanbady shanbady added Waiting on author and removed Needs Review An open Pull Request that is ready for review labels Sep 23, 2024
@mbertrand mbertrand merged commit 4bde7f1 into main Sep 24, 2024
@odlbot odlbot mentioned this pull request Sep 24, 2024
2 tasks
jonkafton added a commit that referenced this pull request Sep 25, 2024
* Codespace opensearch service fix (#1582)

* testing codespace fix

* testing fix

* Release 0.19.2

* Fix extract_openedx_data and backpopulate_mit_edx_data commands to work with course/program datafiles (#1587)

* follow/unfollow popover (#1589)

* adding success variant

* adding working version

* adding working version

* fixing text and adding margin to buttons

* lint fixes

* lint fix

* fixing test cases

* closing popup before posting data

* removing redundancy

* removing redundancy

* removing empty test

* updating styles

* updating styles

* changes to match design

* changes to match design

* Make search mode defaults settable env variables (#1590)

* Release date for 0.19.2

* Release 0.19.3

* Release date for 0.19.3

* Pace and format fields for learning resources (#1588)

* Migrate search settings #5527

---------

Co-authored-by: Shankar Ambady <shanbady@gmail.com>
Co-authored-by: Doof <mitx-devops@mit.edu>
Co-authored-by: Matt Bertrand <mrbertrand@gmail.com>
Co-authored-by: Anastasia Beglova <abeglova@mit.edu>
@mbertrand mbertrand deleted the mb/pace_format branch October 23, 2024 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants