Skip to content

Conversation

mbertrand
Copy link
Member

What are the relevant tickets?

Closes #441

Description (What does it do?)

  • Allows for blank terms/years in OCW courses, and adjusts the readable_id value accordingly
  • If exceptions occur during a batch of ocw url paths processed by the learning_resources.etl.pipelines.ocw_courses_etl function, raise an ExtractException at the end indicating which ocw url's failed.

How can this be tested?

  • Use the same AWS credentials and OCW_* settings as RC in your .env file.
  • Run ./manage.py backpopulate_ocw_data --course-name=res-hs-002-chemistry-behind-the-magic-chemical-demonstrations-for-the-classroom
  • Check that the learning resource imported, and the readable id is RES.HS-002
  • Temporarily modify this line in learning_resources.etl.ocw.transform_run function to be the same as the main branch:
     "year": course_data.get("year")
  • Restart containers and run the mgmt command again (with --overwrite). You should get an error.

…se an error at end of ocw_courses_etl function if any exceptions occurred during processing
@mbertrand mbertrand added the Needs Review An open Pull Request that is ready for review label Feb 6, 2024
@codecov
Copy link

codecov bot commented Feb 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (79891da) 77.01% compared to head (07fdd7b) 76.97%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #475      +/-   ##
==========================================
- Coverage   77.01%   76.97%   -0.04%     
==========================================
  Files         245      245              
  Lines        9998    10007       +9     
  Branches     1707     1708       +1     
==========================================
+ Hits         7700     7703       +3     
- Misses       2130     2135       +5     
- Partials      168      169       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ChristopherChudzicki ChristopherChudzicki self-assigned this Feb 6, 2024
@ChristopherChudzicki
Copy link
Contributor

👍

@mbertrand mbertrand merged commit c05678d into main Feb 7, 2024
@odlbot odlbot mentioned this pull request Feb 7, 2024
26 tasks
@mbertrand mbertrand deleted the mb/ocw_optional_semester_year branch May 13, 2024 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs Review An open Pull Request that is ready for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OCW Courses with no year/semester fail to import

2 participants