mitx - only ingest published courses

[shanbady]

What are the relevant tickets?

Closes https://github.com/mitodl/hq/issues/4579

Description (What does it do?)

Makes it so that when we fetch from mitx we dont pull in courses that are unpublished

How can this be tested?

1. checkout this branch
2. make sure the changes in this branch are picked up by celery (restart the celery container)
3. run ./manage.py backpopulate_mitxonline_data
4. it should show that it pulled in ~84 resources as opposed to main which pulls in ~99.

[rhysyngsun]

This approach works for things going forward, but it doesn't address:

• Removing the currently non-live courses from MIT Open (arguably we could run a one-off script for this)
• In the future should a course be accidentally published and then unpublished

So it's probably a more reliable approach to instead add live and page.live to the computation of published in _transform_course.

[pdpinch]

@shanbady can you give me a high-level description of what the criteria are for including courses here?

cc @collinpreston

[shanbady]

@pdpinch at a high level we are including all courses that are published. On mitx that means we have to set "live=true" and "page_live=true" otherwise we would pull down courses that appear published but should not be.

[rhysyngsun]

LGTM