[DEPR]: Phase out edx-configuration

[feanil]

Proposal Date

30 November 2020

Ticket Acceptance Date

October 13, 2023

Technology Removal Date

After Redwood Cut

First Open edX Named Release Without This Functionality

Sumac

Rationale

N/A

Removal

N/A

Replacement

Tutor

Deprecation

No response

Migration

No response

Additional Info

For now a placeholder, this will eventually describe in detail how edx-configuration will stop being supported by edX, including:

• A date and/or Open edX release from which edx-configuration will no longer be supported
• A suggested alternative (possibly Tutor, via BTR-43)
• A suggested migration plan

Notes from Comments:

Kyle McCormick
February 11, 2022, 9:08 AM
Edited

I want to draw a distinction here:

• In the broader Open edX community, openedx/configuration is already rapidly being phased out.

  • As of Maple, the Ansible Native installation support was officially dropped in favor of Tutor. Some community members still use Ansible Native but I believe most intend to migrate to Tutor.

  • Devstack images are built from openedx/configuration, but we are replacing Devstack with Tutor, and will be probably ready to drop official Devstack support as of Nutmeg or Oak. (I need to make a DEPR ticket for this :)

  • As far as I know, the other miscellaneous playbooks in openedx/configuration are not generally used by the community, although we should check that assumption (eg, does OpenCraft depend on them for OCIM?)

• In my experience at edX/2U, though, they still use openedx/configuration heavily, especially in its older services.

  • They use it for:

    • build AMIs
    • deploying to prod/stage/sandbox
    • various automation needs
    • building devstack images

  • Their newer services use less/none of openedx/configuration.

  • There has been a push away from openedx/configuration, but migrating legacy services off of it will require non-trivial effort

Based on that:

• I see openedx/configuration as already de facto deprecated in the Open edX community. We should clean up this ticket and use it to formally propose the deprecation and see if anyone has objections to the direction we’re already rapidly heading.

• If that ^ DEPR goes though, edX/2U may still depend on the code for a while. So, the “Removal” step of this ticket could be transferring/forking the repostory back to the edx GitHub organization so that they can use it as long as they need to.

  • The work of removing edX/2U’s dependency on edx/configuration repo is probably best represented in their own tracking system instead of in a community DEPR ticket.

Jeremy Bowman
February 11, 2022, 8:33 AM

https://openedx.atlassian.net/wiki/spaces/AC/pages/2107441855/Braindump+on+Configuration+Today+and+Future is apparently the most detailed write-up we have so far regarding this. As noted in the original description, this ticket was less of “we plan to immediately prepare for this” and more “this is coming down the pipeline, if you have any objections or suggestions we’d like to hear them”. Progress on it so far has been gated on spare SRE bandwidth.

Original Jira Issue: https://openedx.atlassian.net/browse/DEPR-122

[dianakhuang]

Big usages of configuration:

• devstack
• 2U/edX deployments of older services (edx-platform, discovery, etc)
• 2U/edX sandboxes

Maybe we can start shaving some of these off the list.

[jmbowman]

I've created openedx-unsupported/devstack#943 as one step towards this and assigned it to the Arbi-BOM squad.

[dianakhuang]

At the very least, we have managed to find a solution for codejail on Tutor, which was one of the blockers of this work.

[robrap]

How will New Relic configuration be handled in the future, like this file: https://github.com/openedx/configuration/blob/master/playbooks/roles/edxapp/templates/newrelic.ini.j2?

I ask because I want to add some settings around logging/tracing, and at this point I'm just planning on changing them here and having it used everywhere. I'm hoping 2U is the only org affected, because no one has complained about any other changes.

[jmbowman]

@robrap There are already several services that don't use the configuration repo, but instead have their production Docker images generated by in-repo Dockerfiles. So if there are any New Relic settings that need to be set via config file that we want active across services, we'll have to make sure the config gets into those images (and probably adjust the cookiecutter to make sure new services' Dockerfiles get that by default).

[robrap]

Thanks @jmbowman. That's good to know. So far, I think we just have one server-side config related to browser. I imagine that only affects server-side templates, but I'm not certain.

[kdmccormick]

Sounds good to me!

[Agrendalath]

@dianakhuang, will this repository become private after this date? We are still using it for our deployments.

We also use it for building devstack images because:

1. There is a big time gap between the releases and the availability of tagged images in the DockerHub.
2. The images must be manually built to work natively on M1 Macs.

[dianakhuang]

@Agrendalath no, we are not planning on moving it private. We are merely trying to indicate that it's the 'old, not-recommended' way of doing things by taking it out of the openedx org and indicating that it is primarily designed to support edx.org. There might be more 2U-specific customization in that case, but it might be worth it to figure out what can and should be split into other scripts/resources that might be more generalizable.

[robrap]

@dianakhuang: This DEPR was moved to communicated without actually including all of the TBD data.

• I assume replacement is Tutor, but we could document and point to that.
• We need to document the first named release we propose will no longer include this.

If others are still using edx-configuration, we might not want to move edx-configuration unless we plan to drop it from the next named release? We could also document that it will be available in the edx org, but once we move it, it has a higher chance of picking up 2U specific changes (as parts of it move or are replaced), and it will likely be archived once we are done with it.

[robrap]

Here is a private edx.org link for jenkins-job-dsl-internal as a reminder of uses of configuration that will need to move. It is accessing requirements files and other scripts in the configuration repo.

[kdmccormick]

Copying a relevant comment: #247

So, I know I had originally been a proponent of deprecating both configuration and devstack "into the edx org", under the understanding that if only 2U is using some parts of the project, then it's easier for both sides if we transfer those parts over to 2U. Since then, I've talked about this with my team at Axim, and come to understand that the situation is more nuanced than that.

The code in the openedx org is built from a mix of Axim-owned contributions and community-contributor-owned contributions (licensed to Axim under the CLA). We hold the code in this org for the good of all current & former Open edX contributors and users, and as the project's stewards we have a mandate to maintain ownership and ensure availability of all its repositories, even unsupported repositories.

Our procedure of archiving repos by transferring to the openedx-unsupported org lets us mark a repo as "unsupported" while still guaranteeing the repo's availability and keeping its ownership clear. So, if/when devstack and configuration are deprecated, we'll need to move them there.

TL;DR: If configuration is deprecated, it would eventually be moved to a public archive in openedx-unsupported.

[kdmccormick]

We'd like to know how folks are still using this. That would make it easier for us to decide whether to keep it maintained in the openedx organization vs. deprecate it and let individual providers fork & continue to use it independently.

@dianakhuang or @robrap , could you note here how you still use this repository? @feanil and I will reach out to other community providers with the same question.

[robrap]

@kdmccormick: We're slowly gathering data in this ticket: edx/edx-arch-experiments#540. We can update the DEPR at some point. Thanks.

[dianakhuang]

2U is still using this repository for:

• deploys of some public repos like edxapp/edx-platform, xqueue, xqwatcher, insights
• deploys of prospectus
• some tools like flower and coursegraph
• Jenkins jobs that do things like sync and package logs, and probably also retire users
• Sandboxes

[kdmccormick]

@dianakhuang I understand that 2U no longer wants to maintain this repository for community use. Just like with Devstack, could you re-communicate this ticket, and announce a 2 week period for someone else to step up as maintainer? If we get a maintainer, then we'll keep this in the openedx org; if not, we'll archive it to openedx-unsupported. Either way, 2U is welcome to create a fork of this repo in their edx organization and make edx.org-specific changes there.

[dianakhuang]

@kdmccormick I think we would like a longer time on this before we are ready to archive it/fork it, but it was requested that we put a deprecation notice up and remove it from the Open edX releases (pre-Redwood) earlier rather than later. I am happy to remove notices/put things back into the release if that's what AXIM would like.

[kdmccormick]

@dianakhuang Gotcha, sorry for the mixed messages, let me talk to my team and make sure we're on the same page :)

[kdmccormick]

@dianakhuang Thanks for bearing with us. The process we'd like to follow, both for this and devstack, are:

• 2U announces intent to stop maintaining on $ACCEPTANCE_DATE; other maintainers are welcome to step up; if they don't, repo will be deprecated.
• does someone step up?
  • yes -> they become maintainer; repos stays in the openedx org; 2U forks it as needed.
  • no -> repository is archived to openedx-unsupported; 2U forks it as needed.

For $ACCEPTANCE_DATE, 2 weeks from now is fine since we've been tossing this around for so long, but you could push it out further to the future if you aren't ready to fork it yet.

[feanil]

We did this here: https://discuss.openedx.org/t/deprecation-edx-configuration/10702 and there was no resonse so we will proceed with archiving this repository.

The current plan for this is to archive this after redwood is cut to give users one last release with configuration.

[feanil]

I've created openedx-unsupported/configuration#7137 to further communicate the deprecation before the removal.

[feanil]

This is blocked until after the Redwood Release. Once Redwood is released, this can be archived.

[feanil]

We ended up doing this early, if we need it tagged for Redwood, we can go back and add redwood branches and tags.