Skip to content

refactor: replaces custom MongoDBProxy implementation with PyMongo's built-in retry functionality#37011

Merged
kdmccormick merged 2 commits intoopenedx:masterfrom
WGU-Open-edX:issue-35210-remove-mongodb-proxy
Jul 21, 2025
Merged

refactor: replaces custom MongoDBProxy implementation with PyMongo's built-in retry functionality#37011
kdmccormick merged 2 commits intoopenedx:masterfrom
WGU-Open-edX:issue-35210-remove-mongodb-proxy

Conversation

@wgu-ram-chandra
Copy link
Copy Markdown
Contributor

@wgu-ram-chandra wgu-ram-chandra commented Jul 12, 2025

Description

This PR removes the usage of the custom mongodb_proxy package from edx-platform, replacing it with MongoDB's built-in retry functionality via retryReads=True.

Changes made:

  • Removed all references to mongodb_proxy.MongoProxy from connection logic.
  • Updated connect_to_mongodb() to always use retryReads=True.
  • Uninstalled openedx-mongodbproxy from edx-platform requirements.
  • Manually smoke-tested LMS and Studio, both function normally without mongodb_proxy.

Built-in vs previous behavior:

  • The current @autoretry_read logic retried AutoReconnect up to 5 times with delay.
  • retryReads=True provides a single automatic retry for transient errors like AutoReconnect, NotPrimaryError, etc
In Current `@autoretry_read` Implementation Built-in `retryReads`(documentation)
Screenshot 2025-07-12 at 3 52 59 PM Screenshot 2025-07-12 at 3 43 10 PM

Proposed Solution

Is a single automatic retry (retryReads=True) sufficient for our use case?

If not, we can add a lightweight wrapper at the call site where additional retries are needed. For example:

def retry_on_autoreconnect(fn, tries=5, wait=0.1):
    for attempt in range(tries):
        try:
            return fn()
        except AutoReconnect:
            if attempt == tries - 1:
                raise
            time.sleep(wait)

# Sample usage
return retry_on_autoreconnect(lambda: collection.find(...))

References

Fixes: #35210

CC: @kdmccormick

@openedx-webhooks openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Jul 12, 2025
@openedx-webhooks
Copy link
Copy Markdown

Thanks for the pull request, @wgu-ram-chandra!

This repository is currently maintained by @openedx/wg-maintenance-edx-platform.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

  • If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
    • This process (including the steps you'll need to take) is documented here.
  • If it doesn't, simply proceed with the next step.
🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

  • Dependencies

    This PR must be merged before / after / at the same time as ...

  • Blockers

    This PR is waiting for OEP-1234 to be accepted.

  • Timeline information

    This PR must be merged by XX date because ...

  • Partner information

    This is for a course on edx.org.

  • Supporting documentation
  • Relevant Open edX discussion forum threads
🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details
Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

  • The size and impact of the changes that it introduces
  • The need for product review
  • Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

@github-project-automation github-project-automation Bot moved this to Needs Triage in Contributions Jul 12, 2025
@wgu-ram-chandra wgu-ram-chandra force-pushed the issue-35210-remove-mongodb-proxy branch 2 times, most recently from 581733c to f829a5a Compare July 13, 2025 00:12
@wgu-ram-chandra wgu-ram-chandra force-pushed the issue-35210-remove-mongodb-proxy branch 2 times, most recently from f1d87fe to c0a97dd Compare July 14, 2025 15:40
Comment thread xmodule/mongo_utils.py Outdated
@wgu-ram-chandra wgu-ram-chandra force-pushed the issue-35210-remove-mongodb-proxy branch 2 times, most recently from e8cc3ff to 405a448 Compare July 14, 2025 17:15
Copy link
Copy Markdown
Contributor

@dwong2708 dwong2708 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest testing it in a sandbox environment with some sample courses and components and the second commit looks like duplicated

@wgu-ram-chandra wgu-ram-chandra added create-sandbox open-craft-grove should create a sandbox environment from this PR mao-onboarding Reviewing this will help onboard devs from an Axim mission-aligned organization (MAO). labels Jul 14, 2025
@wgu-ram-chandra wgu-ram-chandra force-pushed the issue-35210-remove-mongodb-proxy branch 3 times, most recently from 99f0375 to d4fadbc Compare July 14, 2025 18:28
@wgu-ram-chandra wgu-ram-chandra force-pushed the issue-35210-remove-mongodb-proxy branch from d4fadbc to f1b5c2c Compare July 14, 2025 18:36
@open-craft-grove
Copy link
Copy Markdown

Sandbox deployment failed 💥
Please check the settings and requirements.
Retry deployment by pushing a new commit or updating the requirements/settings in the pull request's description.
📜 Failure Logs
ℹ️ Grove Config, Tutor Config, Tutor Requirements

@open-craft-grove
Copy link
Copy Markdown

Sandbox deployment failed 💥
Please check the settings and requirements.
Retry deployment by pushing a new commit or updating the requirements/settings in the pull request's description.
📜 Failure Logs
ℹ️ Grove Config, Tutor Config, Tutor Requirements

@open-craft-grove
Copy link
Copy Markdown

Sandbox deployment failed 💥
Please check the settings and requirements.
Retry deployment by pushing a new commit or updating the requirements/settings in the pull request's description.
📜 Failure Logs
ℹ️ Grove Config, Tutor Config, Tutor Requirements

@open-craft-grove
Copy link
Copy Markdown

Sandbox deployment successful 🚀
🎓 LMS
📝 Studio
ℹ️ Grove Config, Tutor Config, Tutor Requirements

@wgu-ram-chandra
Copy link
Copy Markdown
Contributor Author

@dwong2708 I was able to successfully test course creation in the sandbox, thank you!

@open-craft-grove
Copy link
Copy Markdown

Sandbox deployment failed 💥
Please check the settings and requirements.
Retry deployment by pushing a new commit or updating the requirements/settings in the pull request's description.
📜 Failure Logs
ℹ️ Grove Config, Tutor Config, Tutor Requirements

@mphilbrick211 mphilbrick211 moved this from Needs Triage to Waiting on Author in Contributions Jul 15, 2025
@wgu-ram-chandra wgu-ram-chandra mentioned this pull request Jul 17, 2025
4 tasks
@wgu-ram-chandra wgu-ram-chandra changed the title Issue 35210 remove mongodb proxy refactor: replaces custom MongoDBProxy implementation with PyMongo's built-in retry functionality Jul 18, 2025
Copy link
Copy Markdown
Member

@kdmccormick kdmccormick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks Ram! I'll merge later today.

@wgu-ram-chandra
Copy link
Copy Markdown
Contributor Author

Thanks Kyle!

@kdmccormick kdmccormick merged commit c9912b2 into openedx:master Jul 21, 2025
49 checks passed
@github-project-automation github-project-automation Bot moved this from Waiting on Author to Done in Contributions Jul 21, 2025
@edx-pipeline-bot
Copy link
Copy Markdown
Contributor

2U Release Notice: This PR has been deployed to the edX staging environment in preparation for a release to production.

@edx-pipeline-bot
Copy link
Copy Markdown
Contributor

2U Release Notice: This PR has been deployed to the edX production environment.

1 similar comment
@edx-pipeline-bot
Copy link
Copy Markdown
Contributor

2U Release Notice: This PR has been deployed to the edX production environment.

sumair-arbisoft pushed a commit to sumair-arbisoft/edx-platform that referenced this pull request Aug 5, 2025
…penedx#37011)

This PR removes the usage of the custom `mongodb_proxy` package from
edx-platform, replacing it with MongoDB's built-in retry functionality via
`retryReads=True`.

Changes:

* Removed all references to `mongodb_proxy.MongoProxy` from connection logic.
* Updated `connect_to_mongodb()` to always use `retryReads=True`.
* Uninstalled `openedx-mongodbproxy` from edx-platform requirements.

Fixes: openedx#35210
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

create-sandbox open-craft-grove should create a sandbox environment from this PR mao-onboarding Reviewing this will help onboard devs from an Axim mission-aligned organization (MAO). open-source-contribution PR author is not from Axim or 2U

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Remove MongoDBProxy

8 participants