fix: index and entity link sync issues on parent block deletion by navinkarkera · Pull Request #37541 · openedx/openedx-platform

navinkarkera · 2025-10-24T05:54:37Z

Description

Meilisearch index documents were not synced properly when any block with children blocks like units, subsections, sections etc. were being deleted as the XBLOCK_DELETED is only triggered for the deleted block.
This PR fixes it by deleting all index documents that contain the deleted block in its breadcrumbs field as only blocks that are children of this block will have it its breadcrumbs field.

Similarly, the entity links that store links between course and library blocks was not synced properly due to children ContainerLinks not being deleted.

Useful information to include:

Which edX user roles will this change impact? "Learner", "Course Author", "Developer", and "Operator".

Supporting information

Upstream entity links are not deleted if parent block is deleted. #37427
Discussion in: Import analysis + start import button frontend-app-authoring#2525
Private-ref: https://tasks.opencraft.com/browse/FAL-4267

Testing instructions

Create a section, subsection, units with children in a course.
Check the Meilisearch dashboard: http://meilisearch.:7700/ and verify that the index documents are created for them.
Check the django admin: admin/contentstore/containerlink/ and verify that the container links are created.
Now delete the section without the changes in this PR, the index will still contain documents for subsection, units and its children, only the parent section is removed.
The container links for children containers are not deleted.
Checkout this PR.
Reset meilisearch index using tutor dev run cms ./manage.py cms reindex_studio --experimental --reset and tutor dev run cms ./manage.py cms reindex_studio --experimental
This is required due to addition of breadcrumbs.usage_key field to filterable fields list.
Repeat first 5 steps. All documents under the section will be deleted and all chidlren container links are deleted.

Deadline

"None" if there's no rush, or provide a specific date or event (and reason) if there is one.

Other information

Include anything else that will help reviewers and consumers understand the change.

Does this change depend on other changes elsewhere?
Any special concerns or limitations? For example: deprecations, migrations, security, or accessibility.
If your database migration can't be rolled back easily.

Use `breadcrumbs` field in meilisearch index to filter blocks containing this parent and delete them.

openedx-webhooks · 2025-10-24T05:54:43Z

Thanks for the pull request, @navinkarkera!

This repository is currently maintained by @openedx/wg-maintenance-edx-platform.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
- This process (including the steps you'll need to take) is documented here.
If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

Dependencies

This PR must be merged before / after / at the same time as ...
Blockers

This PR is waiting for OEP-1234 to be accepted.
Timeline information

This PR must be merged by XX date because ...
Partner information

This is for a course on edx.org.
Supporting documentation
Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

The size and impact of the changes that it introduces
The need for product review
Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

navinkarkera · 2025-10-24T06:18:25Z

    Fields.last_published,
    Fields.content + "." + Fields.problem_types,
    Fields.publish_status,
+    Fields.breadcrumbs + "." + Fields.usage_key,


Just want to note this: we had to include breadcrumbs in filterable field list, and I think change requires a reindex.

How can we handle this kind of change in a production environment? Should we create a "migration' to force the reindex?

If you run reindex_studio --init, which should run on any deployment, it will warn you that the index needs to be rebuilt, because of this code: https://github.com/openedx/edx-platform/blob/3dc96a97e99b90060c14f5b8d5b580fcef8287eb/openedx/core/djangoapps/content/search/api.py#L350-L397

But the whole thing is pretty messy right now, and we need to clean it up eventually: #36868

rpenido

LGTM 👍
Thank you for your work, @navinkarkera!

I tested this using the instructions from the PR
I read through the code
I checked for accessibility issues
Includes documentation

rpenido · 2025-10-24T14:41:45Z

    Fields.last_published,
    Fields.content + "." + Fields.problem_types,
    Fields.publish_status,
+    Fields.breadcrumbs + "." + Fields.usage_key,


How can we handle this kind of change in a production environment? Should we create a "migration' to force the reindex?

rpenido · 2025-10-24T14:47:51Z

@navinkarkera Could you update the testing instructions, including that we need to create/delete the containers on a Course (and not on a Library)?

ChrisChV

Looks good!

…edx#37541) Meilisearch index documents were not synced properly when any block with children blocks like units, subsections, sections etc. were being deleted as the `XBLOCK_DELETED` is only triggered for the deleted block. This PR fixes it by deleting all index documents that contain the deleted block in its `breadcrumbs` field as only blocks that are children of this block will have it its breadcrumbs field. Similarly, the entity links that store links between course and library blocks was not synced properly due to children `ContainerLinks` not being deleted.

fix: delete children index data on parent delete

e93cd9e

Use `breadcrumbs` field in meilisearch index to filter blocks containing this parent and delete them.

openedx-webhooks added open-source-contribution PR author is not from Axim or 2U core contributor PR author is a Core Contributor (who may or may not have write access to this repo). labels Oct 24, 2025

openedx-webhooks added this to Contributions Oct 24, 2025

github-project-automation Bot moved this to Needs Triage in Contributions Oct 24, 2025

navinkarkera requested review from bradenmacdonald and rpenido October 24, 2025 05:56

navinkarkera added 4 commits October 24, 2025 11:26

docs: update breadcrumbs description

168aa9d

feat: delete all blocks index documents when course is deleted

a12b5ba

fix: delete container links for all child containers on parent deletion

f618284

test: add tests

976f4b4

navinkarkera force-pushed the navin/fal-4267/analysis-api branch from 268bc61 to 976f4b4 Compare October 24, 2025 05:57

navinkarkera commented Oct 24, 2025

View reviewed changes

rpenido approved these changes Oct 24, 2025

View reviewed changes

navinkarkera marked this pull request as ready for review October 27, 2025 05:10

navinkarkera requested a review from ChrisChV October 27, 2025 05:10

mphilbrick211 added the FC Relates to an Axim Funded Contribution project label Oct 27, 2025

mphilbrick211 moved this from Needs Triage to Ready for Review in Contributions Oct 27, 2025

ChrisChV approved these changes Oct 27, 2025

View reviewed changes

ChrisChV added 2 commits October 27, 2025 12:23

Merge branch 'master' into navin/fal-4267/analysis-api

d26d2a0

Merge branch 'master' into navin/fal-4267/analysis-api

fa48d1f

ChrisChV merged commit 8aaae46 into openedx:master Oct 27, 2025
49 checks passed

ChrisChV deleted the navin/fal-4267/analysis-api branch October 27, 2025 18:44

github-project-automation Bot moved this from Ready for Review to Done in Contributions Oct 27, 2025

ormsbee mentioned this pull request Nov 7, 2025

fix: bump learning-core to 0.30.0 #37614

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: index and entity link sync issues on parent block deletion#37541

fix: index and entity link sync issues on parent block deletion#37541
ChrisChV merged 7 commits intoopenedx:masterfrom
open-craft:navin/fal-4267/analysis-api

navinkarkera commented Oct 24, 2025 •

edited

Loading

Uh oh!

openedx-webhooks commented Oct 24, 2025 •

edited

Loading

Uh oh!

navinkarkera Oct 24, 2025

Uh oh!

rpenido Oct 24, 2025

Uh oh!

bradenmacdonald Oct 24, 2025

Uh oh!

rpenido left a comment

Uh oh!

rpenido Oct 24, 2025

Uh oh!

rpenido commented Oct 24, 2025

Uh oh!

ChrisChV left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

navinkarkera commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Supporting information

Testing instructions

Deadline

Other information

Uh oh!

openedx-webhooks commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

navinkarkera Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

rpenido Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

bradenmacdonald Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

rpenido left a comment

Choose a reason for hiding this comment

Uh oh!

rpenido Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

rpenido commented Oct 24, 2025

Uh oh!

ChrisChV left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

navinkarkera commented Oct 24, 2025 •

edited

Loading

openedx-webhooks commented Oct 24, 2025 •

edited

Loading