feat: introduce send_on_commit and send_async helpers in send_event()

[bradenmacdonald]

Description

When sending events, two very common needs are to send events only when the transaction commits, and to send events asynchronously, to avoid slowdowns in the current user's request processing. Celery even has a delay_on_commit helper for combining these two common requirements.

However, this events library doesn't yet provide any way to do those things, so they're done very inconsistently at the moment.

Sending events asynchronously is a particular pain at the moment, because most event data structures use attrs objects which cannot be serialized to JSON, so you can't directly pass event data to a celery task. This PR solves that by using our existing event serialization mechanisms for celery as well as for event bus serialization.

Other information

Although an on_commit helper was mentioned in one of the ADRs, it was never implemented AFAIK:

openedx-events/docs/decisions/0015-outbox-pattern-and-production-modes.rst

Lines 44 to 54 in b1d710a

	openedx-events will change to support two modes for publishing events when an OpenEdxPublicSignal's ``send_event(...)`` is called:
	- ``on-commit``: Delay publishing to the event bus until after the current transaction commits, or immediately if there is no open transaction (as might occur in a worker process).
	- Atomicity is preserved in the success case, but not in the failure case. (Events published in this mode may occasionally be lost, but should never be sent when a transaction fails.)
	- This does not necessarily preserve ordering of events across multiple hosts.
	- ``outbox``: Prep the signal for publishing, and save in an outbox table for publishing as soon as possible. A worker process will then relay events from the outbox to the broker and mark them as successfully published. Another management command will be needed to periodically purge old processed events.
	- Atomicity is fully preserved.
	- As long as only a single worker per topic is emptying the outbox, ordering of events can be fully maintained.

This PR essentially implements both of the features mentioned in the ADR, although the new send_async parameter does not provide the strong ordering guarantees of the proposed "outbox" mode.

Things I didn't do

It would be awesome to have per-transaction event de-duplication; that is, if your application emits many identical events within a single database transaction using send_on_commit=True, when the transaction commits, only a single event should be emitted. This would make many common content patterns more efficient without requiring any additional work on the part of event emitters.

Ordering of async events - when you use send_async=True, the events may end up getting emitted out of order. Except in test cases, because in tests you generally have CELERY_ALWAYS_EAGER=True which sends them synchronously anyways. The ordering could always be made guaranteed later on.

Testing instructions

Here is a simple way to manually send all the different event combinations, using a shell like ./manage.py cms shell :

from openedx_events.content_authoring.signals import CONTENT_OBJECT_ASSOCIATIONS_CHANGED
from openedx_events.content_authoring.data import ContentObjectChangedData
CONTENT_OBJECT_ASSOCIATIONS_CHANGED.send_event(
    content_object=ContentObjectChangedData(object_id="foo", changes=["tags"])
)
CONTENT_OBJECT_ASSOCIATIONS_CHANGED.send_event(
    content_object=ContentObjectChangedData(object_id="foo", changes=["tags"]),
    send_on_commit=True,
)
CONTENT_OBJECT_ASSOCIATIONS_CHANGED.send_event(
    content_object=ContentObjectChangedData(object_id="foo", changes=["tags"]),
    send_async=True,
)
CONTENT_OBJECT_ASSOCIATIONS_CHANGED.send_event(
    content_object=ContentObjectChangedData(object_id="foo", changes=["tags"]),
    send_on_commit=True,
    send_async=True,
)

Note that the error message "Received invalid content object id" is expected to be returned/printed from the event receiver, since we're passing "foo" as an event parameter.

Deadline

No particular deadline, though it would be nice to use this functionality in some of the event refactors I'm working on for Verawood.

Checklists

Check off if complete or not applicable:

Merge Checklist:

• All reviewers approved
• Reviewer tested the code following the testing instructions
• CI build is green
• Version bumped
• Changelog record added with short description of the change and current date
• Documentation updated (not only docstrings)
• Integration with other services reviewed
• Fixup commits are squashed away
• Unit tests added/updated
• Noted any: Concerns, dependencies, migration issues, deadlines, tickets

Post Merge:

• Create a tag
• Create a release on GitHub
• Check new version is pushed to PyPI after tag-triggered build is
  finished.
• Delete working branch (if not needed anymore)
• Upgrade the package in the Open edX platform requirements (if applicable)

[openedx-webhooks]

Thanks for the pull request, @bradenmacdonald!

This repository is currently maintained by @openedx/committers-openedx-events.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

• If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
  • This process (including the steps you'll need to take) is documented here.
• If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

• Dependencies

  This PR must be merged before / after / at the same time as ...

• Blockers

  This PR is waiting for OEP-1234 to be accepted.

• Timeline information

  This PR must be merged by XX date because ...

• Partner information

  This is for a course on edx.org.

• Supporting documentation
• Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details

---

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

• Overview of Review Process for Community Contributions
• Pull Request Status Guide
• Making changes to your pull request

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

• The size and impact of the changes that it introduces
• The need for product review
• Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.