Add Elements::BEFORE_BULK_ELEMENT_SAVING_EVENT & Elements::AFTER_BULK_ELEMENT_SAVING_EVENT events

[khalwat]

The Problem

Currently, there's no great way to know when a bulk element saving operation is in progress. There is an Element::$resaving property, but that is only set by the ApplyNewPropagationMethod & ResaveElements queue jobs, and in Elements::resaveElements() and Elements::_saveElementInternal()

It doesn't handle other cases, such as:

1. A bulk import via FeedMe
2. A bulk update via a custom module or plugin from an external data source
3. The future import feature that Craft will be implementing at some point in the future to deprecate FeedMe

The Reason

There are a number of plugins such as Blitz and SEOmatic that need to perform lengthy operations after elements are updated, which are typically done by pushing a queue job.

In the case of Blitz, it will regenerate cached pages. In the case of SEOmatic, it will generate sitemaps. However during bulk operations, this can cause repeated and redundant queue jobs to be pushed into the queue unnecessarily.

Thus each plugin must implement some mechanism for attempting to detect when such a bulk operation is in progress. For example, this is how Blitz does it:

https://github.com/putyourlightson/craft-blitz/blob/38ac25588588c04145bae03e38f0621da168d6fe/src/Blitz.php#L395-L400

https://github.com/putyourlightson/craft-blitz/blob/develop/src/drivers/integrations/FeedMeIntegration.php#L29

Unfortunately, this means that each plugin must re-invent the wheel, and it also does not handle cases where a custom plugin or module is doing some kind of bulk element saving operation.

The Solution

A potential solution would be adding the events Elements::BEFORE_BULK_ELEMENT_SAVING_EVENT & Elements::AFTER_BULK_ELEMENT_SAVING_EVENT which would notify any interested listener that a bulk element saving operation has started, and has finished.

This would essentially just formalize the pattern that Blitz is using, and these events could then be triggered by ApplyNewPropagationMethod & ResaveElements queue jobs, and in Elements::resaveElements() and Elements::_saveElementInternal()... as well as in any custom module/plugin that needed to do some kind of bulk operation.

Then anyone listening for the events could know a bulk operation was starting, and postpone any work they would normally do until an event is fired notifying that the operation is done.

Potentially this could also be used to reduce the number of queue jobs that are needed for things like pruning revisions, updating search indexes, etc.

The Bonus Points

Even better would be a somewhat more elaborate system that would allow for the batching of elements being updated.

For example, on an active site with lots of content authors editing various things, a number of queue jobs are generated for updating search indexes, refreshing Blitz caches, pruning revisions, regenerating sitemap caches, and so on.

These are all things that need to be done, but would be more efficient if they were coalesced into batches so that a cascade of often redundant or overlapping queue jobs isn't necessary.

SEOmatic tries to solve this by not pushing too many queue jobs... we can't check the queue jobs table directly, because then it would only work if they are using the db-based queue.

Instead it caches a bit of data so it knows that a queue job for this particular sitemap has already been pushed into the queue, and hasn't completed yet: https://github.com/nystudio107/craft-seomatic/blob/develop-v4/src/models/SitemapTemplate.php#L153

[michaelrog]

Potentially this could also be used to reduce the number of queue jobs that are needed for things like pruning revisions, updating search indexes, etc.

I run into this ☝🏼 a lot.

One possible implementation would be for Craft to provide some start/end structure, sort of analogous to a transaction, for wrapping bulk operations:

Craft::$app->elements->startBulkOperation();
// Do stuff...
Craft::$app->elements->concludeBulkOperation();

or

Craft::$app->elements->asBulkOperation(function() {
  // Do stuff.
});

[brandonkelly]

Took a stab at this here: #14032

Let me know if this looks like it would work for you!

[khalwat]

Looks great... I left a comment on the PR