Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --exclude-group to cron:run #34117

Conversation

convenient
Copy link
Contributor

@convenient convenient commented Sep 17, 2021

Description

https://devdocs.magento.com/guides/v2.4/config-guide/cli/config-cli-subcommands-cron.html#config-cli-cron-group-run

When you decide to split a single cron group with the --group flag it means you need to do so for every group in your instance.

It's pretty common for third party modules to define their own groups whenever they like, so we have to keep up to date with the defined groups in the instance to keep adding these to our crontab when new groups are added.

One of my instances currently has 11 groups to manage, more will come in the future and its just something I need to mentally keep an eye on, a bit annoying.

In my case I only really want to run the crons like

* * * * * bin/magento cron:run --exclude-group index
* * * * * flock -n index.lock bin/magento cron:run --group index

This will give me better granular control over the indexer crons which can be useful when your store reaches hundreds of thousands of SKUs. Any other random cron groups would then still be able to run by default like in normal operation.

For future proofing I've made the --exclude-group flag stackable so you can define it multiple times like

* * * * * bin/magento cron:run --exclude-group index --exclude-group consumers

Manual testing scenarios (*)

  1. Run cron with no params, all groups run
  2. Run cron with --group, only that group runs
  3. Run cron with --group and that group also in --exclude-group, no groups run
  4. Run cron with --exclude-group, that group is excluded
  5. Run cron with multiple --exclude-group, all defined groups are excluded

Contribution checklist (*)

  • Pull request has a meaningful description of its purpose
  • All commits are accompanied by meaningful commit messages
  • All new or changed code is covered with unit/integration tests (if applicable)
  • README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
  • All automated tests passed successfully (all builds are green)

Resolved issues:

  1. resolves [Issue] Add --exclude-group to cron:run #36628: Add --exclude-group to cron:run

https://devdocs.magento.com/guides/v2.4/config-guide/cli/config-cli-subcommands-cron.html#config-cli-cron-group-run

When you decide to split a single cron group with the `--group` flag it means you need to do so for every group in your instance.

It's pretty common for third party modules to define their own groups whenever they like, so we have to keep up to date with the defined groups in the instance to keep adding these to our crontab when required.

One of my instances currently has 11 groups to manage, more will come in the future.

In my case I only really want to run the crons like

```
* * * * * bin/magento cron:run --exclude-group index
* * * * * flock -n index.lock bin/magento cron:run --group index
```

This will give me better granular control over the indexer crons which can be useful when your store reaches hundreds of thousands of SKUs

For future proofing I've made the `--exclude-group` flag stackable so you can define it multiple times like
```
* * * * * bin/magento cron:run --exclude-group index --exclude-group consumers
```
@convenient
Copy link
Contributor Author

@magento run all tests

@m2-assistant
Copy link

m2-assistant bot commented Sep 17, 2021

Hi @convenient. Thank you for your contribution
Here are some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

  • @magento give me test instance - deploy test instance based on PR changes
  • @magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

  • @magento run all tests - run or re-run all required tests against the PR changes
  • @magento run <test-build(s)> - run or re-run specific test build(s)
    For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

  1. Database Compare
  2. Functional Tests CE
  3. Functional Tests EE,
  4. Functional Tests B2B
  5. Integration Tests
  6. Magento Health Index
  7. Sample Data Tests CE
  8. Sample Data Tests EE
  9. Sample Data Tests B2B
  10. Static Tests
  11. Unit Tests
  12. WebAPI Tests
  13. Semantic Version Checker

You can find more information about the builds here

ℹ️ Please run only needed test builds instead of all when developing. Please run all test builds before sending your PR for review.

For more details, please, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@convenient
Copy link
Contributor Author

@magento run all tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

Copy link

@BarnyShergold BarnyShergold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why the sniffer is complaining about the CronCommand.php file as those code elements have been there for sometime.

However the sniffer errors in the test file need to be fixed. If you search the code base for 'array_filter' you will see how the use of 'function' needs to be formatted.

Once the errors are fixed we can give it a proper review :)

@convenient
Copy link
Contributor Author

@BarnyShergold Aye i spotted that as well. I think maybe CronCommand.php hasn't been touched in a while and the static analysis might only run on files changed?

Do you want me to address all static defects in CronCommand.php?

I am on a few days annual leave now, will fix my dodgy array_filters later in the week

@BarnyShergold
Copy link

@BarnyShergold Aye i spotted that as well. I think maybe CronCommand.php hasn't been touched in a while and the static analysis might only run on files changed?

Do you want me to address all static defects in CronCommand.php?

I am on a few days annual leave now, will fix my dodgy array_filters later in the week

if you can, that would be great - save any aggro :)

@convenient
Copy link
Contributor Author

@magento run all tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@convenient
Copy link
Contributor Author

@BarnyShergold Two failures in this test run for the static analysis

PHP Code Sniffer detected 1 violation(s): 

FILE: /var/www/html/app/code/Magento/Cron/Console/Command/CronCommand.php
-------------------------------------------------------------------------
FOUND 2 ERRORS AFFECTING 2 LINES
-------------------------------------------------------------------------
  67 | ERROR | If the @inheritdoc not inline it shouldn’t have braces
 102 | ERROR | {@inheritdoc} does not import parameter annotation
-------------------------------------------------------------------------

As per magento/magento-coding-standard#154 I have no idea what to do with this issue. I've checked a bunch of different core console commands and they all look the same to me. Looking at https://devdocs.magento.com/guides/v2.3/coding-standards/docblock-standard-general.html#inheritdoc it says

The @inheritdoc tag SHOULD NOT be used.

So 🤷 not sure what I can do here. Please advise.

This one seems like a false alert to me, I didn't touch anything remotely near this area so not sure why its popping up.

PHP Copy/Paste Detector has found error(s):
<?xml version="1.0" encoding="UTF-8"?>
<pmd-cpd>
  <duplication lines="28" tokens="92">
    <file path="/var/www/html/app/code/Magento/Quote/Model/Quote/QuantityCollector.php" line="20"/>
    <file path="/var/www/html/app/code/Magento/Quote/Model/Quote/TotalsCollector.php" line="222"/>
    <codefragment>    public function collectItemsQtys(Quote $quote)
    {
        $quoteItems = $quote-&gt;getAllVisibleItems();
        $quote-&gt;setItemsCount(0);
        $quote-&gt;setItemsQty(0);
        $quote-&gt;setVirtualItemsQty(0);

        foreach ($quoteItems as $item) {
            if ($item-&gt;getParentItem()) {
                continue;
            }

            $children = $item-&gt;getChildren();
            if ($children &amp;&amp; $item-&gt;isShipSeparately()) {
                foreach ($children as $child) {
                    if ($child-&gt;getProduct()-&gt;getIsVirtual()) {
                        $quote-&gt;setVirtualItemsQty($quote-&gt;getVirtualItemsQty() + $child-&gt;getQty() * $item-&gt;getQty());
                    }
                }
            }

            if ($item-&gt;getProduct()-&gt;getIsVirtual()) {
                $quote-&gt;setVirtualItemsQty($quote-&gt;getVirtualItemsQty() + $item-&gt;getQty());
            }
            $quote-&gt;setItemsCount($quote-&gt;getItemsCount() + 1);
            $quote-&gt;setItemsQty((float)$quote-&gt;getItemsQty() + $item-&gt;getQty());
        }

</codefragment>
  </duplication>
</pmd-cpd>

Failed asserting that false is true.

Exclude the staging group etc, not relevant for our test
Exclude the staging group etc, not relevant for our test
@convenient
Copy link
Contributor Author

@magento run all tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@convenient
Copy link
Contributor Author

@magento run Functional Tests CE

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@BarnyShergold
Copy link

@magento run Static Tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@convenient
Copy link
Contributor Author

@BarnyShergold Yeah those two same failures again. what to do with the inheritDoc and that completely random unrelated failure 🤔

@engcom-Lima
Copy link
Contributor

engcom-Lima commented Dec 16, 2022

✔️ QA Passed

Preconditions:

  • Install fresh Magento 2.4-develop

Manual testing scenario:

  1. Run cron with no params, all groups run
  2. Run cron with --group, only that group runs
  3. Run cron with --group and that group also in --exclude-group, no groups run
  4. Run cron with --exclude-group, that group is excluded
  5. Run cron with multiple --exclude-group, all defined groups are excluded

Before: ✖️ Not able to run cron using "--exclude-group"
Screenshot 2022-12-15 at 3 51 43 PM

Screenshot 2022-12-15 at 3 50 38 PM

After: ✔️ Now its happening.

image

==========================================================================================

image

==========================================================================================

Additionally tested:
After running the cron executed below scenarios:

  1. Created customer.
  2. Placed order from the customer.
  3. Placed order from the guest.
  4. In admin side -Created new product.
  5. Created invoice, creditmemo, reordered.

No additional manual test case is required as a part of regression.

Since, this branch is having conflicts. Hence, moving this PR to extended testing.

@engcom-Lima engcom-Lima moved this from Testing in Progress to Extended testing (optional) in Community Dashboard Dec 16, 2022
@engcom-Echo
Copy link
Contributor

@magento run all tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

1 similar comment
@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@engcom-Echo
Copy link
Contributor

@magento run Functional Tests CE

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

1 similar comment
@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@engcom-Echo engcom-Echo self-assigned this Dec 19, 2022
@m2-community-project m2-community-project bot moved this from Extended testing (optional) to Ready for Testing in Community Dashboard Dec 19, 2022
@engcom-Echo engcom-Echo moved this from Ready for Testing to Extended testing (optional) in Community Dashboard Dec 19, 2022
@engcom-Echo
Copy link
Contributor

@magento run all tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

2 similar comments
@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@engcom-Echo engcom-Echo moved this from Extended testing (optional) to Merge in Progress in Community Dashboard Dec 20, 2022
@magento-devops-reposync-svc magento-devops-reposync-svc merged commit ef91c61 into magento:2.4-develop Dec 22, 2022
@sidolov sidolov moved this from Merge in Progress to Recently Merged in Community Dashboard Dec 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Auto-Tests: Covered All changes in Pull Request is covered by auto-tests Component: Cron Partner: Ampersand partners-contribution Pull Request is created by Magento Partner Priority: P3 May be fixed according to the position in the backlog. Progress: accept Project: Community Picked PRs upvoted by the community Release Line: 2.4
Projects
Community Dashboard
Recently Merged
Development

Successfully merging this pull request may close these issues.

[Issue] Add --exclude-group to cron:run