Improve cron error logging

[fredden]

Description (*)

This pull request improves logging of errors within cron processes.

Current scenario
Currently any error messages are lost / not recorded. For the main group (and any groups not set to use a separate process), STDERR is left for the scheduler to capture. This is most easily caught with the MAILTO= directive on unix, however experience suggests this is rarely configured usefully. Any cron group currently configured to run in a separate process will have its STDOUT and STDERR lost forever.

Proposed scenario
With the changes here, both STDERR and STDOUT are caught and logged. This is very valuable in scenarios where one cannot configure the scheduler, or when a job running in a separate process is suffering from errors. A good example of this is Magento Cloud hosting where cron processes often fail without any useful diagnostic information available. These changes will mean that STDERR output is readily available, aiding in diagnostics.

Related Pull Requests

Fixed Issues (if relevant)

None

Manual testing scenarios (*)

1. Artificially introduce a memory leak into a cron job, eg indexer_reindex_all_invalid
2. Run php bin/magento cron:run without these changes
3. Observe lack of information about the failure
4. Run php bin/magento cron:run with these changes
5. Observe information available in var/log (in this example, var/log/magento.cron.index.log)

Questions or comments

I can see that using posix_isatty() is discouraged, but no alternative is provided / suggested. There are functions in packages installed via composer as development dependencies which provide the required functionality. (eg here and here) What's the recommended approach in this case? Do we simply remove the output line altogether?

Contribution checklist (*)

• Pull request has a meaningful description of its purpose
• All commits are accompanied by meaningful commit messages
• All new or changed code is covered with unit/integration tests (if applicable)
• All automated tests passed successfully (all builds are green)

Resolved issues:

1. resolves [Issue] Improve cron error logging #37453: Improve cron error logging

[m2-assistant]

Hi @fredden. Thank you for your contribution
Here are some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

• @magento give me test instance - deploy test instance based on PR changes
• @magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

• @magento run all tests - run or re-run all required tests against the PR changes
• @magento run <test-build(s)> - run or re-run specific test build(s)
  For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

1. Database Compare
2. Functional Tests CE
3. Functional Tests EE,
4. Functional Tests B2B
5. Integration Tests
6. Magento Health Index
7. Sample Data Tests CE
8. Sample Data Tests EE
9. Sample Data Tests B2B
10. Static Tests
11. Unit Tests
12. WebAPI Tests
13. Semantic Version Checker

You can find more information about the builds here

ℹ️ Please run only needed test builds instead of all when developing. Please run all test builds before sending your PR for review.

For more details, please, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[evktalo]

I can confirm this issue has caused a problem with monitoring for one of our clients.

[fredden]

I can confirm this issue has caused a problem with monitoring for one of our clients.

@evktalo I'm having a little trouble interpreting your statement. Are you saying that (1) you have applied the changes in this pull request to a website and these have caused a problem somehow? Or, are you saying that (2) you have witnessed the issue that this pull request aims to solve?

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[engcom-Charlie]

@magento run all tests

[engcom-Charlie]

As this PR has been moved from Extended Testing to On hold, now moving back it to Extended Testing. It seems that bot moved it to Testing.

[engcom-Charlie]

@magento run all tests

[engcom-Charlie]

As mentioned here, SVC PR is also got merged.

[engcom-Charlie]

@magento run Functional Tests B2B, Functional Tests CE, Functional Tests EE, Semantic Version Checker, WebAPI Tests

[engcom-Charlie]

@magento run Functional Tests B2B, Functional Tests EE, Semantic Version Checker

[engcom-Charlie]

@magento run all tests

[engcom-Charlie]

@magento run Functional Tests B2B, Functional Tests CE, WebAPI Tests

[engcom-Charlie]

The functional B2B and CE failures are different in recent 2 builds, seems to be flaky, not related to PR.

Functional B2b Run 1:
https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/32690/8702467c9bd89c3d9926ead0a52ad902/Functional/allure-report-b2b/index.html#categories

Run 2:
https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/32690/b10797c20e9e0467f44307e30e270719/Functional/allure-report-b2b/index.html#categories/a644910ae0cb11e839a98baa21929b76/50ebaf5c92c8429c/

Functional CE Run 1:
https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/32690/08671a99533b826f6fc78334f7ffbf3e/Functional/allure-report-ce/index.html#categories

Run 2:
https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/32690/1b99e7cb8377f982e24ca2c0af78da92/Functional/allure-report-ce/index.html#categories

[engcom-Charlie]

The Web API tests seems to be a known failure. Not related to this PR.

https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/32690/062366865d06947dca33b5b66c08c41a/WebApi/console-error-logs.html

[engcom-Charlie]

Once semver version will get release and all the related updation will be done we will take this PR further, till then moving this PR to On Hold.

[engcom-Charlie]

@magento run all tests

[engcom-Charlie]

As mentioned here, the semver version will get release in upcoming month, post that we can take this PR further.

[engcom-Charlie]

@magento run all tests