Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set process title when running cron job #34186

Merged

Conversation

fredden
Copy link
Member

@fredden fredden commented Sep 28, 2021

Description

When managing a busy server, it's helpful to understand what each process is doing. Currently there is no visibility for a systems administrator to see what task the bin/magento cron:run process is currently performing. The most information that's available is that some job (or set of jobs) is running within a PHP process that was started via the system task scheduler (read: cron daemon). Because of the way that Magento spawns additional processes for some groups (depending on their configuration) it is possible to sometimes narrow down the list of possible tasks because the group name is listed in the command arguments.

This pull request sets the title of the currently-running process to clearly list the group and job names.

Before screen-shot

Screenshot_2021-10-04_22-59-50

After screen-shot

Screenshot_2021-10-04_22-56-17

Related Pull Requests

None

Manual testing scenarios

  1. Run the standard Magento cron with bin/magento cron:run when several long-running tasks are scheduled
  2. Observe lack of (or presence of after this change) information about which job is running in the process tree (using tools like ps or htop on Unix - untested on Windows, but the functionality should work there too)

Questions or comments

Should the path to Magento be listed in the process title too? If there are multiple Magento instances on a single server, they should be running under different users. I can't think of a sensible non-development environment where multiple Magento instances run under the same system user.

Contribution checklist

  • Pull request has a meaningful description of its purpose
  • All commits are accompanied by meaningful commit messages
  • All new or changed code is covered with unit/integration tests (if applicable)
  • README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
  • All automated tests passed successfully (all builds are green)

Resolved issues:

  1. resolves [Issue] Set process title when running cron job #34321: Set process title when running cron job

@m2-assistant
Copy link

m2-assistant bot commented Sep 28, 2021

Hi @fredden. Thank you for your contribution
Here are some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

  • @magento give me test instance - deploy test instance based on PR changes
  • @magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

  • @magento run all tests - run or re-run all required tests against the PR changes
  • @magento run <test-build(s)> - run or re-run specific test build(s)
    For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

  1. Database Compare
  2. Functional Tests CE
  3. Functional Tests EE,
  4. Functional Tests B2B
  5. Integration Tests
  6. Magento Health Index
  7. Sample Data Tests CE
  8. Sample Data Tests EE
  9. Sample Data Tests B2B
  10. Static Tests
  11. Unit Tests
  12. WebAPI Tests
  13. Semantic Version Checker

You can find more information about the builds here

ℹ️ Please run only needed test builds instead of all when developing. Please run all test builds before sending your PR for review.

For more details, please, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@hostep
Copy link
Contributor

hostep commented Sep 29, 2021

@fredden: very nice, this sounds super useful!

Can you put a screenshot here, of how it looks with ps or top/htop ?

Should the path to Magento be listed in the process title too? If there are multiple Magento instances on a single server, they should be running under different users. I can't think of a sensible non-development environment where multiple Magento instances run under the same system user.

We do occasionally have this situation, it's usually for some demo shops or something like that, never for a real production shop, so there are still valid cases I believe.

@sidolov sidolov added the Priority: P2 A defect with this priority could have functionality issues which are not to expectations. label Sep 30, 2021
@andrewbess andrewbess self-assigned this Sep 30, 2021
@m2-community-project m2-community-project bot moved this from Pending Review to Review in Progress in High Priority Pull Requests Dashboard Sep 30, 2021
@fredden
Copy link
Member Author

fredden commented Sep 30, 2021

Thanks for your input @hostep. I've added before & after pictures to the main pull request description. I'm still somewhat undecided about what the process title should contain; I'm confident that including the process group & current job name will be helpful. I don't know if there's enough room for a full path too. Perhaps the path should go at the end?

@hostep
Copy link
Contributor

hostep commented Sep 30, 2021

Looks really useful, but it might introduce backwards incompatibilities, for example, the following code will probably stop working ...

If you could somehow try to append the extra info to the original output, it might be better?
For example:

/usr/local/bin/php /var/www/magento2/bin/magento cron:run --group=index --bootstrap=standaloneProcessStarted=1 (group: index, job: indexer_reindex_all_invalid)

But not sure how complicated this would be, not sure if you can fetch the original output using cli_get_process_title to be able to append the extra info?

@fredden
Copy link
Member Author

fredden commented Sep 30, 2021

Thanks for your feedback, Pieter. Based on this I've updated the pull request to preserve the original command line arguments and add the new information at the end. I've updated the screen-shots in the main description.

I'm open to including PHP_BINARY at the start of the new title; let me know if that'd be useful too.

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@hostep
Copy link
Contributor

hostep commented Oct 1, 2021

Thanks for the updates!

The PHP_BINARY is probably useful indeed, for example: if you have a server with multiple php versions installed it might be good that you see which version is being used.

I would try to stick as close as possible to the original output if possible.

@Den4ik
Copy link
Contributor

Den4ik commented Oct 1, 2021

@magento run all tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@Den4ik Den4ik self-assigned this Oct 1, 2021
@Den4ik Den4ik self-requested a review October 1, 2021 22:30
@engcom-Lima engcom-Lima added Progress: testing in progress Triage: Dev.Experience Issue related to Developer Experience and needs help with Triage to Confirm or Reject it labels Feb 15, 2023
@m2-community-project m2-community-project bot moved this from Ready for Testing to Testing in Progress in Community Dashboard Feb 15, 2023
@m2-community-project m2-community-project bot moved this from Ready for Testing to Testing in Progress in Community Dashboard Feb 15, 2023
@m2-community-project m2-community-project bot moved this from Testing in Progress to Ready for Testing in Community Dashboard Feb 15, 2023
@engcom-Echo
Copy link
Contributor

@magento run all tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@engcom-Echo engcom-Echo self-assigned this Feb 16, 2023
@engcom-Echo engcom-Echo moved this from Ready for Testing to Testing in Progress in Community Dashboard Feb 16, 2023
@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@engcom-Echo
Copy link
Contributor

@fredden
Thanks for the collaboration & contribution!

✔️ QA Passed

Set process title when running cron job

Manual Scenario Steps

  1. Run the standard Magento cron with bin/magento cron:run when several long-running tasks are scheduled

Before: ✖️
image

After: ✔️
Screenshot 2023-02-16 at 3 12 15 PM

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@engcom-Echo
Copy link
Contributor

engcom-Echo commented Feb 16, 2023

As Functional Tests CE is failing hence moving to Extended Testing to check the build failures.

@engcom-Echo engcom-Echo moved this from Testing in Progress to Extended testing (optional) in Community Dashboard Feb 16, 2023
@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@Den4ik Den4ik removed their request for review February 16, 2023 13:02
@engcom-Echo
Copy link
Contributor

Functional Tests CE failure are different on last two run.
Run1
image

Run2
image

Hence moving to Merge In Progress.

@engcom-Echo engcom-Echo moved this from Extended testing (optional) to Merge in Progress in Community Dashboard Feb 16, 2023
@magento-devops-reposync-svc magento-devops-reposync-svc merged commit 907114e into magento:2.4-develop Feb 23, 2023
@engcom-Charlie engcom-Charlie moved this from Merge in Progress to Recently Merged in Community Dashboard Feb 23, 2023
@fredden fredden deleted the cron-process-title branch February 23, 2023 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Cron Partner: Fisheye partners-contribution Pull Request is created by Magento Partner Priority: P2 A defect with this priority could have functionality issues which are not to expectations. Progress: accept Project: Community Picked PRs upvoted by the community Release Line: 2.4 Triage: Dev.Experience Issue related to Developer Experience and needs help with Triage to Confirm or Reject it
Projects
Community Dashboard
Recently Merged
Development

Successfully merging this pull request may close these issues.

[Issue] Set process title when running cron job