Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rhsm_register resource creates high load on Red Hat Satellite #9295

Open
donwlewis opened this issue Jan 29, 2020 · 2 comments
Open

rhsm_register resource creates high load on Red Hat Satellite #9295

donwlewis opened this issue Jan 29, 2020 · 2 comments

Comments

@donwlewis
Copy link

@donwlewis donwlewis commented Jan 29, 2020

Description

the rhsm_register resource seems to create quite a bit of load on a Red Hat Satellite server when a large number of machines are registered to it. It seems that when running a subscription-manager status this generates several tasks on the satellite that can cause load to spike when a large number of hosts are doing it all at once. We reached out to Red Hat to help investigate the problem, and they seemed to point to this as being the culprit.

Chef Version

15.4.45

Platform Version

RHEL7

Please let me know if you need and data or testing from me. I realize that satellite is a pretty niche thing, so I will be more than happy to help or contribute to get this resolved.

@donwlewis

This comment has been minimized.

Copy link
Author

@donwlewis donwlewis commented Jan 29, 2020

Here is some more information:

  • When running subscription-manager status on a node, this will reach out to the satellite server and run a Katello::Host::Update on the satellite. Process does several things on the backend.

  • when running subscription-manager repos --list-enabled, this also reaches out to the satellite and runs Katello::Host::Update on the satellite.

As you can see, if you have several nodes running these process, especially multiple times if you are enabling multiple repos, this could cause the number of tasks to build up on the satellite, increasing the load. I am trying to see if Red Hat can provide any type of cache data that can be reviewed on the node vs having to reach out to the satellite all of the time. Seems silly to me that has to do this.

@donwlewis

This comment has been minimized.

Copy link
Author

@donwlewis donwlewis commented Jan 29, 2020

Here is more information. I saw the following displayed when running the rhsm_subscribe resource when it actually executed:

      [execute] Internal Server Error): Required lock is already taken by other running tasks.
                Please inspect their state, fix their errors and resume them.

                Required lock: update
                Conflicts with tasks:
                  https://REDACTED/foreman_tasks/tasks/304c096b f4e6 407c 84a5 a0af02ee7b5b

                Successfully attached a subscription for: Enterprise Linux Extra Packages

The way I am doing the subscription attachment is I am looping through an array of subscription IDs. Maybe not ideal, but not sure how else it should be done since you can't pass an array to the resource itself.

As far as checking subscription, we could possible use subscription-manager list to get the status of the subscription, as this shows the consumed base Red Hat subscription and seems to pull the data from scratch.

The best option for reviewing the repos that are enabled at this point seems to be to read through the /etc/yum.repos.d/redhat.repo file that gets generated when a system is subscribed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.