Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #20622 - puppet import does not refresh all statuses #4749

Merged
merged 1 commit into from Aug 24, 2017

Conversation

lzap
Copy link
Member

@lzap lzap commented Aug 17, 2017

It appears that puppet reports cause all statuses to be updated:

https://github.com/theforeman/foreman/blob/develop/app/services/report_importer.rb#L42

On a heavily loaded system this could mean this is being called 100s of times per second. This is a fairly slow operation to be doing when not necessary.

Puppet reports should only refresh statuses that are relevant to them.

@theforeman-bot
Copy link
Member

Issues: #20622

@lzap
Copy link
Member Author

lzap commented Aug 17, 2017

@jlsherrill how about that? In Katello we can now override the method same way to only refresh relevant status(es).

logger.info("Imported report for #{name} in #{(Time.now.utc - start_time).round(2)} seconds")
host.refresh_statuses
imported_time = Time.now
host.refresh_statuses(statuses_for_refresh)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is the only change in this method, the rest are changes in logging so it produces saner output.

@@ -88,6 +89,10 @@ def report_status
raise NotImplementedError
end

def statuses_for_refresh
HostStatus.status_registry
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder, is there a way to infer which status changed from the importer class rather then defaulting to refreshing all of them and having to edit all of the importers manually?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is, I'd rather prefer separate PR to be honest :-)

def refresh_statuses
HostStatus.status_registry.each do |status_class|
def refresh_statuses(which = HostStatus.status_registry)
which.each do |status_class|
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any situation where one report import needs to update multiple statuses?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the idea came from build substatus where we wanted to flip the state with any other state update. It seems to be displayed correctly but it wouldn't be reflected in DB if it was not updated on this refresh_statuses call, so we wouldn't be able to search for host which token expired.

now when we have active job, we could have a job to verify this state in Time.now + $provisioning_interval

@lzap
Copy link
Member Author

lzap commented Aug 22, 2017

Not really sure why all PG tests failed, rebased. This is ready for review.

@lzap
Copy link
Member Author

lzap commented Aug 23, 2017

@tbrisker mind taking a closer look?

Copy link
Member

@dLobatog dLobatog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marked as 1.15.4 - please label stuff the same as Redmine when it should be cherry-picked

Copy link
Member

@tbrisker tbrisker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, looks good. Could you please open a PR leveraging this change for other importers?

@tbrisker tbrisker merged commit be67061 into theforeman:develop Aug 24, 2017
@tbrisker
Copy link
Member

Thanks @lzap !

@lzap lzap deleted the importer-status-20622 branch August 29, 2017 07:48
@lzap
Copy link
Member Author

lzap commented Aug 29, 2017

Wait @tbrisker which are you referring to? Can't find any.

@lzap
Copy link
Member Author

lzap commented Dec 8, 2017

Updated docs, filed: http://projects.theforeman.org/issues/21910

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants