Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #32203 - Add timeout option to ansible runner #41

Merged
merged 1 commit into from Jul 14, 2021

Conversation

yifatmakias
Copy link

@yifatmakias yifatmakias commented Jun 22, 2021

No description provided.

@xprazak2
Copy link
Contributor

The timeout is respected, run finishes with 0 and is marked as success:

ansible-term

For ssh jobs, timeout is a failure. Any chance we could make it a failure as well when we hit the timeout?

@@ -19,6 +20,7 @@ def initialize(input, suspended_action:)
@check_mode = action_input[:check_mode]
@tags = action_input[:tags]
@tags_flag = action_input[:tags_flag]
@job_invocation_timeout = action_input[:job_invocation_timeout]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we use different naming than ScriptRunner, which use execution_timeout_interval?
I'd love if we could keep the naming matching for the same purpose, or is there a reason for different name?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ezr-ondrej From what I understood the execution_timeout_interval is from the template invocation but if the value is nil in the template it does not mean that in the specific job invocation the timeout is nil as well. so I created another variable for the job_invocation timeout.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well ... if I'm reading the theforeman/foreman_ansible#422 correctly, you're getting it from job_invocation anyway, I don't think template_invocation has timeout. All the user inputs are on job_invocation, so that would be weird.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ezr-ondrej you are right there is no need to add another parameter. I will fix this.

@xprazak2
Copy link
Contributor

xprazak2 commented Jul 7, 2021

Needs a rebase now.
I am still getting the exit as 0, is it just me?

@yifatmakias
Copy link
Author

Needs a rebase now.
I am still getting the exit as 0, is it just me?

No I haven't fixed it yet, just uploaded another update. sorry for the confusion

@yifatmakias
Copy link
Author

Needs a rebase now.
I am still getting the exit as 0, is it just me?

@xprazak2 This should work now.

Copy link
Member

@ezr-ondrej ezr-ondrej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry for keep digging in the code, my limited knowledge prevent me to see everything the first time around. But I strongly believe this one is the last!! xD

hosts = @inventory['all']['hosts']
hosts.each do |hostname|
@exit_statuses[hostname] = 2
publish_data_for(hostname, 'Timeout for execution passed, stopping the job', 'stdout')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this message work as broadcasted by broadcast_data?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes definitely, why is it better to use the broadcast_data?

hosts.each do |hostname|
@exit_statuses[hostname] = 2
end
broadcast_data('Timeout for execution passed, stopping the job', 'stdout')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe stderr would be better? After all it is more of an error condition.

Comment on lines 66 to 69
hosts = @inventory['all']['hosts']
hosts.each do |hostname|
@exit_statuses[hostname] = 2
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could probably go on a single line.

Copy link
Member

@ezr-ondrej ezr-ondrej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@xprazak2
Copy link
Contributor

Works as expected. I see an error in proxy logs when terminating on timeout, could it cause problems?
proxy-term

@yifatmakias
Copy link
Author

Works as expected. I see an error in proxy logs when terminating on timeout, could it cause problems?
proxy-term

I think it is related to this code: https://github.com/theforeman/smart_proxy_ansible/pull/41/files#:~:text=FileUtils.remove_entry(%40root)%20if%20%40tmp_working_dir

but I am not sure why this is happening. @ezr-ondrej did you saw this error too?

@ezr-ondrej
Copy link
Member

but I am not sure why this is happening. @ezr-ondrej did you saw this error too?

Is it possible, than if the ansible-runner process is killed, it cleans after itself? so there is no garbage left to clean for us?
I believe so and in that case, I would just not call close, it is weird to call close from kill anyway :)

@yifatmakias
Copy link
Author

but I am not sure why this is happening. @ezr-ondrej did you saw this error too?

Is it possible, than if the ansible-runner process is killed, it cleans after itself? so there is no garbage left to clean for us?
I believe so and in that case, I would just not call close, it is weird to call close from kill anyway :)

I checked the timeout does not work without the calling the close method. but this line is necessary:
FileUtils.remove_entry(@root) if @tmp_working_dir

It should work now with no errors.

@xprazak2
Copy link
Contributor

I no longer see the error, thanks!

@xprazak2 xprazak2 merged commit d9ec476 into theforeman:master Jul 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants