Skip to content

Default timeout #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 5, 2018
Merged

Default timeout #16

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8cd16e2
Set default timeout to 1 hour. Fix #15
penguinolog Apr 5, 2018
d29c6a8
align docs and docstrings
penguinolog Apr 5, 2018
bd6bd9d
deal with unicode_literals in setup.py
penguinolog Apr 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,12 +121,14 @@ Main methods are `execute`, `check_call` and `check_stderr` for simple executing
and executing, checking return code and checking for empty stderr output.
This methods are almost the same for `SSHCleint` and `Subprocess`, except specific flags.

.. note:: By default ALL methods have timeout 1 hour, infinite waiting can be enabled, but it's special case.

.. code-block:: python

result = helper.execute(
command, # type: str
verbose=False, # type: bool
timeout=None, # type: typing.Optional[int]
timeout=1 * 60 * 60, # type: typing.Optional[int]
**kwargs
)

Expand All @@ -136,7 +138,7 @@ This methods are almost the same for `SSHCleint` and `Subprocess`, except specif
result = helper.check_call(
command, # type: str
verbose=False, # type: bool
timeout=None, # type: typing.Optional[int]
timeout=1 * 60 * 60, # type: typing.Optional[int]
error_info=None, # type: typing.Optional[str]
expected=None, # type: typing.Optional[typing.Iterable[int]]
raise_on_err=True, # type: bool
Expand All @@ -148,7 +150,7 @@ This methods are almost the same for `SSHCleint` and `Subprocess`, except specif
result = helper.check_stderr(
command, # type: str
verbose=False, # type: bool
timeout=None, # type: typing.Optional[int]
timeout=1 * 60 * 60, # type: typing.Optional[int]
error_info=None, # type: typing.Optional[str]
raise_on_err=True, # type: bool
)
Expand Down Expand Up @@ -193,7 +195,7 @@ Possible to call commands in parallel on multiple hosts if it's not produce huge
results = SSHClient.execute_together(
remotes, # type: typing.Iterable[SSHClient]
command, # type: str
timeout=None, # type: typing.Optional[int]
timeout=1 * 60 * 60, # type: typing.Optional[int]
expected=None, # type: typing.Optional[typing.Iterable[int]]
raise_on_err=True # type: bool
)
Expand All @@ -211,7 +213,7 @@ For execute through SSH host can be used `execute_through_host` method:
command, # type: str
auth=None, # type: typing.Optional[SSHAuth]
target_port=22, # type: int
timeout=None, # type: typing.Optional[int]
timeout=1 * 60 * 60, # type: typing.Optional[int]
verbose=False, # type: bool
get_pty=False, # type: bool
)
Expand Down
27 changes: 18 additions & 9 deletions doc/source/SSHClient.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,14 +79,14 @@ API: SSHClient and SSHAuth.

Open context manager

.. versionchanged:: 1.1.0 - lock on enter
.. versionchanged:: 1.1.0 lock on enter

.. py:method:: __exit__(self, exc_type, exc_val, exc_tb)

Close context manager and disconnect

.. versionchanged:: 1.0.0 - disconnect enforced on close
.. versionchanged:: 1.1.0 - release lock on exit
.. versionchanged:: 1.0.0 disconnect enforced on close
.. versionchanged:: 1.1.0 release lock on exit

.. py:method:: sudo(enforce=None)

Expand All @@ -109,9 +109,9 @@ API: SSHClient and SSHAuth.
:type open_stderr: bool
:rtype: ``typing.Tuple[paramiko.Channel, paramiko.ChannelFile, paramiko.ChannelFile, paramiko.ChannelFile]``

.. versionchanged:: 1.2.0 - open_stdout and open_stderr flags
.. versionchanged:: 1.2.0 open_stdout and open_stderr flags

.. py:method:: execute(command, verbose=False, timeout=None, **kwargs)
.. py:method:: execute(command, verbose=False, timeout=1*60*60, **kwargs)

Execute command and wait for return code.

Expand All @@ -124,7 +124,9 @@ API: SSHClient and SSHAuth.
:rtype: ExecResult
:raises: ExecHelperTimeoutError

.. py:method:: check_call(command, verbose=False, timeout=None, error_info=None, expected=None, raise_on_err=True, **kwargs)
.. versionchanged:: 1.2.0 default timeout 1 hour

.. py:method:: check_call(command, verbose=False, timeout=1*60*60, error_info=None, expected=None, raise_on_err=True, **kwargs)

Execute command and check for return code.

Expand All @@ -143,7 +145,9 @@ API: SSHClient and SSHAuth.
:rtype: ExecResult
:raises: CalledProcessError

.. py:method:: check_stderr(command, verbose=False, timeout=None, error_info=None, raise_on_err=True, **kwargs)
.. versionchanged:: 1.2.0 default timeout 1 hour

.. py:method:: check_stderr(command, verbose=False, timeout=1*60*60, error_info=None, raise_on_err=True, **kwargs)

Execute command expecting return code 0 and empty STDERR.

Expand All @@ -161,8 +165,9 @@ API: SSHClient and SSHAuth.
:raises: CalledProcessError

.. note:: expected return codes can be overridden via kwargs.
.. versionchanged:: 1.2.0 default timeout 1 hour

.. py:method:: execute_through_host(hostname, command, auth=None, target_port=22, verbose=False, timeout=None, get_pty=False, **kwargs)
.. py:method:: execute_through_host(hostname, command, auth=None, target_port=22, verbose=False, timeout=1*60*60, get_pty=False, **kwargs)

Execute command on remote host through currently connected host.

Expand All @@ -183,7 +188,9 @@ API: SSHClient and SSHAuth.
:rtype: ExecResult
:raises: ExecHelperTimeoutError

.. py:classmethod:: execute_together(remotes, command, timeout=None, expected=None, raise_on_err=True, **kwargs)
.. versionchanged:: 1.2.0 default timeout 1 hour

.. py:classmethod:: execute_together(remotes, command, timeout=1*60*60, expected=None, raise_on_err=True, **kwargs)

Execute command on multiple remotes in async mode.

Expand All @@ -202,6 +209,8 @@ API: SSHClient and SSHAuth.
:raises: ParallelCallProcessError
:raises: ParallelCallExceptions

.. versionchanged:: 1.2.0 default timeout 1 hour

.. py:method:: open(path, mode='r')

Open file on remote using SFTP session.
Expand Down
23 changes: 14 additions & 9 deletions doc/source/Subprocess.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,15 @@ API: Subprocess

Open context manager

.. versionchanged:: 1.1.0 - lock on enter
.. versionchanged:: 1.1.0 lock on enter

.. py:method:: __exit__(self, exc_type, exc_val, exc_tb)

Close context manager

.. versionchanged:: 1.1.0 - release lock on exit
.. versionchanged:: 1.1.0 release lock on exit

.. py:method:: execute(command, verbose=False, timeout=None, **kwargs)
.. py:method:: execute(command, verbose=False, timeout=1*60*60, **kwargs)

Execute command and wait for return code.

Expand All @@ -37,10 +37,13 @@ API: Subprocess
:rtype: ExecResult
:raises: ExecHelperTimeoutError

.. versionchanged:: 1.1.0 - make method
.. versionchanged:: 1.2.0 - open_stdout and open_stderr flags
.. versionchanged:: 1.1.0 make method
.. versionchanged:: 1.2.0

.. py:method:: check_call(command, verbose=False, timeout=None, error_info=None, expected=None, raise_on_err=True, **kwargs)
open_stdout and open_stderr flags
default timeout 1 hour

.. py:method:: check_call(command, verbose=False, timeout=1*60*60, error_info=None, expected=None, raise_on_err=True, **kwargs)

Execute command and check for return code.

Expand All @@ -59,9 +62,10 @@ API: Subprocess
:rtype: ExecResult
:raises: CalledProcessError

.. versionchanged:: 1.1.0 - make method
.. versionchanged:: 1.1.0 make method
.. versionchanged:: 1.2.0 default timeout 1 hour

.. py:method:: check_stderr(command, verbose=False, timeout=None, error_info=None, raise_on_err=True, **kwargs)
.. py:method:: check_stderr(command, verbose=False, timeout=1*60*60, error_info=None, raise_on_err=True, **kwargs)

Execute command expecting return code 0 and empty STDERR.

Expand All @@ -80,4 +84,5 @@ API: Subprocess

.. note:: expected return codes can be overridden via kwargs.

.. versionchanged:: 1.1.0 - make method
.. versionchanged:: 1.1.0 make method
.. versionchanged:: 1.2.0 default timeout 1 hour
1 change: 1 addition & 0 deletions exec_helpers/_log_templates.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"""Text templates for logging."""

from __future__ import absolute_import
from __future__ import division
from __future__ import unicode_literals

CMD_EXEC = "Executing command:\n{cmd!s}\n"
Expand Down
34 changes: 23 additions & 11 deletions exec_helpers/_ssh_client_base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@

"""SSH client helper based on Paramiko. Base class."""

from __future__ import absolute_import
from __future__ import division
from __future__ import unicode_literals

import base64
Expand All @@ -37,11 +39,12 @@
import threaded
import six

from exec_helpers import exceptions
from exec_helpers import constants
from exec_helpers import exec_result
from exec_helpers import _log_templates
from exec_helpers import exceptions
from exec_helpers import proc_enums
from exec_helpers import ssh_auth
from exec_helpers import _log_templates

__all__ = ('SSHClientBase', )

Expand Down Expand Up @@ -474,16 +477,16 @@ def __del__(self):
def __enter__(self):
"""Get context manager.

.. versionchanged:: 1.1.0 - lock on enter
.. versionchanged:: 1.1.0 lock on enter
"""
self.lock.acquire()
return self

def __exit__(self, exc_type, exc_val, exc_tb):
"""Exit context manager.

.. versionchanged:: 1.0.0 - disconnect enforced on close
.. versionchanged:: 1.1.0 - release lock on exit
.. versionchanged:: 1.0.0 disconnect enforced on close
.. versionchanged:: 1.1.0 release lock on exit
"""
self.close()
self.lock.release()
Expand Down Expand Up @@ -534,7 +537,7 @@ def execute_async(
typing.Optional[paramiko.ChannelFile],
]

.. versionchanged:: 1.2.0 - open_stdout and open_stderr flags
.. versionchanged:: 1.2.0 open_stdout and open_stderr flags
"""
message = _log_templates.CMD_EXEC.format(cmd=command.rstrip())
self.logger.debug(message)
Expand Down Expand Up @@ -690,7 +693,7 @@ def execute(
self,
command, # type: str
verbose=False, # type: bool
timeout=None, # type: typing.Optional[int]
timeout=constants.DEFAULT_TIMEOUT, # type: typing.Optional[int]
**kwargs
): # type: (...) -> exec_result.ExecResult
"""Execute command and wait for return code.
Expand All @@ -703,6 +706,8 @@ def execute(
:type timeout: typing.Optional[int]
:rtype: ExecResult
:raises: ExecHelperTimeoutError

.. versionchanged:: 1.2.0 default timeout 1 hour
"""
(
chan, # type: paramiko.channel.Channel
Expand All @@ -726,7 +731,7 @@ def check_call(
self,
command, # type: str
verbose=False, # type: bool
timeout=None, # type: typing.Optional[int]
timeout=constants.DEFAULT_TIMEOUT, # type: typing.Optional[int]
error_info=None, # type: typing.Optional[str]
expected=None, # type: typing.Optional[typing.Iterable[]]
raise_on_err=True, # type: bool
Expand All @@ -748,6 +753,8 @@ def check_call(
:type raise_on_err: bool
:rtype: ExecResult
:raises: CalledProcessError

.. versionchanged:: 1.2.0 default timeout 1 hour
"""
expected = proc_enums.exit_codes_to_enums(expected)
ret = self.execute(command, verbose, timeout, **kwargs)
Expand All @@ -770,7 +777,7 @@ def check_stderr(
self,
command, # type: str
verbose=False, # type: bool
timeout=None, # type: typing.Optional[int]
timeout=constants.DEFAULT_TIMEOUT, # type: typing.Optional[int]
error_info=None, # type: typing.Optional[str]
raise_on_err=True, # type: bool
**kwargs
Expand All @@ -791,6 +798,7 @@ def check_stderr(
:raises: CalledProcessError

.. note:: expected return codes can be overridden via kwargs.
.. versionchanged:: 1.2.0 default timeout 1 hour
"""
ret = self.check_call(
command, verbose, timeout=timeout,
Expand All @@ -816,7 +824,7 @@ def execute_through_host(
auth=None, # type: typing.Optional[ssh_auth.SSHAuth]
target_port=22, # type: int
verbose=False, # type: bool
timeout=None, # type: typing.Optional[int]
timeout=constants.DEFAULT_TIMEOUT, # type: typing.Optional[int]
get_pty=False, # type: bool
**kwargs
): # type: (...) -> exec_result.ExecResult
Expand All @@ -838,6 +846,8 @@ def execute_through_host(
:type get_pty: bool
:rtype: ExecResult
:raises: ExecHelperTimeoutError

.. versionchanged:: 1.2.0 default timeout 1 hour
"""
if auth is None:
auth = self.auth
Expand Down Expand Up @@ -880,7 +890,7 @@ def execute_together(
cls,
remotes, # type: typing.Iterable[SSHClientBase]
command, # type: str
timeout=None, # type: typing.Optional[int]
timeout=constants.DEFAULT_TIMEOUT, # type: typing.Optional[int]
expected=None, # type: typing.Optional[typing.Iterable[]]
raise_on_err=True, # type: bool
**kwargs
Expand All @@ -901,6 +911,8 @@ def execute_together(
:rtype: typing.Dict[typing.Tuple[str, int], exec_result.ExecResult]
:raises: ParallelCallProcessError
:raises: ParallelCallExceptions

.. versionchanged:: 1.2.0 default timeout 1 hour
"""
@threaded.threadpooled
def get_result(
Expand Down
27 changes: 27 additions & 0 deletions exec_helpers/constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Copyright 2018 Alexey Stepanov aka penguinolog.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should not be the same of the other files?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I realized that it's actually consistent. Never mind.

#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

"""Global constants."""

from __future__ import absolute_import
from __future__ import division
from __future__ import unicode_literals


MINUTE = 60
HOUR = 60 * MINUTE


# Default command timeout
DEFAULT_TIMEOUT = 1 * HOUR
2 changes: 2 additions & 0 deletions exec_helpers/exceptions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@
"""Package specific exceptions."""

from __future__ import absolute_import
from __future__ import division
from __future__ import unicode_literals

import typing

Expand Down
1 change: 1 addition & 0 deletions exec_helpers/exec_result.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"""Execution restult."""

from __future__ import absolute_import
from __future__ import division
from __future__ import unicode_literals

import datetime
Expand Down
1 change: 1 addition & 0 deletions exec_helpers/proc_enums.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
"""

from __future__ import absolute_import
from __future__ import division
from __future__ import unicode_literals

import enum
Expand Down
Loading