[WIP]Common: Added generic utils to common #79

nnDarshan · 2016-12-09T10:56:32Z

This patch adds generic utils like command-executor,
package-installer, service-manager to common as utility.

Signed-off-by: nnDarshan darshan.n.2024@gmail.com

coveralls · 2016-12-09T10:58:56Z

Coverage decreased (-8.4%) to 29.389% when pulling 250f00a on nnDarshan:AddGenericUtilities into d30c66f on Tendrl:master.

nnDarshan · 2016-12-09T10:59:39Z

@r0h4n @nthomas-redhat @shtripat @brainfunked Its initial WIP patch for adding general utilities.

@shtripat Can you please help me by providing an issue for this, when you create one for the re factoring specification.

coveralls · 2016-12-12T09:49:39Z

Coverage decreased (-9.4%) to 28.407% when pulling c86f74e on nnDarshan:AddGenericUtilities into d30c66f on Tendrl:master.

nnDarshan · 2016-12-12T13:28:32Z

@r0h4n @nthomas-redhat @shtripat Please review.

coveralls · 2016-12-12T13:29:09Z

Coverage increased (+14.006%) to 51.823% when pulling d4909a7 on nnDarshan:AddGenericUtilities into d30c66f on Tendrl:master.

nnDarshan · 2016-12-12T13:31:30Z

@shtripat If you find it easy to use these utils as part of your re-factoring, please go ahead. If you find it would make your re-factoring patch huge , I will send a separate PR afterwards to do that. In any case, please provide your thoughts

shtripat · 2016-12-13T04:12:03Z

@nnDarshan I feel doing this as part of different PR is good as refactoring PR is already huge and I am targetting more of cleanup, movement to common module and proper class hierarchies etc as part of my PR.

@r0h4n @brainfunked @nthomas-redhat thoughts??

@nnDarshan also add spec and issues link in commit message

nnDarshan · 2016-12-13T09:54:25Z

tendrl-spec: Tendrl/specifications#31
tendrl-bug-id: #80

shtripat · 2016-12-13T11:29:03Z

tendrl/common/utils/service.py

+    def restart(self):
+        attr = self.attributes
+        attr.update({"state": "restarted"})
+        return self.__run_module(attr)


Is it possible to add a status() function as well. Few modules need that I feel.

The ansible module which we are using to manage the service has only 4 types of operations namely start, stop, restart and reload(http://docs.ansible.com/ansible/service_module.html refer state under options). So its not possible with this approach.
I feel when we monitor systemd dbus signals we will be able to obtain the service states

start and stop are idempotent actions that will not run commands unless necessary, so you can rely on these itself and you get the status in return. Can we take that approach?

But using this approach we can't just read the status without changing it right ? If we already know that we want the state to be either started or stopped we can directly use the start/stopped method. I am not getting the advantage of using a status that way. Could you please elaborate ?

shtripat · 2016-12-13T12:00:32Z

tendrl-common.spec

@@ -7,13 +7,15 @@ Source0: %{name}-%{version}.tar.gz
 License: LGPLv2+
 URL: https://github.com/Tendrl/common

+BuildRequires: ansible


I remember on rhel by default ansible 2.2 was not getting downloaded. We should explicitly mention > 2.2 here if you use any ansible 2.2 specific feature.

shtripat · 2016-12-13T12:02:09Z

tendrl/common/tests/test_ansible_runner.py

+    import AnsibleRunner
+
+
+class Test_ansible_runner_constructor(object):


Better use CamelCase for all the class names

shtripat · 2016-12-13T12:04:09Z

tendrl/common/utils/install.py

@@ -0,0 +1,38 @@
+from ansible_module_runner import AnsibleExecutableGenerationFailed


I feel source could be named as package_installer.py

shtripat · 2016-12-13T12:05:49Z

tendrl/common/utils/service.py

+    def restart(self):
+        attr = self.attributes
+        attr.update({"state": "restarted"})
+        return self.__run_module(attr)


shtripat

LGTM

r0h4n · 2016-12-15T07:09:38Z

tendrl/common/utils/command.py

+
+class Command(object):
+    def __init__(self, command):
+        self.attributes = {"_raw_params": command}


This needs a whitelist of good commands we use or some sorta security check

This patch adds generic utils like command-executor, package-installer, service-manager to common as utility. tendrl-spec: Tendrl/specifications#31 tendrl-bug-id: Tendrl#80 Signed-off-by: nnDarshan <darshan.n.2024@gmail.com>

This commit adds the unit tests for the general purpose utils namely installer, command executor and service manager. tendrl-spec: Tendrl/specifications#31 tendrl-bug-id: Tendrl#80 Signed-off-by: nnDarshan <darshan.n.2024@gmail.com>

Addressed some review comments tendrl-spec:refactor_common_code_and_move tendrl-bug-id: Tendrl#80 Signed-off-by: nnDarshan <darshan.n.2024@gmail.com>

Added a list of safe commands, only these safe commands can be executed by the util. tendrl-spec:add_utils_to_common tendrl-bug-id: Tendrl#80 Signed-off-by: nnDarshan <darshan.n.2024@gmail.com>

coveralls · 2016-12-15T14:37:37Z

Coverage increased (+15.9%) to 53.48% when pulling 01c156f on nnDarshan:AddGenericUtilities into e1aa599 on Tendrl:master.

nnDarshan · 2016-12-15T14:38:29Z

@r0h4n @nthomas-redhat @brainfunked Please Review. Have addressed the commets

shtripat · 2016-12-15T15:22:54Z

tendrl/common/utils/command.py

@@ -5,9 +5,25 @@
 ANSIBLE_MODULE_PATH = "core/commands/command.py"
 LOG = logging.getLogger(__name__)

+SAFE_COMMAND_LIST = [


Should we externalize these command in tendrl owned yaml file? Just a thought, but if we are very strict on list of commands supported, then it ok to keep inside code. My only point is just for a new command we would need a code change later.

Comment??

My opinion is having it in code is better, keeping the security in mind.
@nthomas-redhat @r0h4n @brainfunked Please provide your thoughts

shtripat

Looks good to me

nthomas-redhat · 2016-12-17T13:48:27Z

tendrl/common/utils/service.py

+from ansible_module_runner import AnsibleRunner
+import logging
+
+ANSIBLE_MODULE_PATH = "core/system/systemd.py"


This supports only systemd services only right?
If we want to support old versions of LINUX where systemd is not available(ex;RHEL6), this would not be enough. I think the service module(https://github.com/ansible/ansible-modules-core/blob/devel/system/service.py) has support for different init modules including systemd and BSD init. Have considered that?

nthomas-redhat · 2016-12-17T13:57:43Z

tendrl/common/utils/service.py

+    def restart(self):
+        attr = self.attributes
+        attr.update({"state": "restarted"})
+        return self.__run_module(attr)


start and stop are idempotent actions that will not run commands unless necessary, so you can rely on these itself and you get the status in return. Can we take that approach?

Using service module for service management as it is more generic. Also provided configurable util execution path. tendrl-spec:add_utils_to_common tendrl-bug-id: Tendrl#80 Signed-off-by: nnDarshan <darshan.n.2024@gmail.com>

coveralls · 2016-12-28T10:35:03Z

Coverage increased (+15.8%) to 53.333% when pulling 7a4de9d on nnDarshan:AddGenericUtilities into e1aa599 on Tendrl:master.

coveralls · 2017-01-05T09:00:13Z

Coverage increased (+8.4%) to 56.48% when pulling 0fa1912 on nnDarshan:AddGenericUtilities into 3b96011 on Tendrl:master.

nnDarshan force-pushed the AddGenericUtilities branch from 250f00a to c86f74e Compare December 12, 2016 09:47

shtripat reviewed Dec 13, 2016

View reviewed changes

shtripat mentioned this pull request Dec 13, 2016

node_agent: Atom to check service status + Alerts socket Tendrl/node-agent#67

Closed

shtripat reviewed Dec 13, 2016

View reviewed changes

shtripat reviewed Dec 15, 2016

View reviewed changes

r0h4n suggested changes Dec 15, 2016

View reviewed changes

nnDarshan added 4 commits December 15, 2016 20:03

Common: Added generic utils to common

0475b83

This patch adds generic utils like command-executor, package-installer, service-manager to common as utility. tendrl-spec: Tendrl/specifications#31 tendrl-bug-id: Tendrl#80 Signed-off-by: nnDarshan <darshan.n.2024@gmail.com>

common: Added generic utils to common

c8983e2

This commit adds the unit tests for the general purpose utils namely installer, command executor and service manager. tendrl-spec: Tendrl/specifications#31 tendrl-bug-id: Tendrl#80 Signed-off-by: nnDarshan <darshan.n.2024@gmail.com>

Common: Added generic utils to common

baf1499

Addressed some review comments tendrl-spec:refactor_common_code_and_move tendrl-bug-id: Tendrl#80 Signed-off-by: nnDarshan <darshan.n.2024@gmail.com>

Common: Added generic utils to common

01c156f

Added a list of safe commands, only these safe commands can be executed by the util. tendrl-spec:add_utils_to_common tendrl-bug-id: Tendrl#80 Signed-off-by: nnDarshan <darshan.n.2024@gmail.com>

nnDarshan force-pushed the AddGenericUtilities branch from d04dcb7 to 01c156f Compare December 15, 2016 14:34

shtripat reviewed Dec 15, 2016

View reviewed changes

shtripat reviewed Dec 16, 2016

View reviewed changes

nthomas-redhat requested changes Dec 17, 2016

View reviewed changes

shtripat assigned nnDarshan Dec 19, 2016

Common: Added generic utils to common

7a4de9d

Using service module for service management as it is more generic. Also provided configurable util execution path. tendrl-spec:add_utils_to_common tendrl-bug-id: Tendrl#80 Signed-off-by: nnDarshan <darshan.n.2024@gmail.com>

r0h4n approved these changes Jan 5, 2017

View reviewed changes

Merge branch 'master' into AddGenericUtilities

0fa1912

r0h4n merged commit 01fe6c6 into Tendrl:master Jan 5, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP]Common: Added generic utils to common #79

[WIP]Common: Added generic utils to common #79

nnDarshan commented Dec 9, 2016

coveralls commented Dec 9, 2016

nnDarshan commented Dec 9, 2016

coveralls commented Dec 12, 2016

nnDarshan commented Dec 12, 2016

coveralls commented Dec 12, 2016

nnDarshan commented Dec 12, 2016

shtripat commented Dec 13, 2016 •

edited

Loading

nnDarshan commented Dec 13, 2016

shtripat Dec 13, 2016

nnDarshan Dec 13, 2016

shtripat Dec 13, 2016

nthomas-redhat Dec 17, 2016

nnDarshan Dec 27, 2016

shtripat Dec 13, 2016

nnDarshan Dec 14, 2016

shtripat Dec 13, 2016

nnDarshan Dec 14, 2016

shtripat Dec 13, 2016

nnDarshan Dec 14, 2016

shtripat Dec 13, 2016

shtripat left a comment

r0h4n Dec 15, 2016

coveralls commented Dec 15, 2016

nnDarshan commented Dec 15, 2016

shtripat Dec 15, 2016

nnDarshan Dec 15, 2016

shtripat Dec 27, 2016

shtripat left a comment

nthomas-redhat Dec 17, 2016

nnDarshan Dec 28, 2016

nthomas-redhat Dec 17, 2016

coveralls commented Dec 28, 2016

coveralls commented Jan 5, 2017

		import AnsibleRunner


		class Test_ansible_runner_constructor(object):

		@@ -0,0 +1,38 @@
		from ansible_module_runner import AnsibleExecutableGenerationFailed

[WIP]Common: Added generic utils to common #79

[WIP]Common: Added generic utils to common #79

Conversation

nnDarshan commented Dec 9, 2016

coveralls commented Dec 9, 2016

nnDarshan commented Dec 9, 2016

coveralls commented Dec 12, 2016

nnDarshan commented Dec 12, 2016

coveralls commented Dec 12, 2016

nnDarshan commented Dec 12, 2016

shtripat commented Dec 13, 2016 • edited Loading

nnDarshan commented Dec 13, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shtripat left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coveralls commented Dec 15, 2016

nnDarshan commented Dec 15, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shtripat left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coveralls commented Dec 28, 2016

coveralls commented Jan 5, 2017

shtripat commented Dec 13, 2016 •

edited

Loading