AMBARI-23536. Fix TestCustomServiceOrchestrator.py and TestRegistration #966

aonishuk · 2018-04-11T07:08:25Z

No description provided.

…on.py on branch-3.0-perf (aonishuk)

adoroszlai · 2018-04-11T07:22:58Z

ambari-agent/src/test/python/ambari_agent/TestRegistration.py

@@ -50,17 +50,15 @@ def test_registration_build(self, get_os_version_mock, get_os_type_mock, run_os_
    run_os_cmd_mock.return_value = (3, "", "")
    register = Register(config)
    reference_version = '2.1.0'
-    data = register.build(reference_version, 1)
+    data = register.build()


This gives error on Mac.

File "ambari-agent/src/main/python/ambari_agent/HostInfo.py", line 86, in checkLiveServices if "ntpd" in service and is_redhat7_or_higher: TypeError: argument of type 'NoneType' is not iterable

@adoroszlai can you please provide reproduce of this?

adoroszlai · 2018-04-11T07:36:33Z

ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py

  @patch.object(CustomServiceOrchestrator, "get_py_executor")
  @patch("ambari_commons.shell.kill_process_with_children")
  @patch.object(FileCache, "__init__")
  @patch.object(CustomServiceOrchestrator, "resolve_script_path")
  @patch.object(CustomServiceOrchestrator, "resolve_hook_script_path")
  def test_cancel_backgound_command(self, resolve_hook_script_path_mock,
                                    resolve_script_path_mock, FileCache_mock, kill_process_with_children_mock,
-                                    get_py_executor_mock):
+                                    get_py_executor_mock, get_configuration_mock):


test_cancel_background_command fails for me with:

File "ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py", line 409, in test_cancel_backgound_command self.assertTrue(kill_process_with_children_mock.called) AssertionError: False is not true

@adoroszlai can you please provide reproduce of this?

I don't remember the exact place. But it is fixed you can re-run it for yourself manually.

asfgit · 2018-04-11T07:52:16Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/1763/
Test FAILed.
Test FAILured.

…on.py on branch-3.0-perf (aonishuk)

asfgit · 2018-04-11T10:29:27Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/1766/
Test FAILed.
Test FAILured.

aonishuk · 2018-04-11T10:30:56Z

Agent tests suceeded in jenkins the failure is not related.

adoroszlai · 2018-04-11T10:47:34Z

ambari-agent/src/test/python/ambari_agent/TestActionQueue.py

@@ -45,7 +45,7 @@

 CLUSTER_ID = '0'

-class TestActionQueue:#(TestCase):
+class TestActionQueue(TestCase):


Can you please remove or fix the test cases with @not_for_platform(PLATFORM_LINUX)? The following ones are erroring:

test_execute_retryable_command test_execute_retryable_command_fail_and_succeed test_execute_retryable_command_succeed

due to:

actionQueue = ActionQueue(AmbariConfig(), dummy_controller) TypeError: __init__() takes exactly 2 arguments (3 given)

removed the tests

adoroszlai · 2018-04-11T10:52:46Z

ambari-agent/src/test/python/ambari_agent/TestRegistration.py

    get_os_type_mock.return_value = "suse"
    get_os_version_mock.return_value = "11"
    run_os_cmd_mock.return_value = (3, "", "")
+    from ambari_agent.Register import Register


Unfortunately this fix works only if TestRegistration is run by itself. If all tests are run, then it still fails with the same error.

So I think it's best to return ("ntpd",) for other platforms from get_ntp_service.

adoroszlai · 2018-04-11T11:10:03Z

ambari-agent/src/test/python/ambari_agent/TestActionQueue.py

@@ -491,6 +494,8 @@ def test_do_not_log_execution_commands(self, command_status_dict_mock,
      [call("out\n\nCommand completed successfully!\n", "9"), call("stderr", "9")], any_order=True)
    self.assertEqual(len(reports), 1)
    self.assertEqual(expected, reports[0])
+    import threading
+    print threading.enumerate()


Is this needed?

thanks. Removed.

adoroszlai · 2018-04-11T11:33:24Z

TestCustomServiceOrchestrator is fine if run in itself, but 5 tests are consistently failing locally if all tests are run.

retest this please

asfgit · 2018-04-11T12:22:27Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/1768/
Test FAILed.
Test FAILured.

…on.py on branch-3.0-perf (aonishuk)

adoroszlai · 2018-04-12T09:11:51Z

@aonishuk Thanks for the changes, TestRegistration and TestActionQueue are fine now. Please see my previous comment about TestCustomServiceOrchestrator: it's still failing when all tests are run. Would it be possible to keep that test disabled if it cannot be fixed now?

asfgit · 2018-04-12T09:32:30Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/1800/
Test FAILed.
Test FAILured.

…on.py on branch-3.0-perf (aonishuk)

asfgit · 2018-04-13T03:09:00Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/1829/
Test FAILed.
Test FAILured.

adoroszlai · 2018-04-13T06:56:43Z

Traceback (most recent call last):
  File "ambari-common/src/test/python/mock/mock.py", line 1199, in patched
    return func(*args, **keywargs)
  File "ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py", line 340, in test_cancel_command
    raise Exception(ret)
Exception: {'status': 'FAILED', 'structuredOut': '', 'exitcode': 13, 'stderr': 'stderr', 'stdout': 'out\n\nCommand failed after 1 tries\n'}

aonishuk · 2018-04-13T08:38:35Z

retest this please

aonishuk · 2018-04-13T10:02:26Z

I cannot reproduce these failures in CustomServiceOrch. And debug logging I added doesn't show anything intersting. I will commit these two tests with disabled CustomServiceOrch. @adoroszlai if you want you can investigate the failures (I see you can repro them) and create a fix for that separately.

…gistration.py on branch-3.0-perf (aonishuk)" This reverts commit 2f2e6dd.

…on.py on branch-3.0-perf (aonishuk)

asfgit · 2018-04-13T10:45:40Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/1831/
Test FAILed.
Test FAILured.

asfgit · 2018-04-13T11:17:51Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/1833/
Test PASSed.

AMBARI-23536. Fix TestCustomServiceOrchestrator.py and TestRegistrati…

84649d9

…on.py on branch-3.0-perf (aonishuk)

aonishuk self-assigned this Apr 11, 2018

aonishuk requested review from swagle and vbrodetskyi April 11, 2018 07:08

adoroszlai suggested changes Apr 11, 2018

View reviewed changes

AMBARI-23536. Fix TestCustomServiceOrchestrator.py and TestRegistrati…

f5322df

…on.py on branch-3.0-perf (aonishuk)

adoroszlai suggested changes Apr 11, 2018

View reviewed changes

AMBARI-23536. Fix TestCustomServiceOrchestrator.py and TestRegistrati…

34114fd

…on.py on branch-3.0-perf (aonishuk)

d0zen1 approved these changes Apr 12, 2018

View reviewed changes

swagle approved these changes Apr 12, 2018

View reviewed changes

AMBARI-23536. Fix TestCustomServiceOrchestrator.py and TestRegistrati…

2f2e6dd

…on.py on branch-3.0-perf (aonishuk)

aonishuk added 2 commits April 13, 2018 13:02

Revert "AMBARI-23536. Fix TestCustomServiceOrchestrator.py and TestRe…

de1605a

…gistration.py on branch-3.0-perf (aonishuk)" This reverts commit 2f2e6dd.

AMBARI-23536. Fix TestCustomServiceOrchestrator.py and TestRegistrati…

864eb78

…on.py on branch-3.0-perf (aonishuk)

adoroszlai approved these changes Apr 13, 2018

View reviewed changes

aonishuk merged commit 589bc8b into apache:trunk Apr 16, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AMBARI-23536. Fix TestCustomServiceOrchestrator.py and TestRegistration #966

AMBARI-23536. Fix TestCustomServiceOrchestrator.py and TestRegistration #966

aonishuk commented Apr 11, 2018

adoroszlai Apr 11, 2018

aonishuk Apr 11, 2018

adoroszlai Apr 11, 2018

aonishuk Apr 11, 2018

aonishuk Apr 11, 2018

adoroszlai Apr 11, 2018

aonishuk Apr 11, 2018

asfgit commented Apr 11, 2018

asfgit commented Apr 11, 2018

aonishuk commented Apr 11, 2018

adoroszlai Apr 11, 2018

aonishuk Apr 12, 2018

adoroszlai Apr 11, 2018

aonishuk Apr 12, 2018

adoroszlai Apr 11, 2018

aonishuk Apr 12, 2018

adoroszlai commented Apr 11, 2018

asfgit commented Apr 11, 2018

adoroszlai commented Apr 12, 2018

asfgit commented Apr 12, 2018

asfgit commented Apr 13, 2018

adoroszlai commented Apr 13, 2018

aonishuk commented Apr 13, 2018

aonishuk commented Apr 13, 2018

asfgit commented Apr 13, 2018

asfgit commented Apr 13, 2018

AMBARI-23536. Fix TestCustomServiceOrchestrator.py and TestRegistration #966

AMBARI-23536. Fix TestCustomServiceOrchestrator.py and TestRegistration #966

Conversation

aonishuk commented Apr 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

asfgit commented Apr 11, 2018

asfgit commented Apr 11, 2018

aonishuk commented Apr 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adoroszlai commented Apr 11, 2018

asfgit commented Apr 11, 2018

adoroszlai commented Apr 12, 2018

asfgit commented Apr 12, 2018

asfgit commented Apr 13, 2018

adoroszlai commented Apr 13, 2018

aonishuk commented Apr 13, 2018

aonishuk commented Apr 13, 2018

asfgit commented Apr 13, 2018

asfgit commented Apr 13, 2018