Fix599

[thefab]

Fix #599 by bypassing manage_watchers periodic callback by default in tests

The bypass is made by check_delay < 0

Note: only one test requires the periodic callback (check_delay > 0 forced on this one)

[coveralls]

Coverage remained the same when pulling 22f0582 on thefab:fix599 into 3af1fa5 on mozilla-services:master.

[scottkmaxwell]

I'm seeing a lot more test failures with this change. I just ran 4 times with this code and got 30 or 33 errors each time. Then I switched to the prior version and all tests failed. Switching back to this resulted in 30 errors and 1 failure again. I'll paste my results.

[scottkmaxwell]

test_add_watcher (circus.tests.test_arbiter.TestArbiter) ... ok
test_start_watcher (circus.tests.test_arbiter.TestArbiter) ... ok
test_start_watchers_with_autostart (circus.tests.test_arbiter.TestArbiter) ... ok
test_add_watcher (circus.tests.test_arbiter.TestTrainer) ... ok
test_add_watcher1 (circus.tests.test_arbiter.TestTrainer) ... ok
test_add_watcher2 (circus.tests.test_arbiter.TestTrainer) ... ok
test_add_watcher4 (circus.tests.test_arbiter.TestTrainer) ... ok
test_add_watcher5 (circus.tests.test_arbiter.TestTrainer) ... ok
test_add_watcher6 (circus.tests.test_arbiter.TestTrainer) ... ok
test_add_watcher7 (circus.tests.test_arbiter.TestTrainer) ... ok
test_add_watcher_already_exists (circus.tests.test_arbiter.TestTrainer) ... ok
test_add_watcher_arbiter_stopped (circus.tests.test_arbiter.TestTrainer) ... ok
test_numprocesses (circus.tests.test_arbiter.TestTrainer) ... ok
test_numwatchers (circus.tests.test_arbiter.TestTrainer) ... ok
test_processes (circus.tests.test_arbiter.TestTrainer) ... ok
test_reload (circus.tests.test_arbiter.TestTrainer) ... ok
test_reload1 (circus.tests.test_arbiter.TestTrainer) ... ok
test_reload2 (circus.tests.test_arbiter.TestTrainer) ... ok
test_rm_watcher (circus.tests.test_arbiter.TestTrainer) ... ok
test_stop_watchers (circus.tests.test_arbiter.TestTrainer) ... ok
test_stop_watchers3 (circus.tests.test_arbiter.TestTrainer) ... ok
test_watchers (circus.tests.test_arbiter.TestTrainer) ... ok
test_cli_help (circus.tests.test_circusctl.CLITest) ... ok
test_launch_cli (circus.tests.test_circusctl.CLITest) ... ok
test_add (circus.tests.test_circusctl.CommandlineTest) ... ok
test_add_start (circus.tests.test_circusctl.CommandlineTest) ... ok
test_help_for_add_command (circus.tests.test_circusctl.CommandlineTest) ... ok
test_help_invalid_command (circus.tests.test_circusctl.CommandlineTest) ... ok
test_help_switch_no_command (circus.tests.test_circusctl.CommandlineTest) ... ok
test_daemon (circus.tests.test_circusd.TestCircusd) ... SKIP: Only when Gevent is loaded
test_daemonize (circus.tests.test_circusd.TestCircusd) ... ok
test_main (circus.tests.test_circusd.TestCircusd) ... ok
test_maxfd (circus.tests.test_circusd.TestCircusd) ... ok
test_client (circus.tests.test_client.TestClient) ... ok
test_message_id (circus.tests.test_client.TestWithHook) ... ok
test_decr_proc (circus.tests.test_command_decrproc.DecrProcTest) ... ok
test_incr_proc (circus.tests.test_command_incrproc.IncrProcTest) ... ok
test_incr_proc_message (circus.tests.test_command_incrproc.IncrProcTest) ... ok
test_list_error (circus.tests.test_command_list.ListCommandTest) ... ok
test_list_processors (circus.tests.test_command_list.ListCommandTest) ... ok
test_list_watchers (circus.tests.test_command_list.ListCommandTest) ... ok
test_quit (circus.tests.test_command_quit.QuitTest) ... ok
test_set_stream (circus.tests.test_command_set.SetTest) ... ok
test_console_msg (circus.tests.test_command_stats.StatsCommandTest) ... ok
test_execute (circus.tests.test_command_stats.StatsCommandTest) ... ok
test_config_unexistant (circus.tests.test_config.TestConfig) ... ok
test_copy_env (circus.tests.test_config.TestConfig) ... ok
test_dashes (circus.tests.test_config.TestConfig) ... ok
#473 ... ok
test_env_everywhere (circus.tests.test_config.TestConfig) ... ok
test_env_section (circus.tests.test_config.TestConfig) ... ok
#554 ... ok
test_hooks (circus.tests.test_config.TestConfig) ... ok
test_include_multiple_wildcards (circus.tests.test_config.TestConfig) ... ok
test_include_wildcards (circus.tests.test_config.TestConfig) ... ok
test_issue137 (circus.tests.test_config.TestConfig) ... ok
#310 ... ok
test_issue395 (circus.tests.test_config.TestConfig) ... ok
test_issue567 (circus.tests.test_config.TestConfig) ... ok
test_loglevel (circus.tests.test_config.TestConfig) ... ok
test_logoutput (circus.tests.test_config.TestConfig) ... ok
test_override (circus.tests.test_config.TestConfig) ... ok
test_pidfile (circus.tests.test_config.TestConfig) ... ok
test_variables_everywhere (circus.tests.test_config.TestConfig) ... ok
test_watcher_env_var (circus.tests.test_config.TestConfig) ... ok
test_watcher_graceful_timeout (circus.tests.test_config.TestConfig) ... ok
test_watcher_stop_signal (circus.tests.test_config.TestConfig) ... ok
test_add_job (circus.tests.test_controller.TestController) ... ok
test_env (circus.tests.test_convert_option.TestConvertOption) ... ok
test_hooks (circus.tests.test_convert_option.TestConvertOption) ... ok
test_stdout_and_stderr_stream (circus.tests.test_convert_option.TestConvertOption) ... ok
test_pidfile (circus.tests.test_pidfile.TestPidfile) ... ok
test_default_loop_rate (circus.tests.test_plugin_command_reloader.TestCommandReloader) ... ok
test_handle_recv_implemented (circus.tests.test_plugin_command_reloader.TestCommandReloader) ... ok
test_look_after_known_watcher_triggers_restart (circus.tests.test_plugin_command_reloader.TestCommandReloader) ... ok
test_look_after_new_watcher_does_not_restart (circus.tests.test_plugin_command_reloader.TestCommandReloader) ... ok
test_missing_watcher_gets_removed_from_plugin_dict (circus.tests.test_plugin_command_reloader.TestCommandReloader) ... ok
test_mtime_is_modified (circus.tests.test_plugin_command_reloader.TestCommandReloader) ... ok
test_non_default_loop_rate (circus.tests.test_plugin_command_reloader.TestCommandReloader) ... ok
test_not_modified (circus.tests.test_plugin_command_reloader.TestCommandReloader) ... ok
test_path_is_modified (circus.tests.test_plugin_command_reloader.TestCommandReloader) ... ok
test_above_max_retry_triggers_final_stop (circus.tests.test_plugin_flapping.TestFlapping) ... ok
test_below_max_retry_triggers_restart (circus.tests.test_plugin_flapping.TestFlapping) ... ok
test_beyond_window_resets_tries (circus.tests.test_plugin_flapping.TestFlapping) ... ok
test_default_config (circus.tests.test_plugin_flapping.TestFlapping) ... ok
test_minus_one_max_retry_triggers_restart (circus.tests.test_plugin_flapping.TestFlapping) ... ok
test_reap_message_calls_check (circus.tests.test_plugin_flapping.TestFlapping) ... ok
test_resource_watcher_max_cpu (circus.tests.test_plugin_resource_watcher.TestResourceWatcher) ... ok
test_resource_watcher_max_mem (circus.tests.test_plugin_resource_watcher.TestResourceWatcher) ... /Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py:455: ResourceWarning: unclosed file <_io.BufferedReader name=124>
self.results = {}
ok
test_resource_watcher_min_cpu (circus.tests.test_plugin_resource_watcher.TestResourceWatcher) ... /Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/multiprocessing/util.py:289: ResourceWarning: unclosed file <_io.FileIO name=124 mode='rb'>
_run_finalizers(0)
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/multiprocessing/util.py:289: ResourceWarning: unclosed file <_io.FileIO name=125 mode='wb'>
_run_finalizers(0)
ok
test_resource_watcher_min_mem (circus.tests.test_plugin_resource_watcher.TestResourceWatcher) ... ok
test_service_config_param_is_deprecated (circus.tests.test_plugin_resource_watcher.TestResourceWatcher) ... ok
test_watcher_config_param_is_required (circus.tests.test_plugin_resource_watcher.TestResourceWatcher) ... ok
test_full_stats (circus.tests.test_plugin_statsd.TestFullStats) ... /Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/threading.py:138: ResourceWarning: unclosed file <_io.BufferedReader name=124>
self._release_save = lock._release_save
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/threading.py:138: ResourceWarning: unclosed file <_io.FileIO name=126 mode='rb'>
self._release_save = lock._release_save
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/threading.py:138: ResourceWarning: unclosed file <_io.FileIO name=129 mode='wb'>
self._release_save = lock._release_save
ok
test_watchdog_discovery_found (circus.tests.test_plugin_watchdog.TestPluginWatchDog) ... FAIL
test_watchdog_discovery_not_found (circus.tests.test_plugin_watchdog.TestPluginWatchDog) ... /Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/concurrent/futures/_base.py:498: ResourceWarning: unclosed file <_io.BufferedReader name=119>
for waiter in self._waiters:
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/concurrent/futures/_base.py:498: ResourceWarning: unclosed file <_io.BufferedReader name=121>
for waiter in self._waiters:
ERROR
test_base (circus.tests.test_process.TestProcess) ... ok
test_comparison (circus.tests.test_process.TestProcess) ... ok
test_process_parameters (circus.tests.test_process.TestProcess) ... ok
test_rlimits (circus.tests.test_process.TestProcess) ... ok
test_streams (circus.tests.test_process.TestProcess) ... SKIP: Nose runs without -s
test_reload_addplugins (circus.tests.test_reloadconfig.TestConfig) ... ok
test_reload_addsockets (circus.tests.test_reloadconfig.TestConfig) ... ok
test_reload_addwatchers (circus.tests.test_reloadconfig.TestConfig) ... ok
test_reload_changeplugins (circus.tests.test_reloadconfig.TestConfig) ... ok
test_reload_changesockets (circus.tests.test_reloadconfig.TestConfig) ... ok
test_reload_changewatchers (circus.tests.test_reloadconfig.TestConfig) ... ok
test_reload_delplugins (circus.tests.test_reloadconfig.TestConfig) ... ok
test_reload_delsockets (circus.tests.test_reloadconfig.TestConfig) ... ok
test_reload_delwatchers (circus.tests.test_reloadconfig.TestConfig) ... ok
test_reload_envdictparsed (circus.tests.test_reloadconfig.TestConfig) ... ok
test_reload_ignorearbiterwatchers (circus.tests.test_reloadconfig.TestConfig) ... /Users/scottmax/Source/circus/circus/config.py:163: DeprecationWarning: You defined a stats_endpoint without setting up statsd to True.
DeprecationWarning)
ok
test_reload_numprocesses (circus.tests.test_reloadconfig.TestConfig) ... ok
test_watcher_names (circus.tests.test_reloadconfig.TestConfig) ... ok
test_dummy (circus.tests.test_runner.TestRunner) ... ERROR
test_handler (circus.tests.test_sighandler.TestSigHandler) ... ERROR
test_bind_to_interface (circus.tests.test_sockets.TestSockets) ... SKIP: SO_BINDTODEVICE unsupported
test_inet6 (circus.tests.test_sockets.TestSockets) ... ok
When no proto in the config, the default (0) is used. ... ok
test_load_from_config_umask (circus.tests.test_sockets.TestSockets) ... ok
Unknown proto in the config raises an error. ... ok
test_manager (circus.tests.test_sockets.TestSockets) ... ok
test_socket (circus.tests.test_sockets.TestSockets) ... ok
test_unix_cleanup (circus.tests.test_sockets.TestSockets) ... ok
test_unix_socket (circus.tests.test_sockets.TestSockets) ... ok
test_handler (circus.tests.test_stats_client.TestStatsClient) ... ERROR
test_collector_aggregation (circus.tests.test_stats_collector.TestCollector) ... ok
test_collector_aggregation_when_unknown_values (circus.tests.test_stats_collector.TestCollector) ... ok
test_socketstats (circus.tests.test_stats_collector.TestCollector) ... ok
test_watcherstats (circus.tests.test_stats_collector.TestCollector) ... ok
test_publish (circus.tests.test_stats_publisher.TestStatsPublisher) ... ok
test_publish_reraise_zmq_errors (circus.tests.test_stats_publisher.TestStatsPublisher) ... ok
test_publish_silent_zmq_errors_when_socket_closed (circus.tests.test_stats_publisher.TestStatsPublisher) ... ok
test_get_all_pids (circus.tests.test_stats_streamer.TestStatsStreamer) ... ok
test_get_circus_pids (circus.tests.test_stats_streamer.TestStatsStreamer) ... ok
test_get_pids (circus.tests.test_stats_streamer.TestStatsStreamer) ... ok
test_get_pids_circus (circus.tests.test_stats_streamer.TestStatsStreamer) ... ok
test_remove_pid (circus.tests.test_stats_streamer.TestStatsStreamer) ... ok
test_socketstats (circus.tests.test_stats_streamer.TestStatsStreamer) ... ok
test_color_selections (circus.tests.test_stream.TestFancyStdoutStream) ... ok
test_data_split_into_lines (circus.tests.test_stream.TestFancyStdoutStream) ... ok
test_data_with_extra_lines (circus.tests.test_stream.TestFancyStdoutStream) ... ok
test_random_colored_output (circus.tests.test_stream.TestFancyStdoutStream) ... ok
test_red_colored_output (circus.tests.test_stream.TestFancyStdoutStream) ... ok
test_time_formatting (circus.tests.test_stream.TestFancyStdoutStream) ... ok
test_data_split_into_lines (circus.tests.test_stream.TestFileStream) ... ok
test_data_with_extra_lines (circus.tests.test_stream.TestFileStream) ... ok
test_time_formatting (circus.tests.test_stream.TestFileStream) ... ok
test_file_stream (circus.tests.test_stream.TestWatcher) ... ERROR
test_stream (circus.tests.test_stream.TestWatcher) ... ERROR
test_bytes2human (circus.tests.test_util.TestUtil) ... ok
test_convert_opt (circus.tests.test_util.TestUtil) ... ok
test_get_info (circus.tests.test_util.TestUtil) ... ok
test_get_info_still_works_when_denied_access (circus.tests.test_util.TestUtil) ... /Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/mock-1.0.1-py3.3.egg/mock.py:1914: ResourceWarning: unclosed file <_io.FileIO name=206 mode='rb'>
/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/mock-1.0.1-py3.3.egg/mock.py:1914: ResourceWarning: unclosed file <_io.FileIO name=207 mode='wb'>
ok
test_get_python_version (circus.tests.test_util.TestUtil) ... ok
test_load_virtualenv (circus.tests.test_util.TestUtil) ... ok
test_negative_uid_gid (circus.tests.test_util.TestUtil) ... ok
test_parse_env_str (circus.tests.test_util.TestUtil) ... ok
test_replace_gnu_args (circus.tests.test_util.TestUtil) ... ok
test_to_uid (circus.tests.test_util.TestUtil) ... ok
test_to_uidgid (circus.tests.test_util.TestUtil) ... ok
test_tobool (circus.tests.test_util.TestUtil) ... ok
test_working_dir_return_pwd_when_paths_are_equals (circus.tests.test_util.TestUtil) ... ok
test_hooks (circus.tests.test_validate_option.TestValidateOption) ... ok
test_stderr_stream (circus.tests.test_validate_option.TestValidateOption) ... ok
test_stdout_stream (circus.tests.test_validate_option.TestValidateOption) ... ok
test_uidgid (circus.tests.test_validate_option.TestValidateOption) ... ok
test_not_respawning (circus.tests.test_watcher.RespawnTest) ... ERROR
test_stopping_a_watcher_doesnt_spawn (circus.tests.test_watcher.RespawnTest) ... ok
test_arbiter_reference (circus.tests.test_watcher.TestWatcher) ... ERROR
test_decr_too_much (circus.tests.test_watcher.TestWatcher) ... ERROR
test_max_age (circus.tests.test_watcher.TestWatcher) ... ERROR
test_signal (circus.tests.test_watcher.TestWatcher) ... ERROR
test_stats (circus.tests.test_watcher.TestWatcher) ... ERROR
test_unexisting (circus.tests.test_watcher.TestWatcher) ... ERROR
test_after_signal (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_after_start (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_after_start_fails (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_after_start_false (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_after_stop (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_after_stop_fails (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_after_stop_false (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_before_signal (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_before_spawn (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_before_spawn_failure (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_before_spawn_false (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_before_start (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_before_start_fails (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_before_start_false (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_before_stop (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_before_stop_fails (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_before_stop_false (circus.tests.test_watcher.TestWatcherHooks) ... ERROR
test_missing_hook (circus.tests.test_watcher.TestWatcherHooks) ... ok
test_copy_env (circus.tests.test_watcher.TestWatcherInitialization) ... ok
test_copy_path (circus.tests.test_watcher.TestWatcherInitialization) ... ok
test_hook_in_PYTHON_PATH (circus.tests.test_watcher.TestWatcherInitialization) ... ok
test_venv (circus.tests.test_watcher.TestWatcherInitialization) ... /Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py:389: ResourceWarning: unclosed file <_io.BufferedReader name=300>
self.io_loop.add_future(self.future, runner.result_callback(self.key))
ok
test_venv_site_packages (circus.tests.test_watcher.TestWatcherInitialization) ... ok

ERROR: test_watchdog_discovery_not_found (circus.tests.test_plugin_watchdog.TestPluginWatchDog)

Traceback (most recent call last):
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/testing.py", line 427, in wrapper
functools.partial(f, self), timeout=timeout)
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/ioloop.py", line 370, in run_sync
return future_cell[0].result()
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/concurrent.py", line 129, in result
raise_exc_info(self.__exc_info)
File "", line 3, in raise_exc_info
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/stack_context.py", line 302, in wrapped
ret = fn(_args, *_kwargs)
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 550, in inner
self.set_result(key, result)
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 476, in set_result
self.run()
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 505, in run
yielded = self.gen.throw(_exc_info)
File "/Users/scottmax/Source/circus/circus/tests/test_plugin_watchdog.py", line 58, in test_watchdog_discovery_not_found
yield self.start_arbiter(fqn)
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 496, in run
next = self.yield_point.get_result()
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 395, in get_result
return self.runner.pop_result(self.key).result()
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/concurrent.py", line 129, in result
raise_exc_info(self.__exc_info)
File "", line 3, in raise_exc_info
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/stack_context.py", line 302, in wrapped
ret = fn(_args, *_kwargs)
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 550, in inner
self.set_result(key, result)
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 476, in set_result
self.run()
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 505, in run
yielded = self.gen.throw(_exc_info)
File "/Users/scottmax/Source/circus/circus/tests/support.py", line 128, in start_arbiter
yield self.arbiter.start()
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 496, in run
next = self.yield_point.get_result()
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 395, in get_result
return self.runner.pop_result(self.key).result()
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/concurrent.py", line 129, in result
raise_exc_info(self.__exc_info)
File "", line 3, in raise_exc_info
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 221, in wrapper
runner.run()
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 507, in run
yielded = self.gen.send(next)
File "/Users/scottmax/Source/circus/circus/arbiter.py", line 478, in start
self.initialize()
File "/Users/scottmax/Source/circus/circus/util.py", line 326, in _log
return func(self, _args, *_kw)
File "/Users/scottmax/Source/circus/circus/arbiter.py", line 443, in initialize
self.evpub_socket.bind(self.pubsub_endpoint)
File "socket.pyx", line 432, in zmq.core.socket.Socket.bind (zmq/core/socket.c:4022)
File "checkrc.pxd", line 21, in zmq.core.checkrc._check_rc (zmq/core/socket.c:5838)
nose.proxy.ZMQError: Address already in use
-------------------- >> begin captured logging << --------------------
circus: INFO: Registering signals...
circus: INFO: Starting master on pid 33578
--------------------- >> end captured logging << ---------------------

FAIL: test_watchdog_discovery_found (circus.tests.test_plugin_watchdog.TestPluginWatchDog)

Traceback (most recent call last):
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/testing.py", line 427, in wrapper
functools.partial(f, self), timeout=timeout)
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/ioloop.py", line 370, in run_sync
return future_cell[0].result()
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/concurrent.py", line 129, in result
raise_exc_info(self.__exc_info)
File "", line 3, in raise_exc_info
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/stack_context.py", line 302, in wrapped
ret = fn(_args, *_kwargs)
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 550, in inner
self.set_result(key, result)
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 476, in set_result
self.run()
File "/Users/scottmax/Source/circus/env33/lib/python3.3/site-packages/tornado-3.1.1-py3.3.egg/tornado/gen.py", line 507, in run
yielded = self.gen.send(next)
File "/Users/scottmax/Source/circus/circus/tests/test_plugin_watchdog.py", line 53, in test_watchdog_discovery_found
self.assertEqual(len(pid_status), 1, pid_status)
nose.proxy.AssertionError: 0 != 1 : {}
-------------------- >> begin captured logging << --------------------
circus: INFO: Registering signals...
circus: INFO: Starting master on pid 33578
circus: DEBUG: Initializing watchers
circus: DEBUG: cmd: /Users/scottmax/Source/circus/env33/bin/python
circus: DEBUG: args: ['generic.py', 'circus.tests.test_plugin_watchdog.run_dummy_watchdogged', '/var/folders/vw/x8p8n3vn0hv8sp0mnkb8xg7r0000gn/T/tmpg9vh5g']
circus: DEBUG: process args: ['/Users/scottmax/Source/circus/env33/bin/python', 'generic.py', 'circus.tests.test_plugin_watchdog.run_dummy_watchdogged', '/var/folders/vw/x8p8n3vn0hv8sp0mnkb8xg7r0000gn/T/tmpg9vh5g']
circus: DEBUG: running test process [pid 33725]
circus: INFO: test started
circus: INFO: Arbiter now waiting for commands
circus: DEBUG: got message b'{"properties":{},"command":"list"}'
circus: DEBUG: got message b'{"properties":{"name":"test"},"command":"list"}'
circus: DEBUG: here is the status of the processes []
--------------------- >> end captured logging << ---------------------

[scottkmaxwell]

Every error is "Address already in use" except for test_watchdog_discovery_found.

[thefab]

Is it Py3 specific (no errors at all here (osx with Py2.7) ?

[scottkmaxwell]

I'll check.

[scottkmaxwell]

Same result with Py2.7. This is running what is on master right now.

[scottkmaxwell]

Got the 33 errors and 1 failure this time. Seems to be pretty reliably either 30 or 33 errors and 1 failure every time.

[scottkmaxwell]

Just got a clean run with Py2.6

[scottkmaxwell]

Py2.6 ran clean twice. Switched back to Py2.7 and failed again. Have not gotten any clean runs with Py2.7 or Py3.3.

[scottkmaxwell]

And same failure on Py3.2

[thefab]

Can you skip test_watchdog_discovery_found and test another time ?

[scottkmaxwell]

Did that and got failures from test_plugin_resource_watcher. Commented out TestResourceWatcher and all ran without error.

[scottkmaxwell]

That is on 3.3 and 2.7

[thefab]

Yes ! I reproduce your issue with test_watchdog_discovery_found on Py3.3 (only).

Working on it.

[scottkmaxwell]

In TestResourceWatcher the problem is test_resource_watcher_min_cpu. If I comment out that test, everything works. Even with test_watchdog_discovery_found turned back on! Strange.

[scottkmaxwell]

I am also on OS X, BTW. 10.9.

[thefab]

@scottkmaxwell all fixed for me (Py33 and Py27) with #607, can you test ?

[scottkmaxwell]

Looks good on Py2.7. Will try on Py3.3 as well.

[scottkmaxwell]

Py3.3 looks good too!

[thefab]

Great !

travis report was good too:

https://travis-ci.org/mozilla-services/circus/builds/13059740

Just pushed another commit on this branch to fix latest errors ?