[Feature Request] Automatically only launch tasks during certain hours.

[JackUrb]

Overview

Anecdotally, crowdsourcing quality is often better on certain hours of the day. It would be nice to have some method to only launch tasks during these hours.

Implementation

In either case, the core functionality for this lives in the TaskLauncher.

Simple - for use with max_num_concurrent_units

This implementation is simple, we just don't launch additional tasks outside the specified hours. We can do this by sleeping the additional-launching thread outside of working hours. It could go here:

Mephisto/mephisto/operations/task_launcher.py

Line 179 in bc3526f

time.sleep(UNIT_GENERATOR_WAIT_SECONDS)

If we're outside of set hours, sleep until the next approved hour.

Complex - actively take down tasks outside hours

This implementation is less clear, but may be desired. The goal here would be to expire all of the units that are still alive outside of working hours, and then relaunch them later. This would likely involve creating new assignments that are copies of incomplete ones, and then relaunching those. If this is necessary, can comment and I'll scope further.

[ajyl]

What would be a good way to add tests for this functionality?

[JackUrb]

Hi @ajyl: Your best bet would be to mock time.time, and work through a test similar to the tests in test_operator.py, most likely this one:

Mephisto/test/core/test_operator.py

Line 229 in a6eed61

def test_run_jobs_with_restrictions(self):

You could first launch a task with 1 concurrent unit max, then update the time, complete the unit, note that a new worker is provided with no possible units, update the time again, and then note that a new unit is launched and able to be completed.