Launcher should support initial_job_num

[omry]

Sweeper plugins may call a launcher multiple times, this means the launcher can no longer know the hydra.job.num without preserving state (which is a bad idea).

Change Launcher launch API to:

def launch(self, job_overrides, initial_idx):

For the BasicSweeper, the change amounts to:

        initial_job_idx = 0
        while not self.is_done():
            batch = self.get_job_batch()
            results = self.launcher.launch(batch, initial_job_idx=initial_job_idx)
            initial_job_idx += len(batch)
            returns.append(results)
        return returns

specifically, see how initial_job_idx is passed to the launcher in each batch, and hen being advanced by the batch size.

[omry]

@jrapin, this is landing soon.
It solve a problem you may have missed with sweeper plugins that have multiple batches sent to the launcher, where the output directories of the jobs from each batch would overwrite from the previous batch.

It's a pretty easy fix.
there are also some new generic sweeper tests that you may want to try to include.
In particular, see

class BatchedSweeperTestSuite:
    def test_sweep_2_jobs_2_batches(): ...

This function launches 6 jobs and expectes the sweeper to split them into two batches of 3.
See the example sweeper in the plugin to see how to use it. I am not sure it will be easy to integrate with your sweeper but it would be good to try.

[omry]

@shagunsodhani , I fixed the Ax sweeper to conform but please try to enable the test. (there is a TODO in the plugin test).