Open
Description
when running scripts/run.py, one should be able to select a subset of tasks to run on.
currently the envs accept and instance_ids argument that is settable with -p base.env_kwargs.instance_ids=
but this won't work given that the agent loops over problems = 'all' given that -p base.problems
is not set.
i think there's an error in get_dataset_split
in the envs; when the SweSmith env is initialized with instance_ids != None, how should get_dataset_split behave? right now it returns the list of all the tasks, irrespective of the state of the environment, fixing that would fix the problem.