Skip to content

merge "base.problems" with "env_kwargs.instance_ids" #178

Open
@sordonia

Description

@sordonia

when running scripts/run.py, one should be able to select a subset of tasks to run on.

currently the envs accept and instance_ids argument that is settable with -p base.env_kwargs.instance_ids=

but this won't work given that the agent loops over problems = 'all' given that -p base.problems is not set.

i think there's an error in get_dataset_split in the envs; when the SweSmith env is initialized with instance_ids != None, how should get_dataset_split behave? right now it returns the list of all the tasks, irrespective of the state of the environment, fixing that would fix the problem.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions