Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send pipeline names in benchmark arguments #466

Closed
sarahmish opened this issue Oct 6, 2023 · 0 comments · Fixed by #468 or #470
Closed

Send pipeline names in benchmark arguments #466

sarahmish opened this issue Oct 6, 2023 · 0 comments · Fixed by #468 or #470
Assignees
Labels
enhancement Improvements on the current features
Milestone

Comments

@sarahmish
Copy link
Collaborator

PR #465 introduced a new feature to the benchmark which allows it to be run from the command line with arguments. The argument pipelines accepts a list of pipeline names but we would like it to also allow a dictionary of pipelines like in the original proposal.

A possible issue that might arise when keeping the current state is that loading the hyperparameters per dataset will be impacted. To prevent this issue, we can have two solutions:

  • Parse arguments as dictionaries to have a clear pipeline name.
  • Augment the pipeline names in a "fixed" fashion, e.g. pipeline_name = 'tadgan' if pipeline == 'tadgan_gpu'.
  • Have a new parameter in the JSON file to give an alias name for the pipeline.
@sarahmish sarahmish added the enhancement Improvements on the current features label Oct 6, 2023
@sarahmish sarahmish added this to the 0.5.2 milestone Oct 12, 2023
@sarahmish sarahmish self-assigned this Oct 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improvements on the current features
Projects
None yet
1 participant