Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor getting parameters from Step.process method #355

Merged
merged 1 commit into from
Feb 21, 2024

Conversation

gabrielmbmb
Copy link
Member

Description

This PR refactors the DAG and the Step, to delegate the responsibility of getting the process parameters using the inspect module from the DAG to the Step. This makes the code more clear and will allow applying patterns like the one below:

from distilabel.pipeline.step.base import Step

class MyCustomStep(Step):
    ...
    def process(self, inputs: StepInput, runtime_arg1: int, runtime_arg2: int, **kwargs: Any) -> List[Dict[str, Any]]:
        ...
    
    @property
    def runtime_parameters_names(self) -> Dict[str, bool]:
        runtime_parameters_names = super().runtime_parameters_names()
        
        # required additional runtime parameter
        runtime_parameters_names["my_additional_runtime_parameter"] = False
        return runtime_parameters_names

@gabrielmbmb gabrielmbmb added enhancement New feature or request refactor labels Feb 21, 2024
@gabrielmbmb gabrielmbmb added this to the 1.0.0 milestone Feb 21, 2024
@gabrielmbmb gabrielmbmb self-assigned this Feb 21, 2024
@gabrielmbmb gabrielmbmb merged commit b96eb5a into core-refactor Feb 21, 2024
4 checks passed
@gabrielmbmb gabrielmbmb deleted the refactor_getting_step_parameters branch February 21, 2024 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request refactor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant