Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add serialization for the new components #349

Merged
merged 27 commits into from
Feb 23, 2024
Merged

Add serialization for the new components #349

merged 27 commits into from
Feb 23, 2024

Conversation

plaguss
Copy link
Contributor

@plaguss plaguss commented Feb 20, 2024

Description

This PR adds methods to serialize to the Step, DAG and BasePipeline classes.

It includes a new test/integration, for the moment just with a simple pipeline without LLMs to check the behaviour with something more complex than the unit tests, but we can remove those.

@plaguss plaguss added the enhancement New feature or request label Feb 20, 2024
@plaguss plaguss self-assigned this Feb 20, 2024
Copy link
Member

@alvarobartt alvarobartt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, and AFAIK you didn't touch any of the previously existing files which is nice because we may do a cleanup so that we remove the old stuff from core-refactor to avoid having code that's not necessary anymore!

src/distilabel/utils/serialization_v2.py Outdated Show resolved Hide resolved
tests/pipeline/test_base.py Show resolved Hide resolved
@plaguss plaguss marked this pull request as ready for review February 23, 2024 10:03
Copy link
Member

@alvarobartt alvarobartt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I've left some comments to tackle before merging! Great work 💪🏻

src/distilabel/pipeline/_dag.py Show resolved Hide resolved
src/distilabel/pipeline/_dag.py Outdated Show resolved Hide resolved
src/distilabel/utils/serialization_v2.py Outdated Show resolved Hide resolved
src/distilabel/utils/serialization_v2.py Outdated Show resolved Hide resolved
tests/integration/test_pipe_llms.py Outdated Show resolved Hide resolved
tests/integration/test_pipe_llms.py Show resolved Hide resolved
tests/pipeline/llm/test_serialization.py Show resolved Hide resolved
Co-authored-by: Alvaro Bartolome <alvaro@argilla.io>
@alvarobartt alvarobartt merged commit 66ce10b into core-refactor Feb 23, 2024
0 of 4 checks passed
@alvarobartt alvarobartt deleted the serde branch February 23, 2024 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request refactor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants