Skip to content

cleanups: todo after implementation of pipeline file #3693

@skshetry

Description

@skshetry

Tracking things to cleanup after #3676 is merged, in addition to suggestions in the PR:

  • Remove tags (will make life simpler) tag: getting rid of it #3699
  • Split Repo.collect() for pipelines usage and stage gathering:
    Data related commands don't care about orderings of pipeline stages, whereas for pipelines,
    things are more complex than collect() can handle.
    This will also remove duplicated warnings currently on reproduce.
    Workaround on tag: getting rid of it #3699 for now
  • Why is there schema/serialization/deserialization in 3 places? Yeah, true.
  • Parallel logics of loading/transforming/dumping of Output, Dependency and Stages for single stage and pipeline stages (Requires further investigation)
  • StageLoader is too dynamic.
  • Messed with OutputDuplicationsError: Make sure it's safe to remove current stage from error message.
  • (Feature Request) Some way to remove stage from both pipelines.lock and pipelines.yaml? (Implement removing of the stage #3881)
  • Renaming utils/stage.py and functions inside it. (refactor: rename utils.stage to utils.yaml #3923)
  • Make dump() ordered (Only for Python==3.5).
  • StageParams and OutputParams

Of course, we need lots of tests and UI issues fixes, but this meta issue is only regarding code cleanups.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions