Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exp run: showing unhelpful error message for invalid pipeline #8974

Open
dberenbaum opened this issue Feb 6, 2023 · 1 comment
Open

exp run: showing unhelpful error message for invalid pipeline #8974

dberenbaum opened this issue Feb 6, 2023 · 1 comment
Labels
A: experiments Related to dvc exp bug Did we break something? p2-medium Medium priority, should be done, but less important

Comments

@dberenbaum
Copy link
Contributor

Background in #8973.

Here is the setup from that issue:

# train.py
from random import random

from dvclive import Live

NUM_EPOCHS = 10


with Live(save_dvc_exp=True) as live:

    live.log_param("NUM_EPOCHS", NUM_EPOCHS)

    for epoch in range(NUM_EPOCHS):
        step = live.read_step()
        live.log_metric("metric", random())
        live.log_param("my_step", step)
        live.next_step()
# dvc.yaml
stages:
  train:
    cmd: python train.py
    deps:
      - train.py
    outs:
      - dvclive:
          checkpoint: true
    metrics:
      - dvclive/metrics.json
    plots:
      - dvclive/plots

When running dvc exp run, the error message looks like:

216764996-eb2efe78-b5d0-4c47-b79e-1053280e322f

Expected

dvc repro for a similar setup gives an error like:

ERROR: The output paths:
'dvclive'('train')
'dvclive/metrics.json'('train')
overlap and are thus in the same tracked directory.
To keep reproducibility, outputs should be in separate tracked directories or tracked individually.

The same error from dvc repro should be shown in dvc exp run.

@dberenbaum dberenbaum added bug Did we break something? p2-medium Medium priority, should be done, but less important A: experiments Related to dvc exp labels Feb 6, 2023
@dberenbaum
Copy link
Contributor Author

@skshetry This is another issue to keep in mind re: #9370

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: experiments Related to dvc exp bug Did we break something? p2-medium Medium priority, should be done, but less important
Projects
None yet
Development

No branches or pull requests

1 participant