Skip to content

pull: KeyError crash (or silently skipped target) when mixing .dvc-file targets with granular paths inside a tracked directory #11075

Description

@sfgartland

Bug Report

Description

dvc pull with a target list that mixes a .dvc-file target with granular paths inside a tracked directory misbehaves in two ways, depending on workspace state:

  1. Crash: if the tracked directory contains any untracked (drifted) file, pull dies with ERROR: unexpected error - ('datadir', 'f1.txt') — an uncaught KeyError from checkout's _check_can_delete.
  2. Silent skip: in a clean fresh-clone state (no workspace, no cache), the same command succeeds (exit 0) but only materializes the .dvc-file target; the granular target is never checked out.

Both target forms are documented as supported in the pull command reference ("paths to tracked files or directories (including paths inside tracked directories), .dvc files, and stage names"), and each form works correctly on its own — including a single invocation whose targets are all data paths, which is the workaround.

Reproduce

#!/usr/bin/env bash
set -uo pipefail
cd "$(mktemp -d)"
mkdir remote repo && cd repo
git init -q && git config user.email r@e.pro && git config user.name repro
dvc init -q
dvc remote add -d localremote ../remote
mkdir datadir
printf one > datadir/f1.txt; printf two > datadir/f2.txt; printf three > datadir/f3.txt
printf single > single.csv
dvc add datadir single.csv
git add -A && git commit -qm init
dvc push -q

echo "=== Variant 1: fresh-clone state -> granular target silently skipped ==="
rm -rf datadir single.csv .dvc/cache
dvc pull single.csv.dvc datadir/f1.txt        # exit 0, "1 file fetched and 1 file added"
ls datadir/f1.txt 2>/dev/null || echo "BUG: datadir/f1.txt was not checked out"

dvc pull -q   # restore everything

echo "=== Variant 2: one untracked file inside the tracked dir -> crash ==="
printf extra > datadir/extra-drift.txt
dvc pull single.csv.dvc datadir/f1.txt        # ERROR: unexpected error - ('datadir', 'f1.txt')

echo "=== Variant 3 (control): same drift, data-path targets only -> works ==="
dvc pull single.csv datadir/f1.txt            # "Everything is up to date."

Expected

Either the mixed target list works (both targets checked out, like variant 3), or a proper error message — not an internal KeyError and not a silent partial result.

Traceback (variant 2)

Traceback (most recent call last):
  ...
  File ".../dvc/repo/pull.py", line 42, in pull
  File ".../dvc/repo/checkout.py", line 160, in checkout
    _check_can_delete(diff.files_delete, new, self.root_dir, self.fs)
  File ".../dvc/repo/checkout.py", line 91, in _check_can_delete
    cache_fs, cache_path = index.storage_map.get_cache(entry)
  File ".../dvc_data/index/index.py", line 519, in get_cache
  File ".../dvc_data/index/index.py", line 508, in get_storage
    info = self[entry.key]
  File ".../dvc_data/index/index.py", line 446, in __getitem__
KeyError: ('datadir', 'f1.txt')

Analysis

_check_can_delete anticipates the "entry known but no cache storage" case — it wraps the lookup in try/except ValueError (catching StorageKeyError). But when targets span different artifacts, the filtered index views seem to produce files_delete entries whose keys are entirely absent from the (filtered) storage_map, so StorageMapping.__getitem__ raises a bare KeyError, which escapes to the generic "unexpected error" handler.

Environment

DVC version: 3.67.1 (pip)
-------------------------
Platform: Python 3.12.12 on Linux-7.0.12-101.fc43.x86_64-x86_64-with-glibc2.42
Subprojects:
        dvc_data = 3.18.3
        dvc_objects = 5.2.0
        dvc_render = 1.0.2
        dvc_task = 0.40.2
        scmrepo = 3.6.2
Supports: http, https, s3 (s3fs = 2026.4.0)

Originally hit with an S3 (Cloudflare R2) remote pulling two standalone-file artifacts plus 10 granular paths inside a 70-file directory artifact; the local-remote repro above shows the remote type is irrelevant.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions