get/import: could not perform a HEAD request

[jorgeorpinel]

DVC version: 0.62.1
Python version: 3.7.3
Platform: Darwin-18.7.0-x86_64-i386-64bit
Binary: False
Cache: reflink - True, hardlink - True, symlink - True
Filesystem type (cache directory): ('apfs', '/dev/disk1s1')
Filesystem type (workspace): ('apfs', '/dev/disk1s1')

I'm trying to import a directory versioned in our own dataset registry project into an empty, non-Git DVC project, but getting this cryptic error:

$ dvc import --rev 0547f58 \                               
           git@github.com:iterative/dataset-registry.git \
           use-cases/data
Importing 'use-cases/data (git@github.com:iterative/dataset-registry.git)' -> 'data'
ERROR: failed to import 'use-cases/data' from 'git@github.com:iterative/dataset-registry.git'. - unable to find DVC-file with output '../../../../private/var/folders/_c/3mt_xn_d4xl2ddsx2m98h_r40000gn/T/tmphs83czecdvc-repo/use-cases/data'

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

The directory in question has file name b6923e1e4ad16ea1a7e2b328842d56a2.dir (See use-cases/cats-dogs.dvc of that version). And the default remote is [configured[(https://github.com/iterative/dataset-registry/blob/master/.dvc/config) to https://remote.dvc.org/dataset-registry (which is an HTTP redirect to the s3://dvc-public/remote/dataset-registry bucket). The file seems to be in the remote

Am I just doing something wrong here (hopefully), or is dvc import broken?

[jorgeorpinel]

p.s. I've also tried without --rev and get the same error (different output path).

[shcheklein]

@jorgeorpinel should it be use-cases/cats-dogs?

[jorgeorpinel]

🤦‍♂ Oops. I forgot I changed the directory name from data (original name in the ZIP files used in the Versioning tutorial). But I still can't get it with the correct path:

$ dvc import --rev 0547f58 \
           git@github.com:iterative/dataset-registry.git \
           use-cases/cats-dogs
Importing 'use-cases/cats-dogs (git@github.com:iterative/dataset-registry.git)' -> 'cats-dogs'
WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:                                                             
name: ../../../../private/var/folders/_c/3mt_xn_d4xl2ddsx2m98h_r40000gn/T/tmpfnwm64lqdvc-repo/use-cases/cats-dogs, md5: b6923e1e4ad16ea1a7e2b328842d56a2.dir
Missing cache for directory '../../../../private/var/folders/_c/3mt_xn_d4xl2ddsx2m98h_r40000gn/T/tmpfnwm64lqdvc-repo/use-cases/cats-dogs'. Cache for files inside will be lost. Would you like to continue? Use '-f' to force. [y/n] y
WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:                                                             
name: ../../../../private/var/folders/_c/3mt_xn_d4xl2ddsx2m98h_r40000gn/T/tmpfnwm64lqdvc-repo/use-cases/cats-dogs, md5: b6923e1e4ad16ea1a7e2b328842d56a2.dir
WARNING: Cache 'b6923e1e4ad16ea1a7e2b328842d56a2.dir' not found. File 'cats-dogs' won't be created.                                                           
ERROR: failed to import 'use-cases/cats-dogs' from 'git@github.com:iterative/dataset-registry.git'. - output 'cats-dogs' does not exist

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

With the additional issue of really long and cryptic messages, as well as a prompt I don't understand and just say y. (Both issues already reported in #2599.)

What does the "output 'cats-dogs' does not exist" error mean?

[jorgeorpinel]

Hmmmm... Apparently I didn't push that version of the cats-dogs dir in the dataset registry project. It's not in the remote. I'll have to fix that first... I guess this issue is invalid then, fortunately! The messaging here is still pretty confusing though, should I open another issue about this?

[shcheklein]

@jorgeorpinel yes, please open a new UI issue!

[jorgeorpinel]

Done! #2602

[jorgeorpinel]

The directory in question has file name b6923e1e4ad16ea1a7e2b328842d56a2.dir (See use-cases/cats-dogs.dvc of that version).

So, I pushed the data to the remote now and checked that it actually exists on S3:

$ aws s3 ls s3://dvc-public/remote/dataset-registry/b6/
2019-10-05 01:51:13       6388 2f5c18d1af468fd41c979873a8404b
2019-10-05 01:51:41      22202 4ced1e881cc37c0e0673bafe6e789c
2019-10-12 19:10:03     161184 923e1e4ad16ea1a7e2b328842d56a2.dir  <-- Bingo
2019-10-05 01:50:56      17450 efd10ab38ff17fa593e3b102d088ac

However, I try to import it (into the same empty non-Git DVC project) and, although the progress bar runs for a while up to around 90%, the progress bar suddenly disappears and I get:

$ dvc import --rev 0547f58 \
           git@github.com:iterative/dataset-registry.git \
           use-cases/cats-dogs
Importing 'use-cases/cats-dogs (git@github.com:iterative/dataset-registry.git)' -> 'cats-dogs'
ERROR: failed to import 'use-cases/cats-dogs' from 'git@github.com:iterative/dataset-registry.git'. - could not perform a HEAD request                         

And nothing is downloaded. I've tried several times. My Internet connection is fine:

Expand for SpeedTest screen capture

https://www.speedtest.net/result/8671087724

Is there a single file missing or something? How do I find it? I've tried dvc push from the source project again and it states Everything is up to date. (Investigated in following #2600 (comment))