ipfs add --recursive panics on conversion: is cmdkit.Error, not *coreunix.AddedObject

[come-maiz]

ipfs version --all:

$ ipfs version --all
go-ipfs version: 0.4.14-dev-f3f0179
Repo version: 6
System version: amd64/linux
Golang version: go1.9.2

Type:

Critical - System crash, application panic.

Description:

I am trying to add a full Ubuntu mirror to ipfs. The mirror is more than 1TB of data. When I run the ipfs add --recursive "${dir}" command, it eventually fails with:

panic: interface conversion: interface {} is cmdkit.Error, not *coreunix.AddedObject

goroutine 37 [running]:
github.com/ipfs/go-ipfs/core/commands.glob..func7.1(0xc4202ec060)
        /cwd/parts/ipfs/go/src/github.com/ipfs/go-ipfs/core/commands/add.go:390 +0x9fc
created by github.com/ipfs/go-ipfs/core/commands.glob..func7.2
        /cwd/parts/ipfs/go/src/github.com/ipfs/go-ipfs/core/commands/add.go:449 +0xc7

This is the full script I'm running:
https://raw.githubusercontent.com/JaquerEspeis/apt-transport-ipfs/master/scripts/sync_mirror.sh

[kevina]

This looks like it could be an easy fix, at least to avoid the crash.

[ghost]

I don't mind the crash (well I do but that's not the point), but I do mind that it hides errors :) I was encountering this the other day and had to jump through a few hoops to find out that my repo parition had turned read-only.

[kevina]

@lgierth are you using flatfs or badger?

[ghost]

The error itself was unrelated to go-ipfs, I had just messed with my filesystem mounts.

But it was just giving me the above panic so I had to dig to get to the real error.

[kevina]

@lgierth I am not following. Is your issue related to #4528

[come-maiz]

In my case, it seems that there was something wrong with the rsync of some files. It could be that my harddrive is old and probably dying too.

I found one of the files that causes the crash and tried to add it:

$ ipfs add /media/elopio/f22f4d36-2ea8-4320-9ebc-82ea8f3c7ea7/ubuntu-mirror/dists/artful-updates/main/installer-amd64/20101020ubuntu523.1/images/netboot/boot.img.gz 
Error: Post http://127.0.0.1:5001/api/v0/add?encoding=json&progress=true&stream-channels=true: read /media/elopio/f22f4d36-2ea8-4320-9ebc-82ea8f3c7ea7/ubuntu-mirror/dists/artful-updates/main/installer-amd64/20101020ubuntu523.1/images/netboot/boot.img.gz: input/output error

I downloaded it again from the mirror:

$ wget http://archive.ubuntu.com/ubuntu/dists/artful-updates/main/installer-amd64/20101020ubuntu523.1/images/netboot/boot.img.gz
$ mv boot.img.gz /media/elopio/f22f4d36-2ea8-4320-9ebc-82ea8f3c7ea7/ubuntu-mirror/dists/artful-updates/main/installer-amd64/20101020ubuntu523.1/images/netboot/boot.img.gz

And now ipfs add worked:

$ ipfs add /media/elopio/f22f4d36-2ea8-4320-9ebc-82ea8f3c7ea7/ubuntu-mirror/dists/artful-updates/main/installer-amd64/20101020ubuntu523.1/images/netboot/boot.img.gz 
added QmWScrn1bxnteGaUpMYQppvY3f3ko2BZ46ZC8xeuXMbfMa boot.img.gz

So yeah, the crash shouldn't happen, it should instead report that there's something wrong with the file. Ideally, something more useful than input/output error.

[ghost]

@lgierth I am not following. Is your issue related to #4528

I don't know, but probably. There's no particular reason I mentioned that issue -- I just wanted to add my 👍 to this issue here by saying it's not just a panic, it also hides the real errors :)

[Stebalien]

The commands library has been rewritten since this was filed.