zfs send -R fails, but subsequent -I succeeds

[rincebrain]

System information

Type	Version/Name
Distribution Name	Debian
Distribution Version	9.4
Linux Kernel	4.9.0-6-amd64
Architecture	amd64
ZFS Version	0.7.8
SPL Version	0.7.8

Describe the problem you're observing

Wanted to test migrating a dataset over to using checksum=edonr, so I set up a dataset dst/testbed with checksum=edonr on a pool with feature@edonr=enabled, and then ran:
zfs send -Re source/dataset@lastsnap | zfs recv -v -x checksum -e dst/testbed

It gets through the oldest snapshot, "one-and-done", then apparently while transferring the next snapshot, "zfs-auto-snap_weekly-2016-11-06-1147", it dies with:

internal error: Invalid argument
Aborted

At that point, zfs list -t all -r dst/testbed says:

NAME                                                             USED  AVAIL  REFER  MOUNTPOINT
dst/testbed                                                23.5G  2.94T    96K  /dst/testbed
dst/testbed/3ds_sd                                         23.5G  2.94T  23.5G  /dst/testbed/3ds_sd
dst/testbed/3ds_sd@one-and-done                               0B      -  23.5G  -

Strangely, if I then run
zfs send -Re -I dst/testbed/3ds_sd@{one-and-done,lastsnap} | zfs recv -v -x checksum -e dst/testbed

or just
zfs send -Re -i dst/testbed/3ds_sd@{one-and-done,zfs-auto-snap_weekly-2016-11-06-1147} | zfs recv -v -x checksum -e dst/testbed

it will progress to completion unremarkably.

Removing the -e flag from send doesn't affect the initial failure.

I tried using -s to make an easily resumable failure, but there's no receive-resume-token generated when receive aborts. Oh well.

[rincebrain]

Not entirely surprisingly, both 0.7.9 and git tip (as of 1a5b96b) as receivers will die identically.

[rincebrain]

This is as stupid as I thought it was - it's trying to report it can't set the checksum property on the dataset and dies with an internal error when trying to do that.

(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007f6ee2be942a in __GI_abort () at abort.c:89
#2  0x00007f6ee37c5292 in zfs_verror (hdl=0x55d23fcea060, error=2072, fmt=<optimized out>, ap=<optimized out>) at libzfs_util.c:306
#3  0x00007f6ee37c58b7 in zfs_standard_error_fmt (hdl=0x55d23fcea060, error=22, fmt=fmt@entry=0x7f6ee37cfc2a "%s") at libzfs_util.c:434
#4  0x00007f6ee37c58e1 in zfs_standard_error (hdl=<optimized out>, error=<optimized out>, msg=<optimized out>) at libzfs_util.c:370
#5  0x00007f6ee37bf6e5 in zfs_receive_one (hdl=hdl@entry=0x55d23fcea060, infd=infd@entry=0, tosnap=tosnap@entry=0x7ffeea2baeae "dumbpool", originsnap=originsnap@entry=0x0,
    flags=flags@entry=0x7ffeea2b97a0, drr=drr@entry=0x7ffeea2b7930, drr_noswap=0x7ffeea2b7a70, sendfs=0x7ffeea2b8cf0 "ephemeral/sorted/backups/3ds_sd", stream_nv=0x55d23fceaee0,
    stream_avl=0x55d23fcfcca0, top_zfs=0x7ffeea2b9690, cleanup_fd=7, action_handlep=0x7ffeea2b9698, finalsnap=0x7ffeea2b8d10 "pre_forced_check", cmdprops=0x55d23fceae80) at libzfs_sendrecv.c:3651
#6  0x00007f6ee37c1bea in zfs_receive_impl (hdl=hdl@entry=0x55d23fcea060, tosnap=tosnap@entry=0x7ffeea2baeae "dumbpool", originsnap=originsnap@entry=0x0, flags=flags@entry=0x7ffeea2b97a0,
    infd=infd@entry=0, sendfs=sendfs@entry=0x7ffeea2b8cf0 "ephemeral/sorted/backups/3ds_sd", stream_nv=0x55d23fceaee0, stream_avl=0x55d23fcfcca0, top_zfs=0x7ffeea2b9690, cleanup_fd=7,
    action_handlep=0x7ffeea2b9698, finalsnap=0x7ffeea2b8d10 "pre_forced_check", cmdprops=0x55d23fceae80) at libzfs_sendrecv.c:4000
#7  0x00007f6ee37c1f8d in zfs_receive_package (cmdprops=0x55d23fceae80, action_handlep=0x7ffeea2b9698, cleanup_fd=7, top_zfs=0x7ffeea2b9690, zc=0x7ffeea2b8810, drr=0x7ffeea2b8830,
    flags=0x7ffeea2b97a0, destname=0x7ffeea2baeae "dumbpool", fd=0, hdl=0x55d23fcea060) at libzfs_sendrecv.c:2929
#8  zfs_receive_impl (hdl=hdl@entry=0x55d23fcea060, tosnap=tosnap@entry=0x7ffeea2baeae "dumbpool", originsnap=<optimized out>, flags=flags@entry=0x7ffeea2b97a0, infd=infd@entry=0,
    sendfs=sendfs@entry=0x0, stream_nv=0x0, stream_avl=0x0, top_zfs=0x7ffeea2b9690, cleanup_fd=7, action_handlep=0x7ffeea2b9698, finalsnap=0x0, cmdprops=0x55d23fceae80) at libzfs_sendrecv.c:4006
#9  0x00007f6ee37c3a8f in zfs_receive (hdl=0x55d23fcea060, tosnap=0x7ffeea2baeae "dumbpool", props=0x55d23fceae80, flags=0x7ffeea2b97a0, infd=0, stream_avl=0x0) at libzfs_sendrecv.c:4079
#10 0x000055d23e9e1599 in zfs_do_receive (argc=<optimized out>, argv=0x7ffeea2b9a60) at zfs_main.c:4201
#11 0x000055d23e9d9595 in main (argc=<optimized out>, argv=0x7ffeea2b9a38) at zfs_main.c:7104
(gdb) list libzfs_util.c:306
301
302             if (hdl->libzfs_printerr) {
303                     if (error == EZFS_UNKNOWN) {
304                             (void) fprintf(stderr, dgettext(TEXT_DOMAIN, "internal "
305                                 "error: %s\n"), libzfs_error_description(hdl));
306                             abort();
307                     }
308
309                     (void) fprintf(stderr, "%s: %s\n", hdl->libzfs_action,
310                         libzfs_error_description(hdl));
(gdb) list libzfs_util.c:434
429             case EREMOTEIO:
430                     zfs_verror(hdl, EZFS_ACTIVE_POOL, fmt, ap);
431                     break;
432             default:
433                     zfs_error_aux(hdl, strerror(error));
434                     zfs_verror(hdl, EZFS_UNKNOWN, fmt, ap);
435                     break;
436             }
437
438             va_end(ap);
(gdb) list libzfs_sendrecv.c:3651
3646                                     */
3647                                    (void) snprintf(tbuf, sizeof (tbuf),
3648                                        dgettext(TEXT_DOMAIN,
3649                                        "cannot receive %s property on %s"),
3650                                        nvpair_name(prop_err), name);
3651                                    zfs_setprop_error(hdl, prop, intval, tbuf);
3652                            }
3653                    }
3654            }
3655
(gdb) up
#5  0x00007f6ee37bf6e5 in zfs_receive_one (hdl=hdl@entry=0x55d23fcea060, infd=infd@entry=0, tosnap=tosnap@entry=0x7ffeea2baeae "dumbpool", originsnap=originsnap@entry=0x0,
    flags=flags@entry=0x7ffeea2b97a0, drr=drr@entry=0x7ffeea2b7930, drr_noswap=0x7ffeea2b7a70, sendfs=0x7ffeea2b8cf0 "ephemeral/sorted/backups/3ds_sd", stream_nv=0x55d23fceaee0,
    stream_avl=0x55d23fcfcca0, top_zfs=0x7ffeea2b9690, cleanup_fd=7, action_handlep=0x7ffeea2b9698, finalsnap=0x7ffeea2b8d10 "pre_forced_check", cmdprops=0x55d23fceae80) at libzfs_sendrecv.c:3651
3651                                    zfs_setprop_error(hdl, prop, intval, tbuf);
(gdb) list
3646                                     */
3647                                    (void) snprintf(tbuf, sizeof (tbuf),
3648                                        dgettext(TEXT_DOMAIN,
3649                                        "cannot receive %s property on %s"),
3650                                        nvpair_name(prop_err), name);
3651                                    zfs_setprop_error(hdl, prop, intval, tbuf);
3652                            }
3653                    }
3654            }
3655
(gdb) p prop
$10 = ZFS_PROP_CHECKSUM

I've now got a core file with some variables optimized out in it if anyone cares to look. I'll probably try to look at it tomorrow since I suspect #7755's backtrace will be identical.