Kernel Panic During rsync: spl_kmem_cache_alloc #2701
Comments
|
Okay...is there any more info I can provide, or should I just track with those other issues? |
|
@necouchman Following this panic, have you been able to locate a file on the target filesystem which triggers the problem? If so, could you please run |
|
@dweeezil I was not able to locate the exact file, and, unfortunately, needed to get the system running, again, so I blew away the zpool and re-created it, and left xattr=on instead of xattr=sa. I can try to set up a test system and recreate the issue, if that would help. As far as memory pressure - there is some, but most of it is consumed by buffer/cache, not actual process usage. I don't know about dedup on/off - I have dedup on (really need it for this particular system), and have not tried with xattr=sa and dedup off. |
|
@necouchman I've not been able to reproduce this problem yet but if you can, I'd love to know about it. Unfortunately, I've not had any time this week to look into it further. |
|
Nice... is there a schedule for the next RPM build (for CentOS 6), that would include this commit? I can build from source, but prefer to use the release RPMs. Thanks! - Nick |
|
@necouchman likely a few months. There are still a handful of fixes and features we want in the next tag. You could use the zfs-testing repository to track master, or just grab the latest release from the repository with the fix and stay there until a new tag is made. |
If a spill block's dbuf hasn't yet been written when a spill block is freed, the unwritten version will still be written. This patch handles the case in which a spill block's dbuf is freed and undirties it to prevent it from being written. The most common case in which this could happen is when xattr=sa is being used and a long xattr is immediately replaced by a short xattr as in: setfattr -n user.test -v very_very_very..._long_value <file> setfattr -n user.test -v short_value <file> The first value must be sufficiently long that a spill block is generated and the second value must be short enough to not require a spill block. In practice, this would typically happen due to internal xattr operations as a result of setting acltype=posixacl. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes openzfs#2663 Closes openzfs#2700 Closes openzfs#2701 Closes openzfs#2717 Closes openzfs#2863 Closes openzfs#2884 Conflicts: module/zfs/dbuf.c
If a spill block's dbuf hasn't yet been written when a spill block is freed, the unwritten version will still be written. This patch handles the case in which a spill block's dbuf is freed and undirties it to prevent it from being written. The most common case in which this could happen is when xattr=sa is being used and a long xattr is immediately replaced by a short xattr as in: setfattr -n user.test -v very_very_very..._long_value <file> setfattr -n user.test -v short_value <file> The first value must be sufficiently long that a spill block is generated and the second value must be short enough to not require a spill block. In practice, this would typically happen due to internal xattr operations as a result of setting acltype=posixacl. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #2663 Closes #2700 Closes #2701 Closes #2717 Closes #2863 Closes #2884
|
Looks like this fix was committed to the 0.6.3-1.2 tag, which has now been RPMd (I got it on my last CentOS6 upgrade)? Assuming that's true I should have it, now. |
Using ZFS on CentOS6.5. Just about all packages are up-to-date. My disks are FC-attached (32 x 500GB), and my pool splits those disks into four eight-disk raidz-1 vdevs. I have 64GB of RAM in a Dell PowerEdge 1950-III chassis, with 2 x 4-core CPUs. Not the latest and greatest, but a fairly modest system for what I'm trying to do. I'll note here, early on, that I've tested the RAM with memtest86+, and have not found any bad memory, so I don't think bad RAM is the blame in this case.
I'm trying to use the system to rsync data from several SAN volumes as a sort of disk-based backup system. I'm using iSCSI to attach the SAN volumes, most of which are formatted with XFS. I've scripted it down to a process that goes roughly like this:
On ZFS, I'm using the following:
I'm running into a fairly consistent, repeatable kernel panic during the rsync. I have vmcore and vmcore-dmesg output (text posted below) for the crashes. It doesn't crash in exactly the same place every time, but seems to be fairly early on in the rsync process (~7TB of data total, usually happens in the first 1-2 TB).
I've searched around a bit - hopefully not duplicating an existing bug out there.
==vmcore-dmesg.txt==
The text was updated successfully, but these errors were encountered: