kernel panic related to fuse_ipc.h

[ghost]

I have every now and then a reproducible kernel panic with osxfuse and MooseFS's mfsmount.

Interval Since Last Panic Report: 1539163 sec
Panics Since Last Report: 1
Anonymous UUID: 705A3546-596C-8078-32FA-2E741D660E4D

Sun Apr 28 17:46:21 2013
panic(cpu 2 caller 0xffffff7fb1d42aca): "OSXFUSE: fuse_ticket_release: ticket reference count is 0"@/Users/benjamin/Documents/Projekte/OSXFUSE/Repositories/osxfuse/kext/fuse_ipc.h:338
Backtrace (CPU 2), Frame : Return Address
0xffffff820143bc20 : 0xffffff802fa1d626
0xffffff820143bc90 : 0xffffff7fb1d42aca
0xffffff820143bd30 : 0xffffff802fb1f00b
0xffffff820143bdb0 : 0xffffff802fb10d5d
0xffffff820143be00 : 0xffffff802fb06ea9
0xffffff820143be50 : 0xffffff802fd758ce
0xffffff820143bef0 : 0xffffff802fd756f2
0xffffff820143bf50 : 0xffffff802fde063a
0xffffff820143bfb0 : 0xffffff802facdd23
Kernel Extensions in backtrace:
com.github.osxfuse.filesystems.osxfusefs(2.5.4)[2A591BB7-069D-966B-18F7-0154EF58C0E0]@0xffffff7fb1d41000->0xffffff7fb1d58fff

BSD process name corresponding to current thread: mfsmount

Mac OS version:
12D78

Kernel version:
Darwin Kernel Version 12.3.0: Sun Jan 6 22:37:10 PST 2013; root:xnu-2050.22.13~1/RELEASE_X86_64
Kernel UUID: 3EB7D8A7-C2D3-32EC-80F4-AB37D61492C6
Kernel slide: 0x000000002f800000
Kernel text base: 0xffffff802fa00000
System model name: iMac12,2 (Mac-942B59F58194171B)

System uptime in nanoseconds: 1125934955091443
last loaded kext at 1089803084247128: com.apple.driver.AppleUSBCDC 4.1.23 (addr 0xffffff7fb1e3d000, size 16384)
last unloaded kext at 1089870644084952: com.apple.driver.AppleUSBCDC 4.1.23 (addr 0xffffff7fb1e3d000, size 12288)
loaded kexts:
com.github.osxfuse.filesystems.osxfusefs 2.5.4
at.obdev.nke.LittleSnitch 2.5.4
com.apple.driver.IOBluetoothA2DPAudioDriver 4.1.3f3
com.apple.driver.IOBluetoothSCOAudioDriver 4.1.3f3
com.apple.driver.AppleBluetoothMultitouch 75.19
com.apple.iokit.IOBluetoothSerialManager 4.1.3f3
com.apple.driver.AppleHWSensor 1.9.5d0
com.apple.filesystems.autofs 3.0
com.apple.driver.AudioAUUC 1.60
com.apple.driver.AGPM 100.12.87
com.apple.driver.AppleMikeyHIDDriver 122
com.apple.driver.AppleHDA 2.3.7fc4
com.apple.driver.AppleSMCPDRC 1.0.0
com.apple.driver.AppleUSBDisplays 357
com.apple.iokit.BroadcomBluetoothHCIControllerUSBTransport 4.1.3f3
com.apple.iokit.IOUserEthernet 1.0.0d1
com.apple.Dont_Steal_Mac_OS_X 7.0.0
com.apple.driver.ApplePolicyControl 3.3.0
com.apple.driver.AppleMikeyDriver 2.3.7fc4
com.apple.driver.AppleLPC 1.6.0
com.apple.driver.AppleSMCLMU 2.0.3d0
com.apple.driver.AppleBacklight 170.2.5
com.apple.driver.AppleUpstreamUserClient 3.5.10
com.apple.driver.AppleMCCSControl 1.1.11
com.apple.kext.AMDFramebuffer 8.1.0
com.apple.driver.ACPI_SMC_PlatformPlugin 1.0.0
com.apple.driver.AppleIntelHD3000Graphics 8.1.0
com.apple.AMDRadeonAccelerator 1.0.6
com.apple.driver.AppleIntelSNBGraphicsFB 8.1.0
com.apple.iokit.SCSITaskUserClient 3.5.5
com.apple.driver.AppleUSBCardReader 3.1.7
com.apple.driver.AppleIRController 320.15
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0d1
com.apple.BootCache 34
com.apple.driver.XsanFilter 404
com.apple.iokit.IOAHCIBlockStorage 2.3.1
com.apple.driver.AppleUSBHub 5.5.5
com.apple.driver.AirPort.Atheros40 600.72.2
com.apple.iokit.AppleBCM5701Ethernet 3.6.0b1
com.apple.driver.AppleFWOHCI 4.9.6
com.apple.driver.AppleAHCIPort 2.5.1
com.apple.driver.AppleUSBEHCI 5.5.0
com.apple.driver.AppleEFINVRAM 1.7
com.apple.driver.AppleIntelCPUPowerManagementClient 196.0.0
com.apple.driver.AppleACPIButtons 1.7
com.apple.driver.AppleRTC 1.5
com.apple.driver.AppleHPET 1.8
com.apple.driver.AppleSMBIOS 1.9
com.apple.driver.AppleACPIEC 1.7
com.apple.driver.AppleAPIC 1.6
com.apple.nke.applicationfirewall 4.0.39
com.apple.security.quarantine 2
com.apple.driver.AppleIntelCPUPowerManagement 196.0.0
com.apple.driver.IOBluetoothHIDDriver 4.1.3f3
com.apple.driver.AppleMultitouchDriver 235.29
com.apple.iokit.IOSerialFamily 10.0.6
com.apple.kext.triggers 1.0
com.apple.driver.DspFuncLib 2.3.7fc4
com.apple.iokit.IOFireWireIP 2.2.5
com.apple.iokit.AppleBluetoothHCIControllerUSBTransport 4.1.3f3
com.apple.iokit.IOSurface 86.0.4
com.apple.iokit.IOBluetoothFamily 4.1.3f3
com.apple.driver.AppleSMBusPCI 1.0.11d0
com.apple.driver.AppleGraphicsControl 3.3.0
com.apple.driver.AppleBacklightExpert 1.0.4
com.apple.driver.AppleHDAController 2.3.7fc4
com.apple.iokit.IOHDAFamily 2.3.7fc4
com.apple.driver.AppleSMBusController 1.0.11d0
com.apple.driver.IOPlatformPluginLegacy 1.0.0
com.apple.driver.IOPlatformPluginFamily 5.3.0d51
com.apple.driver.AppleUSBAudio 2.9.0f6
com.apple.iokit.IOAudioFamily 1.8.9fc11
com.apple.kext.OSvKernDSPLib 1.6
com.apple.driver.AppleSMC 3.1.4d2
com.apple.iokit.IOAcceleratorFamily 30.14
com.apple.iokit.IONDRVSupport 2.3.7
com.apple.kext.AMD6000Controller 8.1.0
com.apple.kext.AMDSupport 8.1.0
com.apple.driver.AppleThunderboltEDMSink 1.1.8
com.apple.driver.AppleThunderboltEDMSource 1.1.8
com.apple.iokit.IOGraphicsFamily 2.3.7
com.apple.driver.AppleUSBHIDMouse 175.8
com.apple.driver.AppleHIDMouse 175.8
com.apple.driver.AppleUSBHIDKeyboard 170.2
com.apple.driver.AppleHIDKeyboard 170.2
com.apple.iokit.IOSCSIMultimediaCommandsDevice 3.5.5
com.apple.iokit.IOBDStorageFamily 1.7
com.apple.iokit.IODVDStorageFamily 1.7.1
com.apple.iokit.IOCDStorageFamily 1.7.1
com.apple.iokit.IOAHCISerialATAPI 2.5.1
com.apple.driver.AppleThunderboltDPOutAdapter 1.8.9
com.apple.driver.AppleThunderboltDPInAdapter 1.8.9
com.apple.driver.AppleThunderboltDPAdapterFamily 1.8.9
com.apple.driver.AppleThunderboltPCIDownAdapter 1.2.6
com.apple.iokit.IOSCSIBlockCommandsDevice 3.5.5
com.apple.iokit.IOUSBMassStorageClass 3.5.1
com.apple.iokit.IOSCSIArchitectureModelFamily 3.5.5
com.apple.iokit.IOUSBHIDDriver 5.2.5
com.apple.driver.AppleUSBMergeNub 5.5.5
com.apple.driver.AppleUSBComposite 5.2.5
com.apple.driver.CoreStorage 296.16
com.apple.iokit.IOUSBUserClient 5.5.5
com.apple.driver.AppleThunderboltNHI 1.6.3
com.apple.iokit.IOThunderboltFamily 2.2.6
com.apple.iokit.IO80211Family 522.4
com.apple.iokit.IOEthernetAVBController 1.0.2b1
com.apple.iokit.IONetworkingFamily 3.0
com.apple.iokit.IOFireWireFamily 4.5.5
com.apple.iokit.IOAHCIFamily 2.3.1
com.apple.iokit.IOUSBFamily 5.5.5
com.apple.driver.AppleEFIRuntime 1.7
com.apple.iokit.IOHIDFamily 1.8.1
com.apple.iokit.IOSMBusFamily 1.1
com.apple.security.sandbox 220.2
com.apple.kext.AppleMatch 1.0.0d1
com.apple.security.TMSafetyNet 7
com.apple.driver.DiskImages 345
com.apple.iokit.IOStorageFamily 1.8
com.apple.driver.AppleKeyStore 28.21
com.apple.driver.AppleACPIPlatform 1.7
com.apple.iokit.IOPCIFamily 2.7.3
com.apple.iokit.IOACPIFamily 1.4
com.apple.kec.corecrypto 1.0
Model: iMac12,2, BootROM IM121.0047.B1F, 4 processors, Intel Core i5, 2.7 GHz, 16 GB, SMC 1.72f2
Graphics: AMD Radeon HD 6770M, AMD Radeon HD 6770M, PCIe, 512 MB
Memory Module: BANK 0/DIMM0, 4 GB, DDR3, 1333 MHz, 0x80CE, 0x4D34373142353237334348302D4348392020
Memory Module: BANK 1/DIMM0, 4 GB, DDR3, 1333 MHz, 0x80CE, 0x4D34373142353237334348302D4348392020
Memory Module: BANK 0/DIMM1, 4 GB, DDR3, 1333 MHz, 0x80CE, 0x4D34373142353237334348302D4348392020
Memory Module: BANK 1/DIMM1, 4 GB, DDR3, 1333 MHz, 0x80CE, 0x4D34373142353237334348302D4348392020
AirPort: spairport_wireless_card_type_airport_extreme (0x168C, 0x9A), Atheros 9380: 4.0.72.0-P2P
Bluetooth: Version 4.1.3f3 11349, 2 service, 18 devices, 1 incoming serial ports
Network Service: Ethernet, Ethernet, en0
Serial ATA Device: ST31000528AS, 1 TB
Serial ATA Device: HL-DT-STDVDRW GA32N
USB Device: hub_device, 0x0424 (SMSC), 0x2514, 0xfd100000 / 2
USB Device: hub_device, apple_vendor_id, 0x9126, 0xfd140000 / 5
USB Device: Back-UPS CS 500 FW:808.q8.I USB FW:q8, 0x051d (American Power Conversion), 0x0002, 0xfd141000 / 9
USB Device: Display Audio, apple_vendor_id, 0x1105, 0xfd144000 / 8
USB Device: Display iSight, apple_vendor_id, 0x8508, 0xfd145000 / 7
USB Device: Apple LED Cinema Display, apple_vendor_id, 0x9226, 0xfd146000 / 6
USB Device: Internal Memory Card Reader, apple_vendor_id, 0x8403, 0xfd110000 / 4
USB Device: IR Receiver, apple_vendor_id, 0x8242, 0xfd120000 / 3
USB Device: hub_device, 0x0424 (SMSC), 0x2514, 0xfa100000 / 3
USB Device: hub_device, apple_vendor_id, 0x9126, 0xfa130000 / 6
USB Device: iPhone, apple_vendor_id, 0x1297, 0xfa133000 / 15
USB Device: HP LaserJet M1536dnf MFP, 0x03f0 (Hewlett Packard), 0x012a, 0xfa132000 / 14
USB Device: Display iSight, apple_vendor_id, 0x8508, 0xfa135000 / 13
USB Device: Apple LED Cinema Display, apple_vendor_id, 0x9226, 0xfa136000 / 12
USB Device: Display Audio, apple_vendor_id, 0x1105, 0xfa134000 / 11
USB Device: Keyboard Hub, apple_vendor_id, 0x1006, 0xfa120000 / 5
USB Device: Apple Optical USB Mouse, apple_vendor_id, 0x0304, 0xfa123000 / 10
USB Device: Yubico Yubikey II, 0x1050, 0x0010, 0xfa121000 / 9
USB Device: Apple Keyboard, apple_vendor_id, 0x0221, 0xfa122000 / 8
USB Device: BRCM2046 Hub, 0x0a5c (Broadcom Corp.), 0x4500, 0xfa110000 / 4
USB Device: Bluetooth USB Host Controller, apple_vendor_id, 0x8215, 0xfa111000 / 7
USB Device: FaceTime HD Camera (Built-in), apple_vendor_id, 0x850b, 0xfa200000 / 2

[ghost]

It seem's to be related when the MFS master is under heavy load (currently he's doing heavy chunk rebalancing)

[bfleischer]

Thanks for the bug report. So far I have not found an obvious reason for the panic. But you are using an outdated version of OSXFUSE. Can you confirm that the bug still exists in version 2.5.5? One of the bugs that has been fixed in 2.5.5 might be responsible for your panic.

[ghost]

Will try that and keep you updated.

[bfleischer]

This is the symbolicated stack trace from the panic:

panic
fuse_device_read
spec_read
VNOP_READ
vn_read
dofileread
read_nocancel
unix_syscall64
hndl_unix_scall64

I think I found the bug. When creating a file fails we need to clean up. This cleanup code releases a ticket that really should not be releases at this point. This would explain the panic.

[bfleischer]

OSXFUSE 2.5.6 has been released and addresses this issue. Please test again with version 2.5.6 and reopen this ticket if you are still seeing the panic.