New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kernel oops in NFS-shared zfs #2165

Closed
ingolia opened this Issue Mar 6, 2014 · 1 comment

Comments

Projects
None yet
2 participants
@ingolia

ingolia commented Mar 6, 2014

I had a kernel OOPS on a zfs-backed NFS server -- the syslog message is below, I'm happy to provide further information as useful:

Mar 5 16:52:23 storage kernel: [3634128.457887] BUG: unable to handle kernel NULL pointer dereference at 0000000000000014
Mar 5 16:52:23 storage kernel: [3634128.474112] IP: [] arc_read+0x7a1/0x8d0 [zfs]
Mar 5 16:52:23 storage kernel: [3634128.482540] PGD 0
Mar 5 16:52:23 storage kernel: [3634128.490776] Oops: 0000 [#1] SMP
Mar 5 16:52:23 storage kernel: [3634128.499036] Modules linked in: 8021q garp stp mrp appletalk ipx p8023 p8022 psnap llc svcrdma xprtrdma rdma_ucm rdma_cm ib_addr iw_cm ib_uverbs ext2 ses enclosure vesafb(F) x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd gpio_ich dm_round_robin dm_multipath scsi_dh microcode ib_umad ib_ipoib sb_edac joydev ib_cm edac_core lpc_ich mei_me mei ioatdma nfsd auth_rpcgss nfs_acl wmi nfs lockd sunrpc mlx4_ib fscache ib_sa mac_hid ib_mad ib_core lp parport zfs(POF) zunicode(POF) zavl(POF) zcommon(POF) znvpair(POF) spl(OF) zlib_deflate mlx4_en hid_generic usb_storage igb mlx4_core mpt2sas i2c_algo_bit usbhid dca ahci hid ptp raid_class libahci pps_core scsi_transport_sas
Mar 5 16:52:23 storage kernel: [3634128.599086] CPU: 9 PID: 5069 Comm: nfsd Tainted: PF O 3.11.0-15-generic #23-Ubuntu
Mar 5 16:52:23 storage kernel: [3634128.618536] Hardware name: Supermicro X9DRi-LN4+/X9DR3-LN4+/X9DRi-LN4+/X9DR3-LN4+, BIOS 3.0 07/05/2013
Mar 5 16:52:23 storage kernel: [3634128.638911] task: ffff8820172a1770 ti: ffff8820170b2000 task.ti: ffff8820170b2000
Mar 5 16:52:23 storage kernel: [3634128.660476] RIP: 0010:[] [] arc_read+0x7a1/0x8d0 [zfs]
Mar 5 16:52:23 storage kernel: [3634128.682831] RSP: 0018:ffff8820170b37b0 EFLAGS: 00010282
Mar 5 16:52:23 storage kernel: [3634128.693955] RAX: ffff8805f5716598 RBX: ffffc91199a5f000 RCX: 0000000000000000
Mar 5 16:52:23 storage kernel: [3634128.715696] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8805f5716950
Mar 5 16:52:23 storage kernel: [3634128.738671] RBP: ffff8820170b3838 R08: ffffc90303a71000 R09: 0000000000020000
Mar 5 16:52:23 storage kernel: [3634128.760849] R10: ffff88100e3d6000 R11: 0000000000000000 R12: ffff88128431c1a8
Mar 5 16:52:23 storage kernel: [3634128.783629] R13: ffff8820170b3894 R14: ffff881002a5e000 R15: ffff880faa0ad4f8
Mar 5 16:52:23 storage kernel: [3634128.806763] FS: 0000000000000000(0000) GS:ffff88207fc20000(0000) knlGS:0000000000000000
Mar 5 16:52:23 storage kernel: [3634128.830427] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 5 16:52:23 storage kernel: [3634128.842035] CR2: 0000000000000014 CR3: 0000000001c0e000 CR4: 00000000000407e0
Mar 5 16:52:23 storage kernel: [3634128.864743] Stack:
Mar 5 16:52:23 storage kernel: [3634128.875923] ffffffffa025ad00 ffff881136c83b00 ffff880200000000 0000000000040640
Mar 5 16:52:23 storage kernel: [3634128.898274] ffff880d00000000 00000000170b3838 ffff88062a9c8910 0000000592935e00
Mar 5 16:52:23 storage kernel: [3634128.920513] ffff88100e3d6000 0000000000020000 ff2247bc00000000 ffffffffa0333550
Mar 5 16:52:23 storage kernel: [3634128.942832] Call Trace:
Mar 5 16:52:23 storage kernel: [3634128.953591] [] ? arc_read_done+0x2f0/0x2f0 [zfs]
Mar 5 16:52:23 storage kernel: [3634128.965882] [] dbuf_read+0x214/0x880 [zfs]
Mar 5 16:52:23 storage kernel: [3634128.976894] [] dmu_buf_hold_array_by_dnode+0x181/0x5c0 [zfs]
Mar 5 16:52:23 storage kernel: [3634128.998560] [] dmu_buf_hold_array+0x5d/0x80 [zfs]
Mar 5 16:52:23 storage kernel: [3634129.009277] [] dmu_read_uio+0x41/0xe0 [zfs]
Mar 5 16:52:23 storage kernel: [3634129.019881] [] zfs_read+0x17c/0x4b0 [zfs]
Mar 5 16:52:23 storage kernel: [3634129.030335] [] zpl_read_common+0x52/0x80 [zfs]
Mar 5 16:52:23 storage kernel: [3634129.040524] [] zpl_read+0x60/0x90 [zfs]
Mar 5 16:52:23 storage kernel: [3634129.050359] [] ? zpl_read_common+0x80/0x80 [zfs]
Mar 5 16:52:23 storage kernel: [3634129.060210] [] do_loop_readv_writev+0x62/0x90
Mar 5 16:52:23 storage kernel: [3634129.069996] [] do_readv_writev+0x20d/0x230
Mar 5 16:52:23 storage kernel: [3634129.079598] [] ? exportfs_decode_fh+0xa8/0x2e0
Mar 5 16:52:23 storage kernel: [3634129.088706] [] ? kmem_cache_alloc_trace+0x108/0x130
Mar 5 16:52:23 storage kernel: [3634129.097765] [] ? aa_alloc_task_context+0x27/0x40
Mar 5 16:52:23 storage kernel: [3634129.106578] [] ? apparmor_cred_prepare+0x2f/0x50
Mar 5 16:52:23 storage kernel: [3634129.115306] [] vfs_readv+0x32/0x60
Mar 5 16:52:23 storage kernel: [3634129.123727] [] nfsd_vfs_read.isra.12+0x6e/0x160 [nfsd]
Mar 5 16:52:23 storage kernel: [3634129.132186] [] ? find_client_in_id_table+0x141/0x190 [nfsd]
Mar 5 16:52:23 storage kernel: [3634129.148571] [] ? lookup_clientid+0x51/0x80 [nfsd]
Mar 5 16:52:23 storage kernel: [3634129.156810] [] nfsd_read_file+0x68/0xa0 [nfsd]
Mar 5 16:52:23 storage kernel: [3634129.165386] [] nfsd4_encode_read+0x16d/0x260 [nfsd]
Mar 5 16:52:23 storage kernel: [3634129.173982] [] nfsd4_encode_operation+0x5f/0xc0 [nfsd]
Mar 5 16:52:23 storage kernel: [3634129.182561] [] nfsd4_proc_compound+0x21e/0x7b0 [nfsd]
Mar 5 16:52:23 storage kernel: [3634129.190646] [] nfsd_dispatch+0xbb/0x200 [nfsd]
Mar 5 16:52:23 storage kernel: [3634129.198432] [] svc_process_common+0x46d/0x6e0 [sunrpc]
Mar 5 16:52:23 storage kernel: [3634129.206074] [] svc_process+0x107/0x170 [sunrpc]
Mar 5 16:52:23 storage kernel: [3634129.213744] [] nfsd+0xbf/0x130 [nfsd]
Mar 5 16:52:23 storage kernel: [3634129.221282] [] ? nfsd_destroy+0x80/0x80 [nfsd]
Mar 5 16:52:23 storage kernel: [3634129.229462] [] kthread+0xc0/0xd0
Mar 5 16:52:23 storage kernel: [3634129.236696] [] ? kthread_create_on_node+0x120/0x120
Mar 5 16:52:23 storage kernel: [3634129.245865] [] ret_from_fork+0x7c/0xb0
Mar 5 16:52:23 storage kernel: [3634129.254659] [] ? kthread_create_on_node+0x120/0x120
Mar 5 16:52:23 storage kernel: [3634129.263869] Code: 04 00 48 89 44 24 08 48 c7 04 24 00 ad 25 a0 89 54 24 18 48 8b 54 24 38 89 74 24 10 4c 89 f6 e8 a6 d7 09 00 49 8b 97 f8 00 00 00 <48> 63 52 14 f0 48 01 15 03 0a 0d 00 41 f6 45 00 04 48 89 c7 0f
Mar 5 16:52:23 storage kernel: [3634129.292264] RIP [] arc_read+0x7a1/0x8d0 [zfs]
Mar 5 16:52:23 storage kernel: [3634129.301443] RSP
Mar 5 16:52:23 storage kernel: [3634129.310911] CR2: 0000000000000014
Mar 5 16:52:23 storage kernel: [3634129.334845] ---[ end trace ed56a43c12891658 ]---

@behlendorf

This comment has been minimized.

Show comment
Hide comment
@behlendorf

behlendorf Mar 20, 2014

Member

Closing as duplicate of #2171.

Member

behlendorf commented Mar 20, 2014

Closing as duplicate of #2171.

@behlendorf behlendorf closed this Mar 20, 2014

behlendorf added a commit that referenced this issue Mar 24, 2014

Illumos #4089 NULL pointer dereference in arc_read()
4089 NULL pointer dereference in arc_read()

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Saso Kiselkov <skiselkov.ml@gmail.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/4089
  illumos/illumos-gate@57815f6

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #2171
Issue #2165
Closes #2198

@behlendorf behlendorf modified the milestones: 0.6.3, 0.6.4 Mar 24, 2014

ryao added a commit to ryao/zfs that referenced this issue Apr 9, 2014

Illumos zfsonlinux#4089 NULL pointer dereference in arc_read()
4089 NULL pointer dereference in arc_read()

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Saso Kiselkov <skiselkov.ml@gmail.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/4089
  illumos/illumos-gate@57815f6

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue zfsonlinux#2171
Issue zfsonlinux#2165
Closes zfsonlinux#2198

Conflicts:
	module/zfs/arc.c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment