LockExclusive crash

[arthurtsang]

I'm using rbd-docker-plugin and it crashed when calling LockExclusive.

here's my env,

CentOS 7.2 with kernel 4.5.3-1.el7.elrepo.x86_64,
ceph version 10.2.1 (3a66dd4f30852819c1bdaa8ec23c795d4ad77269)
librados2 & librbd1 1:10.2.1-0.el7

I couldn't see any relevant log messages, in dmesg, i see this usually (it didn't crash if the image is already lock and return a error message that says it can't get the exclusive lock)

[499439.546891] libceph: client14394 fsid bdce5816-e19c-4aac-b6e9-c694920dc9e7
[499439.552582] libceph: mon0 10.10.131.54:6789 session established
[499439.569701] rbd: rbd0: added with size 0x500000
[499439.693347] rbd: rbd1: added with size 0x500000

however when it crash, there's no such messages in the dmesg.

btw, the lock is created and locking an image using rdb command directly works fine.

[tdsparrow]

This issue only affect ceph after infernalis, which should be caused by librbd code change.
rbd_open didn't initialize image_wacher for ImageCtx. Later ImageCtx::notify_update uses the pointer and cause segfault.

[dotnwat]

I'm trying to get caught up on go-ceph issues, sorry about the delay. Do I understand correctly that the code works ok before infernalis and is broken after infernalis?

[dotnwat]

how hard would it be to provide a code snipper that reproduces the problem?

[arthurtsang]

oh, i'm just using rdb-docker-plugin. didn't actually write the code.
and yes, it works with older version of cepth

[david-z]

@noahdesu you are right, this issue won't happen on hammer at least. I also met this issue when using rbd-docker-plugin on recent jewel release. I just made some comments on ceph/ceph#9660 and hope to fix this issue soon.

If you would like to reproduce it, here is a golang code snippet, but pls ensure rbd image is existed before you run this code.

package main

import (
        "github.com/ceph/go-ceph/rbd"
        "github.com/ceph/go-ceph/rados"
        "fmt"
)

func main() {
        fmt.Printf("test_librbd\n")

        conn, _ := rados.NewConn()
        conn.ReadDefaultConfigFile()
        conn.Connect()

        poolname := "rbd"
        ioctx, err := conn.OpenIOContext(poolname)
        if err != nil {
                fmt.Printf("open pool fails: %s\n", err)
                return
        }

        // ensure rbd image is existed already
        imagename := "rbd_image_1"
        img := rbd.GetImage(ioctx, imagename)

        err = img.Open(true)
        if err != nil {
                fmt.Printf("fail to open image: %s\n", err)
                return
        }
        defer img.Close()

        locker := "test_lock"
        err = img.LockExclusive(locker)
        if err != nil {
                fmt.Printf("fail to lock image: %s\n", err)
                return
        }
        fmt.Printf("lock done\n")

        err = img.Unlock(locker)
        if err != nil {
                fmt.Printf("fail to unlock image: %s\n", err)
                return
        }
        fmt.Printf("unlock done\n")
}

[dotnwat]

Perfect, thanks. That will make things much easier!

[phlogistonjohn]

Hello, I'm automatically closing all issues filed prior to 2018 that have not been commented on recently. If this issue is still relevant to you please reopen the issue and restart the discussion. Thank you for your understanding.