Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gfid backend #3

Closed
wants to merge 8 commits into from
Closed

Gfid backend #3

wants to merge 8 commits into from

Conversation

amarts
Copy link
Member

@amarts amarts commented Nov 18, 2011

rebase from avati's iops branch

* This is the first change which brings a change in backend structure,
  a added directory structure.

* Also brings in changes for supporting anonymous fd, ie, a fd which
  is open only with gfid, but no path associated with it.

TODO: a more detailed write up on how the backend is maintained etc
will be done soon.

Change-Id: Ia07996444fb1be40baa3e51212c3e272f49fed2c
Change-Id: Idc70aa16bc4c5d62497175545d62b60838f3c995
Change-Id: I1364664c08cf8a52ab3ff6fb67f9ec6fae9a61a6
Change-Id: I79c713bf68cebafca8d7ab9b1e6d94ed3dbc77e1
Change-Id: I1bfb9accdeb0491dea47b96ad0dfd66acadaceb3
Change-Id: If18de9ff584a7012259919f9f47faa26886f65e4
Change-Id: Id96c438ad89a08d84e4ca0e5af044e37f06ccce3
Change-Id: Ib05b293b9e0e5f5c0bdbb8f945a83f7c3d8ddc0c
@avati avati closed this Sep 7, 2013
vpshastry pushed a commit to vpshastry/glusterfs that referenced this pull request Jun 30, 2014
glusterd: search through the list of hostnames in the peerinfo
avati pushed a commit that referenced this pull request Jul 15, 2014
This patch improves the peer identification mechanism in glusterd and
lays down the framework for further improvements, including better multi
network support in glusterd.

This patch mainly does two things,
1. Extend the peerinfo object to store a list of addresses instead of a
single hostname as it does now. This also includes changes to make the
peer update behaviour of 'peer probe' to add to the list.

2. Improve glusterd_friend_find_by_hostname() to perform better matching
of hostnames. glusterd_friend_find_by_hostname() now does and initial
quick string compare against all the peer addresses known to glusterd,
after which it tries a more thorough search using address resolution and
matching the struc sockaddr's.

The above two changes together improve the peer identification situation
in glusterd a lot.

More information regarding the problem this patch attempts to resolve
and the approach chosen can be found at
http://www.gluster.org/community/documentation/index.php/Features/Better_peer_identification

This commit is a squashed commit of the following changes, the
development branch of which can be viewed at,
https://github.com/kshlm/glusterfs/tree/better-peer-identification or,
https://forge.gluster.org/~kshlm/glusterfs-core/kshlms-glusterfs/commits/better-peer-identification

commit 198f86e
Author: Kaushal M <kaushal@redhat.com>
Date:   Tue Jul 15 14:34:06 2014 +0530

    Update gluster.8

commit 35d597f
Author: Kaushal M <kaushal@redhat.com>
Date:   Tue Jul 15 09:01:01 2014 +0530

    Address review comments

    https://review.gluster.org/#/c/8238/3

commit 47b5331
Merge: c71b12c 78128af
Author: Kaushal M <kaushal@redhat.com>
Date:   Tue Jul 15 08:41:39 2014 +0530

    Merge branch 'master' into better-peer-identification

commit c71b12c
Merge: 57bc9de 0f5719a
Author: Kaushal M <kaushal@redhat.com>
Date:   Thu Jul 10 19:50:19 2014 +0530

    Merge branch 'master' into better-peer-identification

commit 57bc9de
Author: Kaushal M <kaushal@redhat.com>
Date:   Thu Jul 10 19:49:08 2014 +0530

    More fixes to review comments

commit 5482cc3
Author: Kaushal M <kaushal@redhat.com>
Date:   Thu Jul 10 18:36:40 2014 +0530

    Code refactoring in peer-utils based on review comments

    https://review.gluster.org/#/c/8238/2/xlators/mgmt/glusterd/src/glusterd-peer-utils.c

commit 89b22c3
Author: Kaushal M <kaushal@redhat.com>
Date:   Thu Jul 10 12:30:00 2014 +0530

    Hostnames in peer status

commit 63ebf94
Merge: c8c8fdd f5f9721
Author: Kaushal M <kaushal@redhat.com>
Date:   Thu Jul 10 12:06:33 2014 +0530

    Merge remote-tracking branch 'origin/master' into better-peer-identification

commit c8c8fdd
Author: Kaushal M <kaushal@redhat.com>
Date:   Wed Jul 9 18:35:27 2014 +0530

    Hostnames in xml output

commit 732a92a
Author: Kaushal M <kaushal@redhat.com>
Date:   Wed Jul 9 15:12:10 2014 +0530

    Add hostnames to cli rsp dict during list-friends

commit fcf43e3
Merge: c0e2624 72d96e2
Author: Kaushal M <kaushal@redhat.com>
Date:   Wed Jul 9 12:53:03 2014 +0530

    Merge branch 'master' into better-peer-identification

commit c0e2624
Author: Kaushal M <kaushal@redhat.com>
Date:   Mon Jul 7 16:11:19 2014 +0530

    Use list_for_each_entry_safe when cleaning peer hostnames

commit 6132e60
Author: Kaushal M <kaushal@redhat.com>
Date:   Mon Jul 7 15:52:19 2014 +0530

    Fix crash in gd_add_friend_to_dict

commit 88ffa9a
Author: Kaushal M <kaushal@redhat.com>
Date:   Mon Jul 7 13:19:44 2014 +0530

    gd_peerinfo_destroy -> glusterd_peerinfo_destroy

commit 4b36930
Author: Kaushal M <kaushal@redhat.com>
Date:   Mon Jul 7 12:50:12 2014 +0530

    More refactoring

commit ee559b0
Author: Kaushal M <kaushal@redhat.com>
Date:   Mon Jul 7 12:14:40 2014 +0530

    Major refactoring of code based on review comments at
    https://review.gluster.org/#/c/8238/1/xlators/mgmt/glusterd/src/glusterd-peer-utils.h

commit e96dbc7
Merge: 2613d1d 83c09b7
Author: Kaushal M <kaushal@redhat.com>
Date:   Mon Jul 7 09:47:05 2014 +0530

    Merge remote-tracking branch 'origin/master' into better-peer-identification

commit 2613d1d
Merge: b242cf6 9a50211
Author: Kaushal M <kaushal@redhat.com>
Date:   Fri Jul 4 15:28:57 2014 +0530

    Merge remote-tracking branch 'origin/master' into better-peer-identification

commit b242cf6
Author: Kaushal M <kaushal@redhat.com>
Date:   Fri Jul 4 15:08:18 2014 +0530

    Fix a silly mistake, if (ctx->req) => if (ctx->req == NULL)

commit c835ed2
Author: Kaushal M <kaushal@redhat.com>
Date:   Fri Jul 4 14:58:23 2014 +0530

    Fix reverse probe.

commit 9ede17f
Author: Kaushal M <kaushal@redhat.com>
Date:   Fri Jul 4 13:31:32 2014 +0530

    Fix friend import for existing peers

commit 891bf74
Author: Kaushal M <kaushal@redhat.com>
Date:   Fri Jul 4 13:08:36 2014 +0530

    Set first hostname in peerinfo->hostnames to peerinfo->hostname

commit 9421d6a
Author: Kaushal M <kaushal@redhat.com>
Date:   Fri Jul 4 12:21:40 2014 +0530

    Fix gf_asprintf return val check in glusterd_store_peer_write

commit defac97
Author: Kaushal M <kaushal@redhat.com>
Date:   Fri Jul 4 11:16:13 2014 +0530

    Fix store_retrieve_peers to correctly cleanup.

commit 00a799f
Author: Kaushal M <kaushal@redhat.com>
Date:   Fri Jul 4 10:52:11 2014 +0530

    Update address list in glusterd_probe_cbk only when needed.

commit 7a628e8
Merge: d191985 dc46d5e
Author: Kaushal M <kaushal@redhat.com>
Date:   Fri Jul 4 09:24:12 2014 +0530

    Merge remote-tracking branch 'origin/master' into better-peer-identification

commit d191985
Author: Kaushal M <kaushal@redhat.com>
Date:   Tue Jul 1 18:59:49 2014 +0530

    gf_compare_addrinfo -> gf_compare_sockaddr

commit 31d8ef7
Merge: 93bbede 86ee233
Author: Kaushal M <kaushal@redhat.com>
Date:   Tue Jul 1 18:16:13 2014 +0530

    Merge remote-tracking branch 'origin/master' into better-peer-identification

commit 93bbede
Author: Kaushal M <kaushal@redhat.com>
Date:   Tue Jul 1 18:15:16 2014 +0530

    Improve glusterd_friend_find_by_hostname

    glusterd_friend_find_by_hostname will now do an initial quick search for
    the peerinfo performing string comparisions on the given host string. It
    follows it with a more thorough match, by resolving the addresses and
    comparing addrinfos instead of strings.

commit 2542cdb
Author: Kaushal M <kaushal@redhat.com>
Date:   Tue Jul 1 17:21:10 2014 +0530

    New utility gf_compare_addrinfo

commit 338676e
Author: Kaushal M <kaushal@redhat.com>
Date:   Tue Jul 1 14:55:56 2014 +0530

    Use gd_peer_has_address instead of strcmp

commit 28d45be
Merge: 728266e 991dd5e
Author: Kaushal M <kaushal@redhat.com>
Date:   Tue Jul 1 14:54:40 2014 +0530

    Merge branch 'master' into better-peer-identification

commit 728266e
Merge: 7d9b87b 2417de9
Author: Kaushal M <kaushal@redhat.com>
Date:   Tue Jul 1 09:55:13 2014 +0530

    Merge remote-tracking branch 'origin/master' into better-peer-identification

commit 7d9b87b
Merge: b890625 e02275c
Author: Kaushal M <kshlmster@gmail.com>
Date:   Tue Jul 1 08:41:40 2014 +0530

    Merge pull request #4 from vpshastry/better-peer-identification

    Better peer identification

commit e02275c
Merge: 75ee90d b890625
Author: Varun Shastry <vshastry@redhat.com>
Date:   Mon Jun 30 16:44:29 2014 +0530

    Merge branch 'better-peer-identification' of https://github.com/kshlm/glusterfs into better-peer-identification-kaushal-github

commit 75ee90d
Author: Varun Shastry <vshastry@redhat.com>
Date:   Mon Jun 30 15:36:10 2014 +0530

    glusterd: add to the list if the probed uuid pre-exists

    Signed-off-by: Varun Shastry <vshastry@redhat.com>

commit b890625
Merge: 04c5d60 187a7a9
Author: Kaushal M <kaushal@redhat.com>
Date:   Mon Jun 30 11:44:13 2014 +0530

    Merge remote-tracking branch 'origin/master' into better-peer-identification

commit 04c5d60
Merge: 3a5bfa1 e01edb6
Author: Kaushal M <kshlmster@gmail.com>
Date:   Sat Jun 28 19:23:33 2014 +0530

    Merge pull request #3 from vpshastry/better-peer-identification

    glusterd: search through the list of hostnames in the peerinfo

commit 0c64f33
Merge: e01edb6 3a5bfa1
Author: Varun Shastry <vshastry@redhat.com>
Date:   Sat Jun 28 10:43:29 2014 +0530

    Merge branch 'better-peer-identification' of https://github.com/kshlm/glusterfs into better-peer-identification-kaushal-github

commit e01edb6
Author: Varun Shastry <vshastry@redhat.com>
Date:   Fri Jun 27 12:29:36 2014 +0530

    glusterd: search through the list of hostnames in the peerinfo

    Signed-off-by: Varun Shastry <vshastry@redhat.com>

commit 3a5bfa1
Merge: cda6d31 371ea35
Author: Kaushal M <kshlmster@gmail.com>
Date:   Fri Jun 27 11:31:17 2014 +0530

    Merge pull request #1 from vpshastry/better-peer-identification

    glusterd: Add hostname to list instead of replaceing upon update

commit 371ea35
Author: Varun Shastry <vshastry@redhat.com>
Date:   Fri Jun 27 11:24:54 2014 +0530

    glusterd: Add hostname to list instead of replaceing upon update

    Signed-off-by: Varun Shastry <vshastry@redhat.com>

commit cda6d31
Author: Kaushal M <kaushal@redhat.com>
Date:   Thu Jun 26 19:52:52 2014 +0530

    Import address lists

commit 6649b54
Author: Kaushal M <kaushal@redhat.com>
Date:   Thu Jun 26 19:15:37 2014 +0530

    Implement export address list

commit 5599003
Author: Kaushal M <kaushal@redhat.com>
Date:   Thu Jun 26 18:11:59 2014 +0530

    Use first address in list to when setting up the peer RPC.

commit a35fde8
Author: Kaushal M <kaushal@redhat.com>
Date:   Thu Jun 26 18:03:04 2014 +0530

    Properly free addresses on glusterd_peer_destroy

commit 1988081
Author: Kaushal M <kaushal@redhat.com>
Date:   Thu Jun 26 17:52:35 2014 +0530

    Restore peerinfo with address list implemented.

commit 66f524d
Author: Kaushal M <kaushal@redhat.com>
Date:   Mon Jun 23 13:02:23 2014 +0530

    Move out all peer related utilities from glusterd-utils to glusterd-peer-utils

commit 14a2a32
Author: Kaushal M <kaushal@redhat.com>
Date:   Tue May 27 12:16:41 2014 +0530

    Compilation fix

commit c59cd35
Author: Kaushal M <kaushal@redhat.com>
Date:   Mon May 5 12:51:11 2014 +0530

    Add store support for hostname list

commit b70325f
Author: Kaushal M <kaushal@redhat.com>
Date:   Fri May 2 15:58:07 2014 +0530

    Add a hostnames list to glusterd_peerinfo_t

    glusterd_peerinfo_new will now init this list and add the given hostname
    as the lists first member.

Signed-off-by: Kaushal M <kaushal@redhat.com>
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Change-Id: Ief3c5d6d6f16571ee2fab0a45e638b9d6506a06e
BUG: 1119547
Reviewed-on: http://review.gluster.org/8238
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
mscherer pushed a commit that referenced this pull request Jul 28, 2017
…ction

Problem: Sometime brick process is getting crash in notify function at the
         time of cleanup db connection while brick mux is enabled.

Solution: In changetimerrecorder (ctr) notify function after cleanup
          db connection set to db_conn to NULL to avoid reuse the same
          db connection again.

Note: Below is the backtrace pattern showing by brick process
      #0  0x00007ff98a30c1f7 in raise () from /lib64/libc.so.6
      #1  0x00007ff98a30d8e8 in abort () from /lib64/libc.so.6
      #2  0x00007ff98a34bf47 in __libc_message () from /lib64/libc.so.6
      #3  0x00007ff98a351b54 in malloc_printerr () from /lib64/libc.so.6
      #4  0x00007ff98a3537aa in _int_free () from /lib64/libc.so.6
      #5  0x00007ff97d95e311 in gf_sql_connection_fini (sql_connection=sql_connection@entry=0x7ff8e8496b50) at gfdb_sqlite3.c:42
      #6  0x00007ff97d95e38a in gf_sqlite3_fini (db_conn=0x7ff92ca04470) at gfdb_sqlite3.c:507
      #7  0x00007ff97d957156 in fini_db (_conn_node=0x7ff92ca04470) at gfdb_data_store.c:326
      #8  0x00007ff97db78679 in notify (this=0x7ff92c5b3670, event=9, data=0x7ff92c5b5a00) at changetimerecorder.c:2178
      #9  0x00007ff98bca0dc2 in xlator_notify (xl=0x7ff92c5b3670, event=event@entry=9, data=data@entry=0x7ff92c5b5a00) at xlator.c:549
      #10 0x00007ff98bd3ac12 in default_notify (this=this@entry=0x7ff92c5b5a00, event=9, data=data@entry=0x7ff92c5b6d50) at defaults.c:3139

BUG: 1475632
Change-Id: Idd4bfdb4629c4799ac477ade81228065212683fb
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
Reviewed-on: https://review.gluster.org/17888
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
mscherer pushed a commit that referenced this pull request Jul 31, 2017
…ction

Problem: Sometime brick process is getting crash in notify function at the
         time of cleanup db connection while brick mux is enabled.

Solution: In changetimerrecorder (ctr) notify function after cleanup
          db connection set to db_conn to NULL to avoid reuse the same
          db connection again.

Note: Below is the backtrace pattern showing by brick process
      #0  0x00007ff98a30c1f7 in raise () from /lib64/libc.so.6
      #1  0x00007ff98a30d8e8 in abort () from /lib64/libc.so.6
      #2  0x00007ff98a34bf47 in __libc_message () from /lib64/libc.so.6
      #3  0x00007ff98a351b54 in malloc_printerr () from /lib64/libc.so.6
      #4  0x00007ff98a3537aa in _int_free () from /lib64/libc.so.6
      #5  0x00007ff97d95e311 in gf_sql_connection_fini (sql_connection=sql_connection@entry=0x7ff8e8496b50) at gfdb_sqlite3.c:42
      #6  0x00007ff97d95e38a in gf_sqlite3_fini (db_conn=0x7ff92ca04470) at gfdb_sqlite3.c:507
      #7  0x00007ff97d957156 in fini_db (_conn_node=0x7ff92ca04470) at gfdb_data_store.c:326
      #8  0x00007ff97db78679 in notify (this=0x7ff92c5b3670, event=9, data=0x7ff92c5b5a00) at changetimerecorder.c:2178
      #9  0x00007ff98bca0dc2 in xlator_notify (xl=0x7ff92c5b3670, event=event@entry=9, data=data@entry=0x7ff92c5b5a00) at xlator.c:549
      #10 0x00007ff98bd3ac12 in default_notify (this=this@entry=0x7ff92c5b5a00, event=9, data=data@entry=0x7ff92c5b6d50) at defaults.c:3139

> BUG: 1475632
> Change-Id: Idd4bfdb4629c4799ac477ade81228065212683fb
> Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
> Reviewed-on: https://review.gluster.org/17888
> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
> Smoke: Gluster Build System <jenkins@build.gluster.org>
> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
> (cherry picked from commit fc0fce2)

BUG: 1476109
Change-Id: I96b7ab765b596cec5b779d7186ec549615e3b68b
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
Reviewed-on: https://review.gluster.org/17902
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
mscherer pushed a commit that referenced this pull request Aug 8, 2017
Refcounting added for nfs call state in https://review.gluster.org/17696.
This is based on assumption that call state won't NULL when it is freed.
But currently gluster nfs server is crashing in different scenarios at
nfs3_getattr() with following bt

#0  0x00007ff1cfea9205 in _gf_ref_put (ref=ref@entry=0x0) at refcount.c:36
#1  0x00007ff1c1997455 in nfs3_call_state_wipe (cs=cs@entry=0x0) at nfs3.c:559
#2  0x00007ff1c1998931 in nfs3_getattr (req=req@entry=0x7ff1bc0b26d0, fh=fh@entry=0x7ff1c2f76ae0) at nfs3.c:962
#3  0x00007ff1c1998c8a in nfs3svc_getattr (req=0x7ff1bc0b26d0) at nfs3.c:987
#4  0x00007ff1cfbfd8c5 in rpcsvc_handle_rpc_call (svc=0x7ff1bc03e500, trans=trans@entry=0x7ff1bc0c8020, msg=<optimized out>) at rpcsvc.c:695
#5  0x00007ff1cfbfdaab in rpcsvc_notify (trans=0x7ff1bc0c8020, mydata=<optimized out>, event=<optimized out>, data=<optimized out>) at rpcsvc.c:789
#6  0x00007ff1cfbff9e3 in rpc_transport_notify (this=this@entry=0x7ff1bc0c8020, event=event@entry=RPC_TRANSPORT_MSG_RECEIVED, data=data@entry=0x7ff1bc0038d0)
    at rpc-transport.c:538
#7  0x00007ff1c4a2e3d6 in socket_event_poll_in (this=this@entry=0x7ff1bc0c8020, notify_handled=<optimized out>) at socket.c:2306
#8  0x00007ff1c4a3097c in socket_event_handler (fd=21, idx=9, gen=19, data=0x7ff1bc0c8020, poll_in=1, poll_out=0, poll_err=0) at socket.c:2458
#9  0x00007ff1cfe950f6 in event_dispatch_epoll_handler (event=0x7ff1c2f76e80, event_pool=0x5618154d5ee0) at event-epoll.c:572
#10 event_dispatch_epoll_worker (data=0x56181551cbd0) at event-epoll.c:648
#11 0x00007ff1cec99e25 in start_thread () from /lib64/libpthread.so.0
#12 0x00007ff1ce56634d in clone () from /lib64/libc.so.6

This patch add previous NULL check move from __nfs3_call_state_wipe() to nfs3_call_state_wipe()

Change-Id: I2d73632f4be23f14d8467be3d908b09b3a2d87ea
BUG: 1479030
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
Reviewed-on: https://review.gluster.org/17989
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
mscherer pushed a commit that referenced this pull request Aug 8, 2017
Refcounting added for nfs call state in https://review.gluster.org/17696.
This is based on assumption that call state won't NULL when it is freed.
But currently gluster nfs server is crashing in different scenarios at
nfs3_getattr() with following bt

#0  0x00007ff1cfea9205 in _gf_ref_put (ref=ref@entry=0x0) at refcount.c:36
#1  0x00007ff1c1997455 in nfs3_call_state_wipe (cs=cs@entry=0x0) at nfs3.c:559
#2  0x00007ff1c1998931 in nfs3_getattr (req=req@entry=0x7ff1bc0b26d0, fh=fh@entry=0x7ff1c2f76ae0) at nfs3.c:962
#3  0x00007ff1c1998c8a in nfs3svc_getattr (req=0x7ff1bc0b26d0) at nfs3.c:987
#4  0x00007ff1cfbfd8c5 in rpcsvc_handle_rpc_call (svc=0x7ff1bc03e500, trans=trans@entry=0x7ff1bc0c8020, msg=<optimized out>) at rpcsvc.c:695
#5  0x00007ff1cfbfdaab in rpcsvc_notify (trans=0x7ff1bc0c8020, mydata=<optimized out>, event=<optimized out>, data=<optimized out>) at rpcsvc.c:789
#6  0x00007ff1cfbff9e3 in rpc_transport_notify (this=this@entry=0x7ff1bc0c8020, event=event@entry=RPC_TRANSPORT_MSG_RECEIVED, data=data@entry=0x7ff1bc0038d0)
    at rpc-transport.c:538
#7  0x00007ff1c4a2e3d6 in socket_event_poll_in (this=this@entry=0x7ff1bc0c8020, notify_handled=<optimized out>) at socket.c:2306
#8  0x00007ff1c4a3097c in socket_event_handler (fd=21, idx=9, gen=19, data=0x7ff1bc0c8020, poll_in=1, poll_out=0, poll_err=0) at socket.c:2458
#9  0x00007ff1cfe950f6 in event_dispatch_epoll_handler (event=0x7ff1c2f76e80, event_pool=0x5618154d5ee0) at event-epoll.c:572
#10 event_dispatch_epoll_worker (data=0x56181551cbd0) at event-epoll.c:648
#11 0x00007ff1cec99e25 in start_thread () from /lib64/libpthread.so.0
#12 0x00007ff1ce56634d in clone () from /lib64/libc.so.6

This patch add previous NULL check move from __nfs3_call_state_wipe() to
nfs3_call_state_wipe()

Cherry picked from commit 111d6bd:
> Change-Id: I2d73632f4be23f14d8467be3d908b09b3a2d87ea
> BUG: 1479030
> Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
> Reviewed-on: https://review.gluster.org/17989
> Smoke: Gluster Build System <jenkins@build.gluster.org>
> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
> Reviewed-by: Niels de Vos <ndevos@redhat.com>

Change-Id: I2d73632f4be23f14d8467be3d908b09b3a2d87ea
BUG: 1479263
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17994
Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
mscherer pushed a commit that referenced this pull request Aug 11, 2017
Refcounting added for nfs call state in https://review.gluster.org/17696.
This is based on assumption that call state won't NULL when it is freed.
But currently gluster nfs server is crashing in different scenarios at
nfs3_getattr() with following bt

#0  0x00007ff1cfea9205 in _gf_ref_put (ref=ref@entry=0x0) at refcount.c:36
#1  0x00007ff1c1997455 in nfs3_call_state_wipe (cs=cs@entry=0x0) at nfs3.c:559
#2  0x00007ff1c1998931 in nfs3_getattr (req=req@entry=0x7ff1bc0b26d0, fh=fh@entry=0x7ff1c2f76ae0) at nfs3.c:962
#3  0x00007ff1c1998c8a in nfs3svc_getattr (req=0x7ff1bc0b26d0) at nfs3.c:987
#4  0x00007ff1cfbfd8c5 in rpcsvc_handle_rpc_call (svc=0x7ff1bc03e500, trans=trans@entry=0x7ff1bc0c8020, msg=<optimized out>) at rpcsvc.c:695
#5  0x00007ff1cfbfdaab in rpcsvc_notify (trans=0x7ff1bc0c8020, mydata=<optimized out>, event=<optimized out>, data=<optimized out>) at rpcsvc.c:789
#6  0x00007ff1cfbff9e3 in rpc_transport_notify (this=this@entry=0x7ff1bc0c8020, event=event@entry=RPC_TRANSPORT_MSG_RECEIVED, data=data@entry=0x7ff1bc0038d0)
    at rpc-transport.c:538
#7  0x00007ff1c4a2e3d6 in socket_event_poll_in (this=this@entry=0x7ff1bc0c8020, notify_handled=<optimized out>) at socket.c:2306
#8  0x00007ff1c4a3097c in socket_event_handler (fd=21, idx=9, gen=19, data=0x7ff1bc0c8020, poll_in=1, poll_out=0, poll_err=0) at socket.c:2458
#9  0x00007ff1cfe950f6 in event_dispatch_epoll_handler (event=0x7ff1c2f76e80, event_pool=0x5618154d5ee0) at event-epoll.c:572
#10 event_dispatch_epoll_worker (data=0x56181551cbd0) at event-epoll.c:648
#11 0x00007ff1cec99e25 in start_thread () from /lib64/libpthread.so.0
#12 0x00007ff1ce56634d in clone () from /lib64/libc.so.6

This patch add previous NULL check move from __nfs3_call_state_wipe() to
nfs3_call_state_wipe()

Cherry picked from commit 111d6bd:
> Change-Id: I2d73632f4be23f14d8467be3d908b09b3a2d87ea
> BUG: 1479030
> Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
> Reviewed-on: https://review.gluster.org/17989
> Smoke: Gluster Build System <jenkins@build.gluster.org>
> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
> Reviewed-by: Niels de Vos <ndevos@redhat.com>

Change-Id: I2d73632f4be23f14d8467be3d908b09b3a2d87ea
BUG: 1480594
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/18027
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
amarts referenced this pull request in amarts/glusterfs_fork Oct 31, 2017
…ction

Problem: Sometime brick process is getting crash in notify function at the
         time of cleanup db connection while brick mux is enabled.

Solution: In changetimerrecorder (ctr) notify function after cleanup
          db connection set to db_conn to NULL to avoid reuse the same
          db connection again.

Note: Below is the backtrace pattern showing by brick process
      #0  0x00007ff98a30c1f7 in raise () from /lib64/libc.so.6
      #1  0x00007ff98a30d8e8 in abort () from /lib64/libc.so.6
      #2  0x00007ff98a34bf47 in __libc_message () from /lib64/libc.so.6
      #3  0x00007ff98a351b54 in malloc_printerr () from /lib64/libc.so.6
      #4  0x00007ff98a3537aa in _int_free () from /lib64/libc.so.6
      #5  0x00007ff97d95e311 in gf_sql_connection_fini (sql_connection=sql_connection@entry=0x7ff8e8496b50) at gfdb_sqlite3.c:42
      gluster#6  0x00007ff97d95e38a in gf_sqlite3_fini (db_conn=0x7ff92ca04470) at gfdb_sqlite3.c:507
      gluster#7  0x00007ff97d957156 in fini_db (_conn_node=0x7ff92ca04470) at gfdb_data_store.c:326
      gluster#8  0x00007ff97db78679 in notify (this=0x7ff92c5b3670, event=9, data=0x7ff92c5b5a00) at changetimerecorder.c:2178
      gluster#9  0x00007ff98bca0dc2 in xlator_notify (xl=0x7ff92c5b3670, event=event@entry=9, data=data@entry=0x7ff92c5b5a00) at xlator.c:549
      gluster#10 0x00007ff98bd3ac12 in default_notify (this=this@entry=0x7ff92c5b5a00, event=9, data=data@entry=0x7ff92c5b6d50) at defaults.c:3139

BUG: 1475632
Change-Id: Idd4bfdb4629c4799ac477ade81228065212683fb
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
Reviewed-on: https://review.gluster.org/17888
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
amarts referenced this pull request in amarts/glusterfs_fork Oct 31, 2017
Refcounting added for nfs call state in https://review.gluster.org/17696.
This is based on assumption that call state won't NULL when it is freed.
But currently gluster nfs server is crashing in different scenarios at
nfs3_getattr() with following bt

#0  0x00007ff1cfea9205 in _gf_ref_put (ref=ref@entry=0x0) at refcount.c:36
#1  0x00007ff1c1997455 in nfs3_call_state_wipe (cs=cs@entry=0x0) at nfs3.c:559
#2  0x00007ff1c1998931 in nfs3_getattr (req=req@entry=0x7ff1bc0b26d0, fh=fh@entry=0x7ff1c2f76ae0) at nfs3.c:962
#3  0x00007ff1c1998c8a in nfs3svc_getattr (req=0x7ff1bc0b26d0) at nfs3.c:987
#4  0x00007ff1cfbfd8c5 in rpcsvc_handle_rpc_call (svc=0x7ff1bc03e500, trans=trans@entry=0x7ff1bc0c8020, msg=<optimized out>) at rpcsvc.c:695
#5  0x00007ff1cfbfdaab in rpcsvc_notify (trans=0x7ff1bc0c8020, mydata=<optimized out>, event=<optimized out>, data=<optimized out>) at rpcsvc.c:789
gluster#6  0x00007ff1cfbff9e3 in rpc_transport_notify (this=this@entry=0x7ff1bc0c8020, event=event@entry=RPC_TRANSPORT_MSG_RECEIVED, data=data@entry=0x7ff1bc0038d0)
    at rpc-transport.c:538
gluster#7  0x00007ff1c4a2e3d6 in socket_event_poll_in (this=this@entry=0x7ff1bc0c8020, notify_handled=<optimized out>) at socket.c:2306
gluster#8  0x00007ff1c4a3097c in socket_event_handler (fd=21, idx=9, gen=19, data=0x7ff1bc0c8020, poll_in=1, poll_out=0, poll_err=0) at socket.c:2458
gluster#9  0x00007ff1cfe950f6 in event_dispatch_epoll_handler (event=0x7ff1c2f76e80, event_pool=0x5618154d5ee0) at event-epoll.c:572
gluster#10 event_dispatch_epoll_worker (data=0x56181551cbd0) at event-epoll.c:648
gluster#11 0x00007ff1cec99e25 in start_thread () from /lib64/libpthread.so.0
gluster#12 0x00007ff1ce56634d in clone () from /lib64/libc.so.6

This patch add previous NULL check move from __nfs3_call_state_wipe() to nfs3_call_state_wipe()

Change-Id: I2d73632f4be23f14d8467be3d908b09b3a2d87ea
BUG: 1479030
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
Reviewed-on: https://review.gluster.org/17989
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
gluster-ant pushed a commit that referenced this pull request Oct 16, 2018
Fixes below trace of ASan:

Direct leak of 130 byte(s) in 1 object(s) allocated from:
    #0 0x7fa794bb5850 in malloc (/lib64/libasan.so.4+0xde850)
    #1 0x7fa7944e5de9 in __gf_malloc ../../../libglusterfs/src/mem-pool.c:136
    #2 0x40b85c in gf_strndup ../../../libglusterfs/src/mem-pool.h:166
    #3 0x40b85c in gf_strdup ../../../libglusterfs/src/mem-pool.h:183
    #4 0x40b85c in parse_opts ../../../glusterfsd/src/glusterfsd.c:1049
    #5 0x7fa792a98720 in argp_parse (/lib64/libc.so.6+0x101720)
    #6 0x40d89f in parse_cmdline ../../../glusterfsd/src/glusterfsd.c:2041
    #7 0x406d07 in main ../../../glusterfsd/src/glusterfsd.c:2625


updates: bz#1633930
Change-Id: I394b3fc24b7a994c1b03635cb5e973e7290491d3
Signed-off-by: Amar Tumballi <amarts@redhat.com>
gluster-ant pushed a commit that referenced this pull request Nov 28, 2018
Direct leak of 609960 byte(s) in 4485 object(s) allocated from:
    #0 0x7f0d719bea50 in __interceptor_calloc (/lib64/libasan.so.5+0xefa50)
    #1 0x7f0d716dc08f in __gf_calloc ../../../libglusterfs/src/mem-pool.c:111
    #2 0x7f0d5d41d9b2 in __posix_get_mdata_xattr ../../../../../xlators/storage/posix/src/posix-metadata.c:240
    #3 0x7f0d5d41dd6b in posix_get_mdata_xattr ../../../../../xlators/storage/posix/src/posix-metadata.c:317
    #4 0x7f0d5d39e855 in posix_fdstat ../../../../../xlators/storage/posix/src/posix-helpers.c:685
    #5 0x7f0d5d3d65ec in posix_create ../../../../../xlators/storage/posix/src/posix-entry-ops.c:2173

Direct leak of 609960 byte(s) in 4485 object(s) allocated from:
    #0 0x7f0d719bea50 in __interceptor_calloc (/lib64/libasan.so.5+0xefa50)
    #1 0x7f0d716dc08f in __gf_calloc ../../../libglusterfs/src/mem-pool.c:111
    #2 0x7f0d5d41ced2 in posix_set_mdata_xattr ../../../../../xlators/storage/posix/src/posix-metadata.c:359
    #3 0x7f0d5d41e70f in posix_set_ctime ../../../../../xlators/storage/posix/src/posix-metadata.c:616
    #4 0x7f0d5d3d662c in posix_create ../../../../../xlators/storage/posix/src/posix-entry-ops.c:2181

We were freeing only the first context in inode during forget, and not the second.

updates: bz#1633930
Change-Id: Ib61b4453aa3d2039d6ce660f52ef45390539b9db
Signed-off-by: Amar Tumballi <amarts@redhat.com>
gluster-ant pushed a commit that referenced this pull request Dec 6, 2018
Tracebacks:

Direct leak of 96 byte(s) in 1 object(s) allocated from:
 #0 0x7f3acf9eac48 in malloc (/lib64/libasan.so.5+0xeec48)
 #1 0x7f3acf510949 in __gf_malloc ./libglusterfs/src/mem-pool.c:136
 #2 0x7f3acf5111bb in gf_vasprintf ./libglusterfs/src/mem-pool.c:236
 #3 0x7f3acf51138a in gf_asprintf ./libglusterfs/src/mem-pool.c:256
 #4 0x421611 in cli_cmd_volume_set_cbk ./cli/src/cli-cmd-volume.c:868
 #5 0x410599 in cli_cmd_process ./cli/src/cli-cmd.c:135
 #6 0x40f90d in cli_batch ./cli/src/input.c:29
 #7 0x7f3acd78c593 in start_thread pthread_create.c:463

Direct leak of 73 byte(s) in 1 object(s) allocated from:
 #0 0x7f3acf9eac48 in malloc (/lib64/libasan.so.5+0xeec48)
 #1 0x7f3acf510949 in __gf_malloc ./libglusterfs/src/mem-pool.c:136
 #2 0x421519 in gf_strndup ../../libglusterfs/src/mem-pool.h:167
 #3 0x421519 in gf_strdup ../../libglusterfs/src/mem-pool.h:184
 #4 0x421519 in cli_cmd_volume_set_cbk cli/src/cli-cmd-volume.c:859
 #5 0x410599 in cli_cmd_process cli/src/cli-cmd.c:135
 #6 0x40f90d in cli_batch cli/src/input.c:29
 #7 0x7f3acd78c593 in start_thread pthread_create.c:463

Change-Id: I3312751c1e3178672360a678fe15b1f7f1054b22
updates: bz#1633930
Signed-off-by: Kotresh HR <khiremat@redhat.com>
gluster-ant pushed a commit that referenced this pull request Dec 10, 2018
This patch fixes ememory leak reported by ASan.

Tracebacks:
Direct leak of 84 byte(s) in 1 object(s) allocated from:
    #0 0x7f71ea107848 in __interceptor_malloc (/lib64/libasan.so.5+0xef848)
    #1 0x7f71e9e2ac49 in __gf_malloc ./libglusterfs/src/mem-pool.c:136
    #2 0x7f71e9e2b4bb in gf_vasprintf ./libglusterfs/src/mem-pool.c:236
    #3 0x7f71e9e2b68a in gf_asprintf ./libglusterfs/src/mem-pool.c:256
    #4 0x41e8ec in cli_cmd_bitrot_cbk ./cli/src/cli-cmd-volume.c:1847
    #5 0x410b39 in cli_cmd_process ./cli/src/cli-cmd.c:137
    #6 0x40fe9d in cli_batch ./cli/src/input.c:29
    #7 0x7f71e989558d in start_thread (/lib64/libpthread.so.0+0x858d)

updates: bz#1633930
Change-Id: I8977e45add742e67047291f398f0ee79eb09afe4
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
dmantipov added a commit to dmantipov/glusterfs that referenced this pull request Aug 17, 2022
Fix 'qemu-img' crash discovered as follows:

$ gluster volume info test0

Volume Name: test0
Type: Distribute
Volume ID: dc5607a7-fadc-42fd-a532-de0b791097ef
Status: Started
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 192.168.222.111:/pool/0
Brick2: 192.168.222.111:/pool/1
Brick3: 192.168.222.111:/pool/2
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on

$ qemu-img info gluster://192.168.222.111/test0/test0.img
[2022-08-17 08:15:46.704459 +0000] I [io-stats.c:3797:ios_sample_buf_size_configure]
0-test0: Configure ios_sample_buf  size is 1024 because ios_sample_interval is 0
Segmentation fault (core dumped)

$ gdb -q qemu-img core
...
Program terminated with signal SIGSEGV, Segmentation fault.
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
655	    GF_ATOMIC_INC(this->refcount);
...
(gdb) p *this
Cannot access memory at address 0x48003278
(gdb) bt 4
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
gluster#1  0x00007fb96f34e695 in syncop_seek_cbk (frame=frame@entry=0x55a04de5a2c8, cookie=0x7ffea4b96340,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at syncop.c:3167
gluster#2  0x00007fb9669e7a42 in io_stats_seek_cbk (frame=frame@entry=0x55a04de5a3b8, cookie=<optimized out>,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at io-stats.c:2610
gluster#3  0x00007fb96f39d47d in default_seek_cbk (frame=0x55a04de5b698, cookie=<optimized out>,
    this=<optimized out>, op_ret=-1, op_errno=77, offset=0, xdata=0x48003278) at defaults.c:1615
gluster#4  0x00007fb96c174f47 in client4_0_seek (frame=0x7fb948000eb8, this=<optimized out>, data=<optimized out>)
    at client-rpc-fops_v2.c:5299
(More stack frames follow...)

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: gluster#1000
amarts pushed a commit that referenced this pull request Aug 18, 2022
Fix 'qemu-img' crash discovered as follows:

$ gluster volume info test0

Volume Name: test0
Type: Distribute
Volume ID: dc5607a7-fadc-42fd-a532-de0b791097ef
Status: Started
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 192.168.222.111:/pool/0
Brick2: 192.168.222.111:/pool/1
Brick3: 192.168.222.111:/pool/2
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on

$ qemu-img info gluster://192.168.222.111/test0/test0.img
[2022-08-17 08:15:46.704459 +0000] I [io-stats.c:3797:ios_sample_buf_size_configure]
0-test0: Configure ios_sample_buf  size is 1024 because ios_sample_interval is 0
Segmentation fault (core dumped)

$ gdb -q qemu-img core
...
Program terminated with signal SIGSEGV, Segmentation fault.
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
655	    GF_ATOMIC_INC(this->refcount);
...
(gdb) p *this
Cannot access memory at address 0x48003278
(gdb) bt 4
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
#1  0x00007fb96f34e695 in syncop_seek_cbk (frame=frame@entry=0x55a04de5a2c8, cookie=0x7ffea4b96340,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at syncop.c:3167
#2  0x00007fb9669e7a42 in io_stats_seek_cbk (frame=frame@entry=0x55a04de5a3b8, cookie=<optimized out>,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at io-stats.c:2610
#3  0x00007fb96f39d47d in default_seek_cbk (frame=0x55a04de5b698, cookie=<optimized out>,
    this=<optimized out>, op_ret=-1, op_errno=77, offset=0, xdata=0x48003278) at defaults.c:1615
#4  0x00007fb96c174f47 in client4_0_seek (frame=0x7fb948000eb8, this=<optimized out>, data=<optimized out>)
    at client-rpc-fops_v2.c:5299
(More stack frames follow...)

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000
dmantipov added a commit to dmantipov/glusterfs that referenced this pull request Aug 18, 2022
Fix 'qemu-img' crash discovered as follows:

$ gluster volume info test0

Volume Name: test0
Type: Distribute
Volume ID: dc5607a7-fadc-42fd-a532-de0b791097ef
Status: Started
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 192.168.222.111:/pool/0
Brick2: 192.168.222.111:/pool/1
Brick3: 192.168.222.111:/pool/2
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on

$ qemu-img info gluster://192.168.222.111/test0/test0.img
[2022-08-17 08:15:46.704459 +0000] I [io-stats.c:3797:ios_sample_buf_size_configure]
0-test0: Configure ios_sample_buf  size is 1024 because ios_sample_interval is 0
Segmentation fault (core dumped)

$ gdb -q qemu-img core
...
Program terminated with signal SIGSEGV, Segmentation fault.
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
655         GF_ATOMIC_INC(this->refcount);
...
(gdb) p *this
Cannot access memory at address 0x48003278
(gdb) bt 4
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
gluster#1  0x00007fb96f34e695 in syncop_seek_cbk (frame=frame@entry=0x55a04de5a2c8, cookie=0x7ffea4b96340,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at syncop.c:3167
gluster#2  0x00007fb9669e7a42 in io_stats_seek_cbk (frame=frame@entry=0x55a04de5a3b8, cookie=<optimized out>,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at io-stats.c:2610
gluster#3  0x00007fb96f39d47d in default_seek_cbk (frame=0x55a04de5b698, cookie=<optimized out>,
    this=<optimized out>, op_ret=-1, op_errno=77, offset=0, xdata=0x48003278) at defaults.c:1615
gluster#4  0x00007fb96c174f47 in client4_0_seek (frame=0x7fb948000eb8, this=<optimized out>, data=<optimized out>)
    at client-rpc-fops_v2.c:5299
(More stack frames follow...)

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: gluster#1000
dmantipov added a commit to dmantipov/glusterfs that referenced this pull request Aug 19, 2022
Workaround (actually not a fix, see below) 'qemu-img' hang
(caused by an undefined behavior) comparing two gluster://
images. Discovered as follows:

Storage configuration:

Volume Name: test0
Type: Distribute
Volume ID: 2930e059-29b3-440f-9a45-d5da735aa8cf
Status: Started
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 192.168.222.111:/pool/0
Brick2: 192.168.222.111:/pool/1
Brick3: 192.168.222.111:/pool/2
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on

Volume Name: test1
Type: Distribute
Volume ID: 5531b9fe-59cc-4b7d-a47a-7d3097a5cedb
Status: Started
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 192.168.222.111:/pool/3
Brick2: 192.168.222.111:/pool/4
Brick3: 192.168.222.111:/pool/5
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on

Test:

qemu-img create gluster://192.168.222.111/test0/test0.img 16G
qemu-img create gluster://192.168.222.111/test1/test1.img 16G
qemu-img compare gluster://192.168.222.111/test0/test0.img gluster://192.168.222.111/test1/test1.img

Issue:

# gdb -q /usr/local/sbin/qemu-img
Reading symbols from /usr/local/sbin/qemu-img...
(gdb) b rpc_clnt_destroy
Function "rpc_clnt_destroy" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (rpc_clnt_destroy) pending.
(gdb) r compare gluster://192.168.222.111/test0/test0.img gluster://192.168.222.111/test1/test1.img
Starting program: /usr/local/sbin/qemu-img compare gluster://192.168.222.111/test0/test0.img gluster://192.168.222.111/test1/test1.img
...
Thread 14 "glfs_timer" hit Breakpoint 1, rpc_clnt_destroy (rpc=0x7fffc003b808) at rpc-clnt.c:1741
1741	    rpcclnt_cb_program_t *program = NULL;
(gdb) bt
#0  rpc_clnt_destroy (rpc=0x7fffc003b808) at rpc-clnt.c:1741                    ;; RPC object 0x7fffc003b808 is going to be freed
gluster#1  0x00007ffff6bceba8 in rpc_clnt_notify (trans=0x7fffc003b9b8, mydata=0x7fffc003b838,
    event=RPC_TRANSPORT_CLEANUP, data=0x0) at rpc-clnt.c:875
gluster#2  0x00007ffff6bcac3d in rpc_transport_unref (this=0x7fffc003b9b8) at rpc-transport.c:502
gluster#3  0x00007ffff6bd0948 in rpc_clnt_trigger_destroy (rpc=0x7fffc003b808) at rpc-clnt.c:1734
gluster#4  0x00007ffff6bd0aff in rpc_clnt_unref (rpc=0x7fffc003b808) at rpc-clnt.c:1792
gluster#5  0x00007ffff6bcd3a4 in call_bail (data=0x7fffc003b808) at rpc-clnt.c:190
gluster#6  0x00007ffff6a93c5b in gf_timer_proc (data=0x5555561dab58) at timer.c:152
gluster#7  0x00007ffff77cbfdd in start_thread (arg=<optimized out>) at pthread_create.c:442
gluster#8  0x00007ffff784d5a0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) p rpc->refcount
$1 = {lk = 0x7fffc003b968 "", value = 0}
(gdb) p rpc->ctx->cleanup_started
$2 = 1 '\001'
(gdb) b rpc_clnt_start_ping
Breakpoint 2 at 0x7ffff6bd464b: file rpc-clnt-ping.c, line 281.
(gdb) c
Continuing.
...
Thread 14 "glfs_timer" hit Breakpoint 2, rpc_clnt_start_ping (rpc_ptr=0x7fffc003b808) at rpc-clnt-ping.c:281
281	    struct rpc_clnt *rpc = NULL;
(gdb) bt
#0  rpc_clnt_start_ping (rpc_ptr=0x7fffc003b808) at rpc-clnt-ping.c:281         ;; RPC object 0x7fffc003b808 is used after free
gluster#1  0x00007ffff6a93c5b in gf_timer_proc (data=0x5555561dab58) at timer.c:152
gluster#2  0x00007ffff77cbfdd in start_thread (arg=<optimized out>) at pthread_create.c:442
gluster#3  0x00007ffff784d5a0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

The problem is when ctx->cleanup_started is true, gf_timer_call_cancel()
refuses to remove RPC client's ping timer (rpc->conn.ping_timer), and
the latter ticks _after_ the corresponding RPC client object is destroyed.

This issue raises a design questions: what should be done when GFAPI
client calls 'glfs_fini()' but there is an internal timer scheduled
10 seconds thereafter? Should 'glfs_fini()' cancel any internal timers
at the very beginning? I'm not sure, but the quick and dirty workaround
is to indicate success returned from gf_timer_call_cancel() but mark the
corresponding timer as canceled and do not call its callback before
freeing the timer itself.

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: gluster#1000
Shwetha-Acharya pushed a commit to Shwetha-Acharya/glusterfs that referenced this pull request Aug 23, 2022
Fix 'qemu-img' crash discovered as follows:

$ gluster volume info test0

Volume Name: test0
Type: Distribute
Volume ID: dc5607a7-fadc-42fd-a532-de0b791097ef
Status: Started
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 192.168.222.111:/pool/0
Brick2: 192.168.222.111:/pool/1
Brick3: 192.168.222.111:/pool/2
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on

$ qemu-img info gluster://192.168.222.111/test0/test0.img
[2022-08-17 08:15:46.704459 +0000] I [io-stats.c:3797:ios_sample_buf_size_configure]
0-test0: Configure ios_sample_buf  size is 1024 because ios_sample_interval is 0
Segmentation fault (core dumped)

$ gdb -q qemu-img core
...
Program terminated with signal SIGSEGV, Segmentation fault.
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
655	    GF_ATOMIC_INC(this->refcount);
...
(gdb) p *this
Cannot access memory at address 0x48003278
(gdb) bt 4
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
gluster#1  0x00007fb96f34e695 in syncop_seek_cbk (frame=frame@entry=0x55a04de5a2c8, cookie=0x7ffea4b96340,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at syncop.c:3167
gluster#2  0x00007fb9669e7a42 in io_stats_seek_cbk (frame=frame@entry=0x55a04de5a3b8, cookie=<optimized out>,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at io-stats.c:2610
gluster#3  0x00007fb96f39d47d in default_seek_cbk (frame=0x55a04de5b698, cookie=<optimized out>,
    this=<optimized out>, op_ret=-1, op_errno=77, offset=0, xdata=0x48003278) at defaults.c:1615
gluster#4  0x00007fb96c174f47 in client4_0_seek (frame=0x7fb948000eb8, this=<optimized out>, data=<optimized out>)
    at client-rpc-fops_v2.c:5299
(More stack frames follow...)

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: gluster#1000
Shwetha-Acharya pushed a commit to Shwetha-Acharya/glusterfs that referenced this pull request Aug 23, 2022
Fix 'qemu-img' crash discovered as follows:

$ gluster volume info test0

Volume Name: test0
Type: Distribute
Volume ID: dc5607a7-fadc-42fd-a532-de0b791097ef
Status: Started
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 192.168.222.111:/pool/0
Brick2: 192.168.222.111:/pool/1
Brick3: 192.168.222.111:/pool/2
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on

$ qemu-img info gluster://192.168.222.111/test0/test0.img
[2022-08-17 08:15:46.704459 +0000] I [io-stats.c:3797:ios_sample_buf_size_configure]
0-test0: Configure ios_sample_buf  size is 1024 because ios_sample_interval is 0
Segmentation fault (core dumped)

$ gdb -q qemu-img core
...
Program terminated with signal SIGSEGV, Segmentation fault.
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
655	    GF_ATOMIC_INC(this->refcount);
...
(gdb) p *this
Cannot access memory at address 0x48003278
(gdb) bt 4
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
gluster#1  0x00007fb96f34e695 in syncop_seek_cbk (frame=frame@entry=0x55a04de5a2c8, cookie=0x7ffea4b96340,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at syncop.c:3167
gluster#2  0x00007fb9669e7a42 in io_stats_seek_cbk (frame=frame@entry=0x55a04de5a3b8, cookie=<optimized out>,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at io-stats.c:2610
gluster#3  0x00007fb96f39d47d in default_seek_cbk (frame=0x55a04de5b698, cookie=<optimized out>,
    this=<optimized out>, op_ret=-1, op_errno=77, offset=0, xdata=0x48003278) at defaults.c:1615
gluster#4  0x00007fb96c174f47 in client4_0_seek (frame=0x7fb948000eb8, this=<optimized out>, data=<optimized out>)
    at client-rpc-fops_v2.c:5299
(More stack frames follow...)

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: gluster#1000
Shwetha-Acharya pushed a commit to Shwetha-Acharya/glusterfs that referenced this pull request Aug 23, 2022
Fix 'qemu-img' crash discovered as follows:

$ gluster volume info test0

Volume Name: test0
Type: Distribute
Volume ID: dc5607a7-fadc-42fd-a532-de0b791097ef
Status: Started
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 192.168.222.111:/pool/0
Brick2: 192.168.222.111:/pool/1
Brick3: 192.168.222.111:/pool/2
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on

$ qemu-img info gluster://192.168.222.111/test0/test0.img
[2022-08-17 08:15:46.704459 +0000] I [io-stats.c:3797:ios_sample_buf_size_configure]
0-test0: Configure ios_sample_buf  size is 1024 because ios_sample_interval is 0
Segmentation fault (core dumped)

$ gdb -q qemu-img core
...
Program terminated with signal SIGSEGV, Segmentation fault.
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
655	    GF_ATOMIC_INC(this->refcount);
...
(gdb) p *this
Cannot access memory at address 0x48003278
(gdb) bt 4
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
gluster#1  0x00007fb96f34e695 in syncop_seek_cbk (frame=frame@entry=0x55a04de5a2c8, cookie=0x7ffea4b96340,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at syncop.c:3167
gluster#2  0x00007fb9669e7a42 in io_stats_seek_cbk (frame=frame@entry=0x55a04de5a3b8, cookie=<optimized out>,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at io-stats.c:2610
gluster#3  0x00007fb96f39d47d in default_seek_cbk (frame=0x55a04de5b698, cookie=<optimized out>,
    this=<optimized out>, op_ret=-1, op_errno=77, offset=0, xdata=0x48003278) at defaults.c:1615
gluster#4  0x00007fb96c174f47 in client4_0_seek (frame=0x7fb948000eb8, this=<optimized out>, data=<optimized out>)
    at client-rpc-fops_v2.c:5299
(More stack frames follow...)

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: gluster#1000
Shwetha-Acharya pushed a commit that referenced this pull request Aug 23, 2022
Fix 'qemu-img' crash discovered as follows:

$ gluster volume info test0

Volume Name: test0
Type: Distribute
Volume ID: dc5607a7-fadc-42fd-a532-de0b791097ef
Status: Started
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 192.168.222.111:/pool/0
Brick2: 192.168.222.111:/pool/1
Brick3: 192.168.222.111:/pool/2
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on

$ qemu-img info gluster://192.168.222.111/test0/test0.img
[2022-08-17 08:15:46.704459 +0000] I [io-stats.c:3797:ios_sample_buf_size_configure]
0-test0: Configure ios_sample_buf  size is 1024 because ios_sample_interval is 0
Segmentation fault (core dumped)

$ gdb -q qemu-img core
...
Program terminated with signal SIGSEGV, Segmentation fault.
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
655         GF_ATOMIC_INC(this->refcount);
...
(gdb) p *this
Cannot access memory at address 0x48003278
(gdb) bt 4
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
#1  0x00007fb96f34e695 in syncop_seek_cbk (frame=frame@entry=0x55a04de5a2c8, cookie=0x7ffea4b96340,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at syncop.c:3167
#2  0x00007fb9669e7a42 in io_stats_seek_cbk (frame=frame@entry=0x55a04de5a3b8, cookie=<optimized out>,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at io-stats.c:2610
#3  0x00007fb96f39d47d in default_seek_cbk (frame=0x55a04de5b698, cookie=<optimized out>,
    this=<optimized out>, op_ret=-1, op_errno=77, offset=0, xdata=0x48003278) at defaults.c:1615
#4  0x00007fb96c174f47 in client4_0_seek (frame=0x7fb948000eb8, this=<optimized out>, data=<optimized out>)
    at client-rpc-fops_v2.c:5299
(More stack frames follow...)

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Shwetha-Acharya added a commit that referenced this pull request Sep 16, 2022
Fix 'qemu-img' crash discovered as follows:

$ gluster volume info test0

Volume Name: test0
Type: Distribute
Volume ID: dc5607a7-fadc-42fd-a532-de0b791097ef
Status: Started
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 192.168.222.111:/pool/0
Brick2: 192.168.222.111:/pool/1
Brick3: 192.168.222.111:/pool/2
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on

$ qemu-img info gluster://192.168.222.111/test0/test0.img
[2022-08-17 08:15:46.704459 +0000] I [io-stats.c:3797:ios_sample_buf_size_configure]
0-test0: Configure ios_sample_buf  size is 1024 because ios_sample_interval is 0
Segmentation fault (core dumped)

$ gdb -q qemu-img core
...
Program terminated with signal SIGSEGV, Segmentation fault.
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
655	    GF_ATOMIC_INC(this->refcount);
...
(gdb) p *this
Cannot access memory at address 0x48003278
(gdb) bt 4
#0  dict_ref (this=this@entry=0x48003278) at dict.c:655
#1  0x00007fb96f34e695 in syncop_seek_cbk (frame=frame@entry=0x55a04de5a2c8, cookie=0x7ffea4b96340,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at syncop.c:3167
#2  0x00007fb9669e7a42 in io_stats_seek_cbk (frame=frame@entry=0x55a04de5a3b8, cookie=<optimized out>,
    this=<optimized out>, op_ret=op_ret@entry=-1, op_errno=op_errno@entry=77, offset=offset@entry=0,
    xdata=0x48003278) at io-stats.c:2610
#3  0x00007fb96f39d47d in default_seek_cbk (frame=0x55a04de5b698, cookie=<optimized out>,
    this=<optimized out>, op_ret=-1, op_errno=77, offset=0, xdata=0x48003278) at defaults.c:1615
#4  0x00007fb96c174f47 in client4_0_seek (frame=0x7fb948000eb8, this=<optimized out>, data=<optimized out>)
    at client-rpc-fops_v2.c:5299
(More stack frames follow...)

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Co-authored-by: Dmitry Antipov <dantipov@cloudlinux.com>
csabahenk pushed a commit to csabahenk/glusterfs that referenced this pull request Mar 7, 2023
The general idea of the changes is to prevent resetting event generation
to zero in the inode ctx, since event gen is something that should
follow 'causal order'.

Change gluster#1:
For a read txn, in inode refresh cbk, if event_generation is
found zero, we are failing the read fop. This is not needed
because change in event gen is only a marker for the next inode refresh to
happen and should not be taken into account by the current read txn.

Change gluster#2:
The event gen being zero above can happen if there is a racing lookup,
which resets even get (in afr_lookup_done) if there are non zero afr
xattrs. The resetting is done only to trigger an inode refresh and a
possible client side heal on the next lookup. That can be acheived by
setting the need_refresh flag in the inode ctx. So replaced all
occurences of resetting even gen to zero with a call to
afr_inode_need_refresh_set().

Change gluster#3:
In both lookup and discover path, we are doing an inode refresh which is
not required since all 3 essentially do the same thing- update the inode
ctx with the good/bad copies from the brick replies. Inode refresh also
triggers background heals, but I think it is okay to do it when we call
refresh during the read and write txns and not in the lookup path.

The .ts which relied on inode refresh in lookup path to trigger heals are
now changed to do read txn so that inode refresh and the heal happens.

Upstream patch details:
> Change-Id: Iebf39a9be6ffd7ffd6e4046c96b0fa78ade6c5ec
> Fixes: gluster#1179
> Signed-off-by: Ravishankar N <ravishankar@redhat.com>
> Reported-by: Erik Jacobson <erik.jacobson at hpe.com>
Upstream patch: https://review.gluster.org/#/c/glusterfs/+/24316/

BUG: 1640148
Change-Id: Iebf39a9be6ffd7ffd6e4046c96b0fa78ade6c5ec
Signed-off-by: karthik-us <ksubrahm@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/222074
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Ravishankar Narayanankutty <ravishankar@redhat.com>
mohit84 added a commit to mohit84/glusterfs that referenced this pull request Oct 20, 2023
The client is throwing below stacktrace while asan is enabled.
The client is facing an issue while application is trying
to call removexattr in 2x1 subvol and non-mds subvol is down.
As we can see in below stacktrace dht_setxattr_mds_cbk is calling
dht_setxattr_non_mds_cbk and dht_setxattr_non_mds_cbk is trying to
wipe local because call_cnt is 0 but dht_setxattr_mds_cbk is trying
to access frame->local that;s why it is crashed.

x621000051c34 is located 1844 bytes inside of 4164-byte region [0x621000051500,0x621000052544)
freed by thread T7 here:
    #0 0x7f916ccb9388 in __interceptor_free.part.0 (/lib64/libasan.so.8+0xb9388)
    GlusterFS#1 0x7f91654af204 in dht_local_wipe /root/glusterfs_new/glusterfs/xlators/cluster/dht/src/dht-helper.c:713
    gluster#2 0x7f91654af204 in dht_setxattr_non_mds_cbk /root/glusterfs_new/glusterfs/xlators/cluster/dht/src/dht-common.c:3900
    gluster#3 0x7f91694c1f42 in client4_0_removexattr_cbk /root/glusterfs_new/glusterfs/xlators/protocol/client/src/client-rpc-fops_v2.c:1061
    gluster#4 0x7f91694ba26f in client_submit_request /root/glusterfs_new/glusterfs/xlators/protocol/client/src/client.c:288
    gluster#5 0x7f91695021bd in client4_0_removexattr /root/glusterfs_new/glusterfs/xlators/protocol/client/src/client-rpc-fops_v2.c:4480
    gluster#6 0x7f91694a5f56 in client_removexattr /root/glusterfs_new/glusterfs/xlators/protocol/client/src/client.c:1439
    gluster#7 0x7f91654a1161 in dht_setxattr_mds_cbk /root/glusterfs_new/glusterfs/xlators/cluster/dht/src/dht-common.c:3979
    gluster#8 0x7f91694c1f42 in client4_0_removexattr_cbk /root/glusterfs_new/glusterfs/xlators/protocol/client/src/client-rpc-fops_v2.c:1061
    gluster#9 0x7f916cbc4340 in rpc_clnt_handle_reply /root/glusterfs_new/glusterfs/rpc/rpc-lib/src/rpc-clnt.c:723
    gluster#10 0x7f916cbc4340 in rpc_clnt_notify /root/glusterfs_new/glusterfs/rpc/rpc-lib/src/rpc-clnt.c:890
    gluster#11 0x7f916cbb7ec5 in rpc_transport_notify /root/glusterfs_new/glusterfs/rpc/rpc-lib/src/rpc-transport.c:504
    gluster#12 0x7f916a1aa5fa in socket_event_poll_in_async /root/glusterfs_new/glusterfs/rpc/rpc-transport/socket/src/socket.c:2358
    gluster#13 0x7f916a1bd7c2 in gf_async ../../../../libglusterfs/src/glusterfs/async.h:187
    gluster#14 0x7f916a1bd7c2 in socket_event_poll_in /root/glusterfs_new/glusterfs/rpc/rpc-transport/socket/src/socket.c:2399
    gluster#15 0x7f916a1bd7c2 in socket_event_handler /root/glusterfs_new/glusterfs/rpc/rpc-transport/socket/src/socket.c:2790
    gluster#16 0x7f916a1bd7c2 in socket_event_handler /root/glusterfs_new/glusterfs/rpc/rpc-transport/socket/src/socket.c:2710
    gluster#17 0x7f916c946d22 in event_dispatch_epoll_handler /root/glusterfs_new/glusterfs/libglusterfs/src/event-epoll.c:614
    gluster#18 0x7f916c946d22 in event_dispatch_epoll_worker /root/glusterfs_new/glusterfs/libglusterfs/src/event-epoll.c:725
    gluster#19 0x7f916be8cdec in start_thread (/lib64/libc.so.6+0x8cdec)

Solution: Use switch instead of using if statement to wind a operation, in case of switch
          the code will not try to access local after wind a operation for last dht
          subvol.

Fixes: gluster#3732
Change-Id: I031bc814d6df98058430ef4de7040e3370d1c677
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
mohit84 added a commit that referenced this pull request Oct 23, 2023
The client is throwing below stacktrace while asan is enabled.
The client is facing an issue while application is trying
to call removexattr in 2x1 subvol and non-mds subvol is down.
As we can see in below stacktrace dht_setxattr_mds_cbk is calling
dht_setxattr_non_mds_cbk and dht_setxattr_non_mds_cbk is trying to
wipe local because call_cnt is 0 but dht_setxattr_mds_cbk is trying
to access frame->local that;s why it is crashed.

x621000051c34 is located 1844 bytes inside of 4164-byte region [0x621000051500,0x621000052544)
freed by thread T7 here:
    #0 0x7f916ccb9388 in __interceptor_free.part.0 (/lib64/libasan.so.8+0xb9388)
    #1 0x7f91654af204 in dht_local_wipe /root/glusterfs_new/glusterfs/xlators/cluster/dht/src/dht-helper.c:713
    #2 0x7f91654af204 in dht_setxattr_non_mds_cbk /root/glusterfs_new/glusterfs/xlators/cluster/dht/src/dht-common.c:3900
    #3 0x7f91694c1f42 in client4_0_removexattr_cbk /root/glusterfs_new/glusterfs/xlators/protocol/client/src/client-rpc-fops_v2.c:1061
    #4 0x7f91694ba26f in client_submit_request /root/glusterfs_new/glusterfs/xlators/protocol/client/src/client.c:288
    #5 0x7f91695021bd in client4_0_removexattr /root/glusterfs_new/glusterfs/xlators/protocol/client/src/client-rpc-fops_v2.c:4480
    #6 0x7f91694a5f56 in client_removexattr /root/glusterfs_new/glusterfs/xlators/protocol/client/src/client.c:1439
    #7 0x7f91654a1161 in dht_setxattr_mds_cbk /root/glusterfs_new/glusterfs/xlators/cluster/dht/src/dht-common.c:3979
    #8 0x7f91694c1f42 in client4_0_removexattr_cbk /root/glusterfs_new/glusterfs/xlators/protocol/client/src/client-rpc-fops_v2.c:1061
    #9 0x7f916cbc4340 in rpc_clnt_handle_reply /root/glusterfs_new/glusterfs/rpc/rpc-lib/src/rpc-clnt.c:723
    #10 0x7f916cbc4340 in rpc_clnt_notify /root/glusterfs_new/glusterfs/rpc/rpc-lib/src/rpc-clnt.c:890
    #11 0x7f916cbb7ec5 in rpc_transport_notify /root/glusterfs_new/glusterfs/rpc/rpc-lib/src/rpc-transport.c:504
    #12 0x7f916a1aa5fa in socket_event_poll_in_async /root/glusterfs_new/glusterfs/rpc/rpc-transport/socket/src/socket.c:2358
    #13 0x7f916a1bd7c2 in gf_async ../../../../libglusterfs/src/glusterfs/async.h:187
    #14 0x7f916a1bd7c2 in socket_event_poll_in /root/glusterfs_new/glusterfs/rpc/rpc-transport/socket/src/socket.c:2399
    #15 0x7f916a1bd7c2 in socket_event_handler /root/glusterfs_new/glusterfs/rpc/rpc-transport/socket/src/socket.c:2790
    #16 0x7f916a1bd7c2 in socket_event_handler /root/glusterfs_new/glusterfs/rpc/rpc-transport/socket/src/socket.c:2710
    #17 0x7f916c946d22 in event_dispatch_epoll_handler /root/glusterfs_new/glusterfs/libglusterfs/src/event-epoll.c:614
    #18 0x7f916c946d22 in event_dispatch_epoll_worker /root/glusterfs_new/glusterfs/libglusterfs/src/event-epoll.c:725
    #19 0x7f916be8cdec in start_thread (/lib64/libc.so.6+0x8cdec)

Solution: Use switch instead of using if statement to wind a operation, in case of switch
          the code will not try to access local after wind a operation for last dht
          subvol.

Fixes: #3732
Change-Id: I031bc814d6df98058430ef4de7040e3370d1c677

Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants