Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

glusterfs: write operations fail when the size is equal or greater than 1 GB #1518

Closed
rkothiya opened this issue Sep 27, 2020 · 2 comments · Fixed by #1557
Closed

glusterfs: write operations fail when the size is equal or greater than 1 GB #1518

rkothiya opened this issue Sep 27, 2020 · 2 comments · Fixed by #1557
Labels
DocApproved Mandatory flag to pass smoke. Provide user facing document to get approval. SpecApproved This is a mandatory flag for passing the smoke for feature. Provide spec of feature to get approval Type:Bug

Comments

@rkothiya
Copy link

rkothiya commented Sep 27, 2020 •
edited

Description of problem:

Write APIs (eg. glfs_write(), glfs_pwrite() or *async()) with a size >= 1GB there is a strange behaviour:

- an error is printed in the client log (no error in the server log)
- glfs_write() doesn't return any error, but the write operation is not executed
- subsequent operations fail with "Transport endpoint is not connected"

Version-Release number of selected component (if applicable):
glusterfs-server-3.12.2-40.el7rhgs.x84_64

How reproducible:
100%

Steps to Reproduce:

  1. Change server, volume and path in the glfs_write_bug.c
  2. gcc `pkg-config --cflags glusterfs-api` `pkg-config --libs glusterfs-api` glfs_write_bug.c -o glfs_write_bug
  3. ./glfs_write_bug

Actual results:
TEST glfs_write - size: 512 MiB pattern: 171
glfs_write - size: 536870912 ret: 536870912
glfs_read - size: 1024 ret: 1024
PASS
TEST glfs_write - size: 1023 MiB pattern: 42
glfs_write - size: 1072693248 ret: 1072693248
glfs_read - size: 1024 ret: 1024
PASS
TEST glfs_write - size: 1024 MiB pattern: 203
glfs_write - size: 1073741824 ret: 1073741824
[2019-03-21 10:43:24.607883] E [rpc-clnt.c:346:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x131)[0x7f436bd83131] (--> /lib64/libgfrpc.so.0(+0xda01)[0x7f436bd48a01] (--> /lib64/libgfrpc.so.0(+0xdb22)[0x7f436bd48b22] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x94)[0x7f436bd4a0b4] (--> /lib64/libgfrpc.so.0(+0xfc50)[0x7f436bd4ac50] ))))) 0-gv33-client-0: forced unwinding frame type(GlusterFS 3.3) op(WRITE(13)) called at 2019-03-21 10:43:24.607433 (xid=0x11)
glfs_read - size: 1024 ret: -1
glfs_read: Transport endpoint is not connected
END ret=-1

Expected results:
TEST glfs_write - size: 512 MiB pattern: 171
glfs_write - size: 536870912 ret: 536870912
glfs_read - size: 1024 ret: 1024
PASS
TEST glfs_write - size: 1023 MiB pattern: 42
glfs_write - size: 1072693248 ret: 1072693248
glfs_read - size: 1024 ret: 1024
PASS
TEST glfs_write - size: 1024 MiB pattern: 203
glfs_write - size: 1073741824 ret: 1073741824
glfs_read - size: 1024 ret: 1024
END ret=0

Additional info:
As clients, I used both RHEL8 (glusterfs-api-3.12.2-40.2.el8.x86_64) and F29 (glusterfs-api-5.5-1.fc29.x86_64)

I had the same issue also using a Fedora 29 server (glusterfs-server-5.5-1.fc29.x86_64):
TEST glfs_write - size: 1024 MiB pattern: 172
glfs_write - size: 1073741824 ret: 1073741824
[2019-03-21 10:55:08.998979] E [rpc-clnt.c:346:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x131)[0x7fbf924ec131] (--> /lib64/libgfrpc.so.0(+0xda01)[0x7fbf924b1a01] (--> /lib64/libgfrpc.so.0(+0xdb22)[0x7fbf924b1b22] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x94)[0x7fbf924b30b4] (--> /lib64/libgfrpc.so.0(+0xfc50)[0x7fbf924b3c50] ))))) 0-gv0-client-0: forced unwinding frame type(GlusterFS 4.x v1) op(WRITE(13)) called at 2019-03-21 10:55:08.998126 (xid=0x11)
glfs_read - size: 1024 ret: -1
glfs_read: Transport endpoint is not connected
@rkothiya
Copy link
Author

glfs_write_bug.c useful to reproduce the bug
glfs_write_bug.tar.gz

@gluster-ant
Copy link
Collaborator

A patch https://review.gluster.org/25035 has been posted that references this issue.

core: give appropriate error when size exceeds

This patch help generate appropriate error message
when the gfapi tries to write data equal to or
greater than 1 Gb

fixes: #1518

Change-Id: I1234a0b5a6e675a0b20c6b1afe0f4390fd721f6f
Signed-off-by: Rinku Kothiya rkothiya@redhat.com

@rkothiya rkothiya added Type:Bug SpecApproved This is a mandatory flag for passing the smoke for feature. Provide spec of feature to get approval DocApproved Mandatory flag to pass smoke. Provide user facing document to get approval. labels Sep 30, 2020
rkothiya pushed a commit to rkothiya/glusterfs that referenced this issue Oct 7, 2020
gfapi: give appropriate error when size exceeds
c91c903 
This patch help generate appropriate error message
when the gfapi tries to write data equal to or
greater than 1 Gb due to the limitation at the
socket layer.

fixes: gluster#1518

Change-Id: I1234a0b5a6e675a0b20c6b1afe0f4390fd721f6f
Signed-off-by: Rinku Kothiya <rkothiya@redhat.com>
@rkothiya rkothiya mentioned this issue Oct 7, 2020
Merged
xhernandez pushed a commit that referenced this issue Nov 3, 2020
@xhernandez
gfapi: give appropriate error when size exceeds
7ccb1f1 
This patch help generate appropriate error message
when the gfapi tries to write data equal to or
greater than 1 Gb due to the limitation at the
socket layer.

fixes: #1518

Change-Id: I1234a0b5a6e675a0b20c6b1afe0f4390fd721f6f
Signed-off-by: Rinku Kothiya <rkothiya@redhat.com>
csabahenk pushed a commit to csabahenk/glusterfs that referenced this issue Mar 7, 2023
@sunilheggodu
gfapi: give appropriate error when size exceeds
f78a5d8 
This patch help generate appropriate error message
when the gfapi tries to write data equal to or
greater than 1 Gb due to the limitation at the
socket layer.

Upstream:
> Reviewed-on: gluster#1557
> fixes: gluster#1518
> Change-Id: I1234a0b5a6e675a0b20c6b1afe0f4390fd721f6f
> Signed-off-by: Rinku Kothiya <rkothiya@redhat.com>

BUG: 1691320
Change-Id: I1234a0b5a6e675a0b20c6b1afe0f4390fd721f6f
Signed-off-by: Rinku Kothiya <rkothiya@redhat.com>
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/219998
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
DocApproved Mandatory flag to pass smoke. Provide user facing document to get approval. SpecApproved This is a mandatory flag for passing the smoke for feature. Provide spec of feature to get approval Type:Bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

gfapi: give appropriate error when size exceeds rkothiya/glusterfs
2 participants
@gluster-ant @rkothiya