Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid (when possible) calling sys_*getxattr with NULL size #720

Closed
2 of 6 tasks
mykaul opened this issue Sep 3, 2019 · 6 comments
Closed
2 of 6 tasks

Avoid (when possible) calling sys_*getxattr with NULL size #720

mykaul opened this issue Sep 3, 2019 · 6 comments
Milestone

Comments

@mykaul
Copy link
Contributor

mykaul commented Sep 3, 2019

In several cases in the code [for example, posix_cs_set_state() ], we have this code convention that we do some getxattr without a buffer to get the buffer we need, then we CALLOC (we could MALLOC, btw) the buffer and do another fetch with the size.

In other cases, we are more efficient. We have some buffer, and we either use it, or dynamically realloc if the first getxattr fails. This is better as it removes an xattr in case the value fits the buffer.

Bad example:

       xattrsize = sys_fgetxattr(*fd, GF_CS_OBJECT_REMOTE, NULL, 0);
        if (xattrsize != -1) {
            value = GF_CALLOC(1, xattrsize + 1, gf_posix_mt_char);
            if (!value) {
                gf_msg(this->name, GF_LOG_ERROR, 0, 0, "no memory for value");
                ret = -1;
                goto out;
            }
            /* TODO: Add check for ENODATA */
            xattrsize = sys_fgetxattr(*fd, GF_CS_OBJECT_REMOTE, value,
                                      xattrsize + 1);

Slightly better example: (but still suffers from extra memory copy which can be removed)

    if (filler->real_path)
        xattr_size = sys_lgetxattr(filler->real_path, key, val_buf,
                                   sizeof(val_buf) - 1);
    else
        xattr_size = sys_fgetxattr(filler->fdnum, key, val_buf,
                                   sizeof(val_buf) - 1);

    if (xattr_size >= 0) {
        have_val = _gf_true;
    } else if (xattr_size == -1 && errno != ERANGE) {
        ret = -1;
        goto out;
    }

    if (have_val) {
        /*No need to do getxattr*/
    } else if (filler->real_path) {
        xattr_size = sys_lgetxattr(filler->real_path, key, NULL, 0);
    } else {
        xattr_size = sys_fgetxattr(filler->fdnum, key, NULL, 0);
    }

    if (xattr_size != -1) {
        value = GF_MALLOC(xattr_size + 1, gf_posix_mt_char);
        if (!value)
            goto out;

        if (have_val) {
            memcpy(value, val_buf, xattr_size);
        } else {
            bzero(value, xattr_size + 1);
            if (filler->real_path) {
                xattr_size = sys_lgetxattr(filler->real_path, key, value,
                                           xattr_size);

The places I found the bad implementation:

  • posix_fetch_mdata_xattr()
  • posix_cs_set_state()
  • posix_cs_check_status()
  • _posix_xattr_get_set_from_backend
  • _get_list_xattr() - really bad (called many times)
  • posix_fgetxattr
@atinmu
Copy link

atinmu commented Sep 9, 2019

@pranithk @raghavendrabhat ^^

@spalai
Copy link

spalai commented Sep 9, 2019

Will send a fix for this.

@mykaul
Copy link
Contributor Author

mykaul commented Sep 19, 2019

https://review.gluster.org/#/c/glusterfs/+/23452/ is an attempt to do that for _posix_xattr_get_set_from_backend()

gluster-ant pushed a commit that referenced this issue Oct 11, 2019
The code is simplified to avoid needless copy as well as simplified
overall for readability.
Such changes are needed elsewhere too (see
#720 )

Few other minor changes here and there, nothing functional.

Change-Id: Ia1167849f54d9cacbfe32ddd712dc1699760daf5
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
amarts pushed a commit to amarts/glusterfs that referenced this issue Oct 11, 2019
The code is simplified to avoid needless copy as well as simplified
overall for readability.
Such changes are needed elsewhere too (see
gluster/glusterfs#720 )

Few other minor changes here and there, nothing functional.

Change-Id: Ia1167849f54d9cacbfe32ddd712dc1699760daf5
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
gluster-ant pushed a commit that referenced this issue Oct 21, 2019
The code is simplified to avoid needless copy as well as simplified
overall for readability.
Such changes are needed elsewhere too (see
#720 )

Few other minor changes here and there, nothing functional.

updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

Change-Id: I14f9dd2c32a8932bfcc80ebe92c9aa77701095ff
amarts pushed a commit to amarts/glusterfs that referenced this issue Oct 21, 2019
The code is simplified to avoid needless copy as well as simplified
overall for readability.
Such changes are needed elsewhere too (see
gluster/glusterfs#720 )

Few other minor changes here and there, nothing functional.

updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

Change-Id: I14f9dd2c32a8932bfcc80ebe92c9aa77701095ff
@mykaul
Copy link
Contributor Author

mykaul commented Dec 1, 2019

_get_list_xattr is fixed via https://review.gluster.org/#/c/glusterfs/+/23714/

gluster-ant pushed a commit that referenced this issue Jan 6, 2020
Another location where instead of 2 sys calls we strive to get the
xattr in a single call, by guesstimating the required size
And avoid (or try to) not to first read the xattr len,
then another call to actually fetch. Instead, use a sane size
(256 bytes - worth checking if it makes sense or by default
use a larger size), and see if we can fetch it.
If we fail, we'll read the size and re-fetch.

Such changes are needed elsewhere too (see
#720 )

Change-Id: I466cea9d8b12fc45f6b37d202b1294ca28cd1fdd
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
@mykaul
Copy link
Contributor Author

mykaul commented Jan 13, 2020

@amarts - I hope to complete all of this (or most) for Gluster 8.

https://review.gluster.org/#/c/glusterfs/+/23714/ is in the works.
https://review.gluster.org/#/c/glusterfs/+/23823/ is another one in the list for these, ready for review.

@amarts amarts added this to the Release 8 milestone Jan 14, 2020
@mykaul
Copy link
Contributor Author

mykaul commented Jan 23, 2020

@amarts - I hope to complete all of this (or most) for Gluster 8.

https://review.gluster.org/#/c/glusterfs/+/23714/ is in the works.
https://review.gluster.org/#/c/glusterfs/+/23823/ is another one in the list for these, ready for review.

I'm not going to pursue these for Gluster anymore - patches abandoned .

@mykaul mykaul closed this as completed Jan 23, 2020
amarts pushed a commit to amarts/glusterfs that referenced this issue Jan 24, 2020
Another location where instead of 2 sys calls we strive to get the
xattr in a single call, by guesstimating the required size
And avoid (or try to) not to first read the xattr len,
then another call to actually fetch. Instead, use a sane size
(256 bytes - worth checking if it makes sense or by default
use a larger size), and see if we can fetch it.
If we fail, we'll read the size and re-fetch.

Such changes are needed elsewhere too (see
gluster/glusterfs#720 )

Change-Id: I466cea9d8b12fc45f6b37d202b1294ca28cd1fdd
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
amarts pushed a commit to amarts/glusterfs that referenced this issue Mar 24, 2020
The code is simplified to avoid needless copy as well as simplified
overall for readability.
Such changes are needed elsewhere too (see
gluster/glusterfs#720 )

Few other minor changes here and there, nothing functional.

Change-Id: Ia1167849f54d9cacbfe32ddd712dc1699760daf5
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
amarts pushed a commit to amarts/glusterfs that referenced this issue Mar 24, 2020
The code is simplified to avoid needless copy as well as simplified
overall for readability.
Such changes are needed elsewhere too (see
gluster/glusterfs#720 )

Few other minor changes here and there, nothing functional.

updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

Change-Id: I14f9dd2c32a8932bfcc80ebe92c9aa77701095ff
amarts pushed a commit to amarts/glusterfs that referenced this issue Mar 24, 2020
Another location where instead of 2 sys calls we strive to get the
xattr in a single call, by guesstimating the required size
And avoid (or try to) not to first read the xattr len,
then another call to actually fetch. Instead, use a sane size
(256 bytes - worth checking if it makes sense or by default
use a larger size), and see if we can fetch it.
If we fail, we'll read the size and re-fetch.

Such changes are needed elsewhere too (see
gluster/glusterfs#720 )

Change-Id: I466cea9d8b12fc45f6b37d202b1294ca28cd1fdd
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants