Expose linux extended filesystem attributes

[benjaminp]

BPO	12720
Nosy	@pitrou, @benjaminp, @skrah
Files	xattrs.patch xattrs.patch xattrs.patch xattrs.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2011-09-14.15:46:22.542>
created_at = <Date 2011-08-09.20:24:55.181>
labels = ['extension-modules', 'type-feature']
title = 'Expose linux extended filesystem attributes'
updated_at = <Date 2011-09-14.15:46:22.541>
user = 'https://github.com/benjaminp'

bugs.python.org fields:

activity = <Date 2011-09-14.15:46:22.541>
actor = 'python-dev'
assignee = 'none'
closed = True
closed_date = <Date 2011-09-14.15:46:22.542>
closer = 'python-dev'
components = ['Extension Modules']
creation = <Date 2011-08-09.20:24:55.181>
creator = 'benjamin.peterson'
dependencies = []
files = ['22869', '22937', '23056', '23057']
hgrepos = []
issue_num = 12720
keywords = ['patch']
message_count = 12.0
messages = ['141842', '142293', '142372', '143095', '143104', '143105', '143106', '143294', '143987', '143989', '144027', '144029']
nosy_count = 6.0
nosy_names = ['pitrou', 'benjamin.peterson', 'Arfrever', 'skrah', 'neologix', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue12720'
versions = ['Python 3.3']

[benjaminp]

These allow extra metadata to be attached to files.

[pitrou]

I think the functions should:

• accept both str and bytes for keys and values; when it's an str, encoding it using the FS encoding
• return str for attribute names, using the FS encoding
• return bytes for attribute values, although returning str might be reasonable as well

I also get the following test failures:

======================================================================
FAIL: test_fds (test.test_os.ExtendedAttributeTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/antoine/cpython/default/Lib/test/test_os.py", line 1579, in test_fds
    self._check_xattrs(getxattr, setxattr, removexattr, listxattr)
  File "/home/antoine/cpython/default/Lib/test/test_os.py", line 1542, in _check_xattrs
    setxattr(fn, b"user.test", b"bye", os.XATTR_CREATE)
AssertionError: OSError not raised

======================================================================
FAIL: test_lpath (test.test_os.ExtendedAttributeTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/antoine/cpython/default/Lib/test/test_os.py", line 1564, in test_lpath
    os.llistxattr)
  File "/home/antoine/cpython/default/Lib/test/test_os.py", line 1542, in _check_xattrs
    setxattr(fn, b"user.test", b"bye", os.XATTR_CREATE)
AssertionError: OSError not raised

======================================================================
FAIL: test_simple (test.test_os.ExtendedAttributeTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/antoine/cpython/default/Lib/test/test_os.py", line 1560, in test_simple
    os.listxattr)
  File "/home/antoine/cpython/default/Lib/test/test_os.py", line 1542, in _check_xattrs
    setxattr(fn, b"user.test", b"bye", os.XATTR_CREATE)
AssertionError: OSError not raised

[benjaminp]

Here is a new patch, implementing Antoine's suggestions.

[benjaminp]

And here is the next version, taking into account neologix's review.

[pitrou]

Is it normal that listxattr() succeeds but getxattr() fails with ENOTSUPP?

>>> os.listxattr("/")
[]
>>> os.getxattr("/", "foo")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 95] Operation not supported

This is on 2.6.38.8.

[benjaminp]

2011/8/28 Antoine Pitrou <report@bugs.python.org>:
>
> Antoine Pitrou <pitrou@free.fr> added the comment:
>
> Is it normal that listxattr() succeeds but getxattr() fails with ENOTSUPP?
>
>>>> os.listxattr("/")
> []
>>>> os.getxattr("/", "foo")
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> OSError: [Errno 95] Operation not supported

The reason you're getting ENOSUP is you have to use the proper prefix.
"user.*" for example.

[benjaminp]

After Antoine's review...

[python-dev]

New changeset 4eb0b1819bda by Benjamin Peterson in branch 'default':
expose linux extended file system attributes (closes bpo-12720)
http://hg.python.org/cpython/rev/4eb0b1819bda

[Arfrever]

There is an inconsistency in used header and library.
attr/xattr.h and libattr.so belong to attr package (http://savannah.nongnu.org/projects/attr).
glibc provides sys/xattr.h and libc.so.
Both libattr.so and libc.so define getxattr(), setxattr() and other functions.
Modules/posixmodule.c includes attr/xattr.h from attr package, but libpython3.3.so isn't linked against libattr.so and is linked against libc.so, so functions from glibc are used at run time.

I suggest to use sys/xattr.h:

• sys/xattr.h instead of attr/xattr.h in Modules/posixmodule.c, configure, configure.in and pyconfig.h.in
• HAVE_SYS_XATTR_H instead of HAVE_ATTR_XATTR_H in Modules/posixmodule.c and pyconfig.h.in

[python-dev]

New changeset 33f7044b5682 by Benjamin Peterson in branch 'default':
Use xattr functions from sys/xattr.h instead of attr/xattr.h (closes bpo-12720)
http://hg.python.org/cpython/rev/33f7044b5682

[skrah]

The OS X buildbots fail to compile posixmodule.c:

gcc -fno-strict-aliasing -g -O0 -Wall -Wstrict-prototypes -I. -I./Include -DPy_BUILD_CORE -c ./Modules/posixmodule.c -o Modules/posixmodule.o
./Modules/posixmodule.c: In function ‘try_getxattr’:
./Modules/posixmodule.c:10045: error: ‘XATTR_SIZE_MAX’ undeclared (first use in this function)
./Modules/posixmodule.c:10045: error: (Each undeclared identifier is reported only once
./Modules/posixmodule.c:10045: error: for each function it appears in.)
./Modules/posixmodule.c: In function ‘getxattr_common’:
./Modules/posixmodule.c:10083: error: ‘XATTR_SIZE_MAX’ undeclared (first use in this function)
./Modules/posixmodule.c: In function ‘posix_getxattr’:
./Modules/posixmodule.c:10101: warning: passing argument 3 of ‘getxattr_common’ from incompatible pointer type
./Modules/posixmodule.c: In function ‘posix_lgetxattr’:
./Modules/posixmodule.c:10119: error: ‘lgetxattr’ undeclared (first use in this function)
./Modules/posixmodule.c: In function ‘wrap_fgetxattr’:
./Modules/posixmodule.c:10129: error: too few arguments to function ‘fgetxattr’
./Modules/posixmodule.c:10130: warning: control reaches end of non-void function
./Modules/posixmodule.c: In function ‘posix_setxattr’:
./Modules/posixmodule.c:10165: error: too few arguments to function ‘setxattr’
./Modules/posixmodule.c: In function ‘posix_lsetxattr’:
./Modules/posixmodule.c:10190: warning: implicit declaration of function ‘lsetxattr’
./Modules/posixmodule.c: In function ‘posix_fsetxattr’:
./Modules/posixmodule.c:10216: error: too few arguments to function ‘fsetxattr’
./Modules/posixmodule.c: In function ‘posix_removexattr’:
./Modules/posixmodule.c:10239: error: too few arguments to function ‘removexattr’
./Modules/posixmodule.c: In function ‘posix_lremovexattr’:
./Modules/posixmodule.c:10262: warning: implicit declaration of function ‘lremovexattr’
./Modules/posixmodule.c: In function ‘posix_fremovexattr’:
./Modules/posixmodule.c:10285: error: too few arguments to function ‘fremovexattr’
./Modules/posixmodule.c: In function ‘listxattr_common’:
./Modules/posixmodule.c:10327: error: ‘XATTR_LIST_MAX’ undeclared (first use in this function)
./Modules/posixmodule.c: In function ‘posix_listxattr’:
./Modules/posixmodule.c:10369: warning: passing argument 2 of ‘listxattr_common’ from incompatible pointer type
./Modules/posixmodule.c: In function ‘posix_llistxattr’:
./Modules/posixmodule.c:10385: error: ‘llistxattr’ undeclared (first use in this function)
./Modules/posixmodule.c: In function ‘wrap_flistxattr’:
./Modules/posixmodule.c:10394: error: too few arguments to function ‘flistxattr’
./Modules/posixmodule.c:10395: warning: control reaches end of non-void function
./Modules/posixmodule.c: In function ‘all_ins’:
./Modules/posixmodule.c:11342: error: ‘XATTR_SIZE_MAX’ undeclared (first use in this function)
make: *** [Modules/posixmodule.o] Error 1
program finished with exit code 2
elapsedTime=20.601350

[python-dev]

New changeset f325439d7f84 by Benjamin Peterson in branch 'default':
only compile xattrs on glibc (closes bpo-12720)
http://hg.python.org/cpython/rev/f325439d7f84