Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

readdir_r is deprecated #15943

Closed
p5pRT opened this issue Apr 26, 2017 · 12 comments
Closed

readdir_r is deprecated #15943

p5pRT opened this issue Apr 26, 2017 · 12 comments

Comments

@p5pRT
Copy link

@p5pRT p5pRT commented Apr 26, 2017

Migrated from rt.perl.org#131214 (status was 'resolved')

Searchable as RT131214$

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 26, 2017

From @khwilliamson

This is a bug report for perl from khw@​khw.(none),
generated with the help of perlbug 1.40 running under perl 5.26.0.


glibc 2.24 deprecates readdir_r. And, compiling perl creates annoying
warning noise.

Starting with that version, plain readdir is reentrant, but not if two
threads share a stream. For those, it recommends using readdir with
synchronization exclusion (i.e., mutexes I suppose).

A man page is attached.



Flags​:
  category=core
  severity=low


Site configuration information for perl 5.26.0​:

Configured by khw at Mon Apr 24 08​:31​:45 MDT 2017.

Summary of my perl5 (revision 5 version 26 subversion 0) configuration​:
  Commit id​: 63e6b12
  Platform​:
  osname=linux
  osvers=4.10.0-19-generic
  archname=x86_64-linux-thread-multi-ld
  uname='linux khw 4.10.0-19-generic #21-ubuntu smp thu apr 6
17​:04​:57 utc 2017 x86_64 x86_64 x86_64 gnulinux '
  config_args='-des -Uversiononly -Dprefix=/home/khw/blead -Dusedevel
-D'optimize=-ggdb3' -A'optimize=-ggdb3' -A'optimize=-O0'
-Accflags='-DPERL_BOOL_AS_CHAR' -Accflags='-DPERL_EXTERNAL_GLOB'
-Dman1dir=none -Dman3dir=none -Dcc=g++ -DDEBUGGING -Dusemorebits
-Dusecbacktrace -Dusethreads'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=define
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='g++'
  ccflags ='-D_REENTRANT -D_GNU_SOURCE -DPERL_BOOL_AS_CHAR
-DPERL_EXTERNAL_GLOB -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
  optimize='-ggdb3 -ggdb3 -O0'
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_BOOL_AS_CHAR
-DPERL_EXTERNAL_GLOB -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='6.3.0 20170406'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='long double'
  nvsize=16
  Off_t='off_t'
  lseeksize=8
  alignbytes=16
  prototype=define
  Linker and Libraries​:
  ld='g++'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/include/c++/6 /usr/include/x86_64-linux-gnu/c++/6
/usr/include/c++/6/backward /usr/local/lib
/usr/lib/gcc/x86_64-linux-gnu/6/include-fixed
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.24.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.24'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -ggdb3 -ggdb3 -O0 -L/usr/local/lib
-fstack-protector-strong'


@​INC for perl 5.26.0​:
  /home/khw/perl/blead/lib
  /home/khw/perl/blead/t
  /home/khw/blead/lib/perl5/site_perl/5.26.0/x86_64-linux-thread-multi-ld
  /home/khw/blead/lib/perl5/site_perl/5.26.0
  /home/khw/blead/lib/perl5/5.26.0/x86_64-linux-thread-multi-ld
  /home/khw/blead/lib/perl5/5.26.0
  /home/khw/blead/lib/perl5/site_perl/5.25.12
  /home/khw/blead/lib/perl5/site_perl/5.25.11
  /home/khw/blead/lib/perl5/site_perl/5.25.10
  /home/khw/blead/lib/perl5/site_perl/5.25.9
  /home/khw/blead/lib/perl5/site_perl/5.25.8
  /home/khw/blead/lib/perl5/site_perl/5.25.7
  /home/khw/blead/lib/perl5/site_perl/5.25.6
  /home/khw/blead/lib/perl5/site_perl/5.25.5
  /home/khw/blead/lib/perl5/site_perl/5.25.4
  /home/khw/blead/lib/perl5/site_perl/5.25.3
  /home/khw/blead/lib/perl5/site_perl/5.25.2
  /home/khw/blead/lib/perl5/site_perl/5.25.1
  /home/khw/blead/lib/perl5/site_perl/5.24.0
  /home/khw/blead/lib/perl5/site_perl/5.23.10
  /home/khw/blead/lib/perl5/site_perl/5.23.9
  /home/khw/blead/lib/perl5/site_perl/5.23.8
  /home/khw/blead/lib/perl5/site_perl/5.23.7
  /home/khw/blead/lib/perl5/site_perl/5.23.6
  /home/khw/blead/lib/perl5/site_perl/5.23.5
  /home/khw/blead/lib/perl5/site_perl/5.23.4
  /home/khw/blead/lib/perl5/site_perl/5.23.3
  /home/khw/blead/lib/perl5/site_perl/5.23.2
  /home/khw/blead/lib/perl5/site_perl/5.23.1
  /home/khw/blead/lib/perl5/site_perl/5.23.0
  /home/khw/blead/lib/perl5/site_perl/5.22.0
  /home/khw/blead/lib/perl5/site_perl/5.21.12
  /home/khw/blead/lib/perl5/site_perl/5.21.11
  /home/khw/blead/lib/perl5/site_perl/5.21.10
  /home/khw/blead/lib/perl5/site_perl/5.21.9
  /home/khw/blead/lib/perl5/site_perl/5.21.8
  /home/khw/blead/lib/perl5/site_perl/5.21.7
  /home/khw/blead/lib/perl5/site_perl/5.21.6
  /home/khw/blead/lib/perl5/site_perl/5.21.5
  /home/khw/blead/lib/perl5/site_perl/5.21.4
  /home/khw/blead/lib/perl5/site_perl/5.21.3
  /home/khw/blead/lib/perl5/site_perl/5.21.2
  /home/khw/blead/lib/perl5/site_perl/5.21.1
  /home/khw/blead/lib/perl5/site_perl/5.20.0
  /home/khw/blead/lib/perl5/site_perl/5.19.12
  /home/khw/blead/lib/perl5/site_perl/5.19.11
  /home/khw/blead/lib/perl5/site_perl/5.19.10
  /home/khw/blead/lib/perl5/site_perl


Environment for perl 5.26.0​:
  HOME=/home/khw
  LANG=en_US.UTF-8
  LANGUAGE=en_US
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)

PATH=/usr/lib/ccache​:/home/khw/bin​:/home/khw/perl5/perlbrew/bin​:/home/khw/print/bin​:/bin​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/sbin​:/usr/games​:/usr/local/games​:/home/khw/iands/www​:/home/khw/cxoffice/bin
  PERL5OPT=-w
  PERL_BADLANG (unset)
  PERL_DIFF_TOOL=wgdiff
  PERL_POD_PEDANTIC=1
  SHELL=/bin/ksh

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 26, 2017

From @khwilliamson

READDIR_R(3) Linux Programmer's Manual READDIR_R(3)

NAME
  readdir_r - read a directory

SYNOPSIS
  #include <dirent.h>

  int readdir_r(DIR *dirp, struct dirent *entry, struct dirent **result);

  Feature Test Macro Requirements for glibc (see feature_test_macros(7))​:

  readdir_r()​:
  _POSIX_C_SOURCE
  || /* Glibc versions <= 2.19​: */ _BSD_SOURCE || _SVID_SOURCE

DESCRIPTION
  This function is deprecated; use readdir(3) instead.

  The readdir_r() function was invented as a reentrant version of read‐
  dir(3). It reads the next directory entry from the directory stream
  dirp, and returns it in the caller-allocated buffer pointed to by
  entry. For details of the dirent structure, see readdir(3).

  A pointer to the returned buffer is placed in *result; if the end of
  the directory stream was encountered, then NULL is instead returned in
  *result.

  It is recommended that applications use readdir(3) instead of read‐
  dir_r(). Furthermore, since version 2.24, glibc deprecates read‐
  dir_r(). The reasons are as follows​:

  * On systems where NAME_MAX is undefined, calling readdir_r() may be
  unsafe because the interface does not allow the caller to specify
  the length of the buffer used for the returned directory entry.

  * On some systems, readdir_r() can't read directory entries with very
  long names. When the glibc implementation encounters such a name,
  readdir_r() fails with the error ENAMETOOLONG after the final direc‐
  tory entry has been read. On some other systems, readdir_r() may
  return a success status, but the returned d_name field may not be
  null terminated or may be truncated.

  * In the current POSIX.1 specification (POSIX.1-2008), readdir(3) is
  not required to be thread-safe. However, in modern implementations
  (including the glibc implementation), concurrent calls to readdir(3)
  that specify different directory streams are thread-safe. There‐
  fore, the use of readdir_r() is generally unnecessary in multi‐
  threaded programs. In cases where multiple threads must read from
  the same directory stream, using readdir(3) with external synchro‐
  nization is still preferable to the use of readdir_r(), for the rea‐
  sons given in the points above.

  * It is expected that a future version of POSIX.1 will make read‐
  dir_r() obsolete, and require that readdir(3) be thread-safe when
  concurrently employed on different directory streams.

RETURN VALUE
  The readdir_r() function returns 0 on success. On error, it returns a
  positive error number (listed under ERRORS). If the end of the direc‐
  tory stream is reached, readdir_r() returns 0, and returns NULL in
  *result.

ERRORS
  EBADF Invalid directory stream descriptor dirp.

  ENAMETOOLONG
  A directory entry whose name was too long to be read was encoun‐
  tered.

ATTRIBUTES
  For an explanation of the terms used in this section, see
  attributes(7).

  ┌────────────┬───────────────┬─────────┐
  │Interface │ Attribute │ Value │
  ├────────────┼───────────────┼─────────┤
  │readdir_r() │ Thread safety │ MT-Safe │
  └────────────┴───────────────┴─────────┘
CONFORMING TO
  POSIX.1-2001, POSIX.1-2008.

SEE ALSO
  readdir(3)

COLOPHON
  This page is part of release 4.09 of the Linux man-pages project. A
  description of the project, information about reporting bugs, and the
  latest version of this page, can be found at
  https://www.kernel.org/doc/man-pages/.

  2016-03-01 READDIR_R(3)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented May 24, 2017

From @tonycoz

On Wed, 26 Apr 2017 11​:24​:08 -0700, public@​khwilliamson.com wrote​:

glibc 2.24 deprecates readdir_r. And, compiling perl creates annoying
warning noise.

Starting with that version, plain readdir is reentrant, but not if two
threads share a stream. For those, it recommends using readdir with
synchronization exclusion (i.e., mutexes I suppose).

A man page is attached.

This is something we'd need to detect, somehow.

POSIX still defines readdir_r() and doesn't require that readdir() be thread-safe (though that may change, see http​://austingroupbugs.net/view.php?id=696 )

Tony

@p5pRT
Copy link
Author

@p5pRT p5pRT commented May 24, 2017

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

@p5pRT p5pRT commented May 24, 2017

From @Tux

On Tue, 23 May 2017 19​:11​:24 -0700, "Tony Cook via RT"
<perlbug-followup@​perl.org> wrote​:

On Wed, 26 Apr 2017 11​:24​:08 -0700, public@​khwilliamson.com wrote​:

glibc 2.24 deprecates readdir_r. And, compiling perl creates annoying
warning noise.

Starting with that version, plain readdir is reentrant, but not if two
threads share a stream. For those, it recommends using readdir with
synchronization exclusion (i.e., mutexes I suppose).

A man page is attached.

This is something we'd need to detect, somehow.

POSIX still defines readdir_r() and doesn't require that readdir() be thread-safe (though that may change, see http​://austingroupbugs.net/view.php?id=696 )

This has already been addressed in branch arc_readdir_after5260
Aaron promised to merge after 5.26.0

--
H.Merijn Brand http​://tux.nl Perl Monger http​://amsterdam.pm.org/
using perl5.00307 .. 5.25 porting perl5 on HP-UX, AIX, and openSUSE
http​://mirrors.develooper.com/hpux/ http​://www.test-smoke.org/
http​://qa.perl.org http​://www.goldmark.org/jeff/stupid-disclaimers/

@p5pRT
Copy link
Author

@p5pRT p5pRT commented May 27, 2017

From @xsawyerx

On 05/24/2017 09​:46 AM, H.Merijn Brand wrote​:

On Tue, 23 May 2017 19​:11​:24 -0700, "Tony Cook via RT"
<perlbug-followup@​perl.org> wrote​:

On Wed, 26 Apr 2017 11​:24​:08 -0700, public@​khwilliamson.com wrote​:

glibc 2.24 deprecates readdir_r. And, compiling perl creates annoying
warning noise.

Starting with that version, plain readdir is reentrant, but not if two
threads share a stream. For those, it recommends using readdir with
synchronization exclusion (i.e., mutexes I suppose).

A man page is attached.
This is something we'd need to detect, somehow.

POSIX still defines readdir_r() and doesn't require that readdir() be thread-safe (though that may change, see http​://austingroupbugs.net/view.php?id=696 )
This has already been addressed in branch arc_readdir_after5260
Aaron promised to merge after 5.26.0

This can be merged into blead-next.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 19, 2018

From @khwilliamson

On Sat, 27 May 2017 10​:15​:59 -0700, xsawyerx@​gmail.com wrote​:

On 05/24/2017 09​:46 AM, H.Merijn Brand wrote​:

On Tue, 23 May 2017 19​:11​:24 -0700, "Tony Cook via RT"
<perlbug-followup@​perl.org> wrote​:

On Wed, 26 Apr 2017 11​:24​:08 -0700, public@​khwilliamson.com wrote​:

glibc 2.24 deprecates readdir_r. And, compiling perl creates
annoying
warning noise.

Starting with that version, plain readdir is reentrant, but not if
two
threads share a stream. For those, it recommends using readdir
with
synchronization exclusion (i.e., mutexes I suppose).

A man page is attached.
This is something we'd need to detect, somehow.

POSIX still defines readdir_r() and doesn't require that readdir()
be thread-safe (though that may change, see
http​://austingroupbugs.net/view.php?id=696 )
This has already been addressed in branch arc_readdir_after5260
Aaron promised to merge after 5.26.0

This can be merged into blead-next.

This is on our blockers list for 5.28. Did this merge ever happen?
--
Karl Williamson

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 20, 2018

From @xsawyerx

On 03/20/2018 01​:23 AM, Karl Williamson via RT wrote​:

On Sat, 27 May 2017 10​:15​:59 -0700, xsawyerx@​gmail.com wrote​:

On 05/24/2017 09​:46 AM, H.Merijn Brand wrote​:

On Tue, 23 May 2017 19​:11​:24 -0700, "Tony Cook via RT"
<perlbug-followup@​perl.org> wrote​:

On Wed, 26 Apr 2017 11​:24​:08 -0700, public@​khwilliamson.com wrote​:

glibc 2.24 deprecates readdir_r. And, compiling perl creates
annoying
warning noise.

Starting with that version, plain readdir is reentrant, but not if
two
threads share a stream. For those, it recommends using readdir
with
synchronization exclusion (i.e., mutexes I suppose).

A man page is attached.
This is something we'd need to detect, somehow.

POSIX still defines readdir_r() and doesn't require that readdir()
be thread-safe (though that may change, see
http​://austingroupbugs.net/view.php?id=696 )
This has already been addressed in branch arc_readdir_after5260
Aaron promised to merge after 5.26.0
This can be merged into blead-next.
This is on our blockers list for 5.28. Did this merge ever happen?

I found this​: c6709bf.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 20, 2018

From @khwilliamson

I'm closing it then.
--
Karl Williamson

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 20, 2018

@khwilliamson - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 23, 2018

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release yesterday of Perl 5.28.0, this and 185 other issues have been
resolved.

Perl 5.28.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.28.0

If you find that the problem persists, feel free to reopen this ticket.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 23, 2018

@khwilliamson - Status changed from 'pending release' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant