Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

experimental bitwise feature breaks PDL's & operator #16307

Closed
p5pRT opened this issue Dec 15, 2017 · 9 comments
Closed

experimental bitwise feature breaks PDL's & operator #16307

p5pRT opened this issue Dec 15, 2017 · 9 comments

Comments

@p5pRT
Copy link

@p5pRT p5pRT commented Dec 15, 2017

Migrated from rt.perl.org#132589 (status was 'rejected')

Searchable as RT132589$

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 15, 2017

From @djerius

Created by @djerius

The experimental bitwise feature breaks PDL code which uses the overloaded '&' operator.

Sample code​:

-----------------------------------

use PDL;

print "\n";

print "bitwise off​: ", pdl( 1 ) & pdl(2 ), "\n";

use feature 'bitwise';

print "bitwise on​: ", pdl( 1 ) & pdl(2 ), "\n";

-----------------------------------

The output​:

-----------------------------------

% perl bitwise.pl
The bitwise feature is experimental at bitwise.pl line 9.

bitwise off​: 0
Usage​: PDL​::and2(a,b,c,swap) (you may leave temporaries or output variables out of list) at bitwise.pl line 9.

-----------------------------------

PDL​::and2 is an XS function which provides the overloaded behavior. Its arguments are getting changed by
whatever bitwise is doing.

Thanks,
Diab

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl 5.22.1:

Configured by dj at Wed Jul 27 15:59:25 EDT 2016.

Summary of my perl5 (revision 5 version 22 subversion 1) configuration:
   
  Platform:
    osname=linux, osvers=3.10.0-327.22.2.el7.x86_64, archname=x86_64-linux_CentOS-7-thread-multi
    uname='linux sparkles.cfa.harvard.edu 3.10.0-327.22.2.el7.x86_64 #1 smp thu jun 23 17:05:11 utc 2016 x86_64 x86_64 x86_64 gnulinux '
    config_args='-de -Dprefix=/proj/axaf/ots/pkgs/perl-5.22/x86_64-linux_CentOS-7 -Uinstallusrbinperl -Dperladmin=none -Dlocincpth=/proj/axaf/ots/include -Darchname=x86_64-linux_CentOS-7 -Dusethreads -Dotherlibdirs=/proj/axaf/simul/lib/perl5:/proj/axaf/simul/x86_64-linux_CentOS-7/lib/perl5 -Dloclibpth=/proj/axaf/ots/x86_64-linux_CentOS-7/lib -Dglibpth=/lib /usr/lib -Dldflags=-L/proj/axaf/ots/x86_64-linux_CentOS-7/lib -Wl,-rpath=/proj/axaf/ots/x86_64-linux_CentOS-7/lib -Duseshrplib'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/proj/axaf/ots/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/proj/axaf/ots/include'
    ccversion='', gccversion='4.8.5 20150623 (Red Hat 4.8.5-4)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags ='-L/proj/axaf/ots/x86_64-linux_CentOS-7/lib -Wl,-rpath=/proj/axaf/ots/x86_64-linux_CentOS-7/lib -fstack-protector-strong'
    libpth=/usr/local/lib /usr/lib /proj/axaf/ots/x86_64-linux_CentOS-7/lib /lib/../lib64 /usr/lib/../lib64 /lib
    libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.17.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.17'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/proj/axaf/ots/pkgs/perl-5.22/x86_64-linux_CentOS-7/lib/5.22.1/x86_64-linux_CentOS-7-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/proj/axaf/ots/x86_64-linux_CentOS-7/lib -fstack-protector-strong'



@INC for perl 5.22.1:
    /proj/axaf/simul/lib/perl5
    /home/dj/lib/perl5
    /home/dj/research/lib/perl5
    /proj/axaf/ots/pkgs/perl-5.22/x86_64-linux_CentOS-7/lib/site_perl/5.22.1/x86_64-linux_CentOS-7-thread-multi
    /proj/axaf/ots/pkgs/perl-5.22/x86_64-linux_CentOS-7/lib/site_perl/5.22.1
    /proj/axaf/ots/pkgs/perl-5.22/x86_64-linux_CentOS-7/lib/5.22.1/x86_64-linux_CentOS-7-thread-multi
    /proj/axaf/ots/pkgs/perl-5.22/x86_64-linux_CentOS-7/lib/5.22.1
    /proj/axaf/simul/lib/perl5
    /proj/axaf/simul/x86_64-linux_CentOS-7/lib/perl5
    .


Environment for perl 5.22.1:
    HOME=/home/dj
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/soft/SYBASE15.7/DataAccess64/ODBC/lib:/soft/SYBASE15.7/DataAccess/ODBC/lib:/soft/SYBASE15.7/OCS-15_0/lib:/soft/SYBASE15.7/OCS-15_0/lib3p64:/soft/SYBASE15.7/OCS-15_0/lib3p:/soft/SYBASE15.7/DataAccess64/ODBC/lib:/soft/SYBASE15.7/DataAccess/ODBC/lib:/soft/SYBASE15.7/OCS-15_0/lib:/soft/SYBASE15.7/OCS-15_0/lib3p64:/soft/SYBASE15.7/OCS-15_0/lib3p:
    LOGDIR (unset)
    PATH=/home/dj/.plenv/libexec:/home/dj/.plenv/plugins/link-axaf/bin:/home/dj/.plenv/plugins/perl-build/bin:/home/dj/.plenv/plugins/plenv-contrib/bin:/soft/SYBASE15.7/OCS-15_0/bin:/home/dj/.rbenv/shims:/home/dj/.rbenv/bin:/home/dj/.pyenv/plugins/pyenv-virtualenv/shims:/home/dj/.pyenv/shims:/home/dj/.pyenv/bin:/home/dj/.plenv/shims:/home/dj/.plenv/bin:/home/dj/root/bin:/home/dj/root/bin/hosts:/home/dj/research/bin:/proj/axaf/ots/pkgs/perl-5.22/x86_64-linux_CentOS-7/bin:/proj/axaf/bin:/proj/axaf/simul/bin:/proj/axaf/simul/x86_64-linux_CentOS-7/bin:/proj/axaf/ots/x86_64-linux_CentOS-7/bin:/proj/axaf/ots/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/sbin:/bin:/usr/bin:/soft/saord/bin
    PERL5LIB=/proj/axaf/simul/lib/perl5:/home/dj/lib/perl5:/home/dj/research/lib/perl5
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 17, 2017

From @cpansprout

On Fri, 15 Dec 2017 14​:28​:25 -0800, djerius wrote​:

This is a bug report for perl from djerius@​cpan.org,
generated with the help of perlbug 1.40 running under perl 5.22.1.

-----------------------------------------------------------------
[Please describe your issue here]

The experimental bitwise feature breaks PDL code which uses the
overloaded '&' operator.

Usage​: PDL​::and2(a,b,c,swap) (you may leave temporaries or output
variables out of list) at bitwise.pl line 9.

Due to the nature of the bitwise feature, it is necessary to pass extra arguments to overload functions. overload.pm says the following​:

  If the experimental "bitwise" feature is enabled (see feature), a fifth
  TRUE argument is passed to subroutines handling "&", "|", "^" and "~".
  This indicates that the caller is expecting numeric behaviour. The fourth
  argument will be "undef", as that position ($_[3]) is reserved for use by
  "nomethod".

I have submitted a PDL patch at <https://rt.cpan.org/Ticket/Display.html?id=123901>.

There is no Perl bug here.

--

Father Chrysostomos

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 17, 2017

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 17, 2017

@cpansprout - Status changed from 'open' to 'rejected'

@p5pRT p5pRT closed this as completed Dec 17, 2017
@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 18, 2017

From @sisyphus

Note that both Math​::GMP and Math​::GMPz are also similarly broken by
enabling the bitwise feature. For example​:

################################
C​:\_32>cat bitwise.pl
use warnings;
use Math​::GMP;
use feature 'bitwise';

print Math​::GMP->new(2) & Math​::GMP->new(3);

C​:\_32>perl bitwise.pl
The bitwise feature is experimental at bitwise.pl line 5.
Usage​: Math​::GMP​::band(m, n, swap) at bitwise.pl line 5.

C​:\_32>
################################

I've submitted a bug report against Math​::GMP for this (see
https://rt.cpan.org/Ticket/Display.html?id=123907 )

I'll fix Math​::GMPz as soon as I work out how best to deal with this from
inside an XSub.
I need to find out​:
1) How does the XSub establish that the caller has enabled the bitwise
feature;
2) How does the XSub silence the warning.

I wonder how many other modules might be broken by this feature.
I've checked both Math​::BigInt and Bit​::Vector (as they were the first to
come to mind), but they're unaffected.
I haven't checked any other modules.

Cheers,
Rob

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 18, 2017

From zefram@fysh.org

Sisyphus via RT wrote​:

I need to find out​:
1) How does the XSub establish that the caller has enabled the bitwise
feature;

From the fifth argument supplied to the xsub. But Math​::GMP has no need
to know that, because its bitwise operators only provide numeric behaviour
in any case. There's no difference to Math​::GMP between the old ambiguous
conjunction and the new numeric-only conjunction. Math​::GMP​::band()
should ignore the fifth argument (and the fourth, and the third).

2) How does the XSub silence the warning.

It doesn't. The warning happens in the calling scope, which is where
the bitwise feature is enabled, and it can be silenced by that scope's
warning flags.

-zefram

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 18, 2017

From @djerius

On Sun, Dec 17, 2017 at 2​:08 PM, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Fri, 15 Dec 2017 14​:28​:25 -0800, djerius wrote​:

This is a bug report for perl from djerius@​cpan.org,
generated with the help of perlbug 1.40 running under perl 5.22.1.

-----------------------------------------------------------------
[Please describe your issue here]

The experimental bitwise feature breaks PDL code which uses the
overloaded '&' operator.

Usage​: PDL​::and2(a,b,c,swap) (you may leave temporaries or output
variables out of list) at bitwise.pl line 9.

Due to the nature of the bitwise feature, it is necessary to pass extra arguments to overload functions. overload.pm says the following​:

If the experimental "bitwise" feature is enabled \(see feature\)\, a fifth
TRUE argument is passed to subroutines handling "&"\, "|"\, "^" and "~"\.
This indicates that the caller is expecting numeric behaviour\. The fourth
argument will be "undef"\, as that position \($\_\[3\]\) is reserved for use by
"nomethod"\.

I have submitted a PDL patch at <https://rt.cpan.org/Ticket/Display.html?id=123901>.

There is no Perl bug here.

Thanks.

PDL doesn't use rt.cpan.org for bug-tracking, so I've submitted a
report at their SourceForge site​:

https://sourceforge.net/p/pdl/bugs/442/

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 19, 2017

From @sisyphus

On Mon, 18 Dec 2017 04​:32​:40 -0800, zefram@​fysh.org wrote​:

Sisyphus via RT wrote​:

I need to find out​:
1) How does the XSub establish that the caller has enabled the bitwise
feature;

From the fifth argument supplied to the xsub. But Math​::GMP has no need
to know that, because its bitwise operators only provide numeric behaviour
in any case. There's no difference to Math​::GMP between the old ambiguous
conjunction and the new numeric-only conjunction. Math​::GMP​::band()
should ignore the fifth argument (and the fourth, and the third).

2) How does the XSub silence the warning.

It doesn't. The warning happens in the calling scope, which is where
the bitwise feature is enabled, and it can be silenced by that scope's
warning flags.

-zefram

Oh, I see.
It's as simple as appending a "..." to the argument list.
At least that's all that Math​::GMPz needs, and I presume the same would hold for Math​::GMP.

The far more elaborate PDL patch that FC provided threw me a bit. But I can well believe that PDL, being what it is, would require a more elaborate fix.

Cheers,
Rob

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 20, 2017

From @cpansprout

On Mon, 18 Dec 2017 16​:06​:00 -0800, sisyphus wrote​:

Oh, I see.
It's as simple as appending a "..." to the argument list.
At least that's all that Math​::GMPz needs, and I presume the same
would hold for Math​::GMP.

The far more elaborate PDL patch that FC provided threw me a bit. But
I can well believe that PDL, being what it is, would require a more
elaborate fix.

Cheers,
Rob

FYI, I have added an explanation to perlxs.pod in commit daf3bcf.

--

Father Chrysostomos

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant