[EXPERIMENT] autoderef, the implicit deref in push REF and others

[p5pRT]

Migrated from rt.perl.org#119437 (status was 'resolved')

Searchable as RT119437$

[p5pRT]

From @rjbs

Perl 5.14.0 introduced this behavior, described in perl5140delta as​:

  Array and hash container functions accept references

  Warning​: This feature is considered experimental, as the exact
  behaviour may change in a future version of Perl.

  All builtin functions that operate directly on array or hash containers
  now also accept unblessed hard references to arrays or hashes​:

  [table]

  This allows these builtin functions to act on long dereferencing chains
  or on the return value of subroutines without needing to wrap them in
  "@​{}" or "%{}"​:

The behavior of autoderef changed between David Golden's original work and the
final shipped product.

Specific questions​:

* do we revert the behavior changes, at least in some cases?
* do we add an 'experimental' warning for perl 5.20 if the experiment isn't
  "accepted" yet by then?
* does this feature stick around if postfix deref is added

On the third point, my current thought is that it seems like removing it just
because a maybe-better form came around is going to lead to the same kind of
groaning that we've gotten with the addition of warnings for ~~ in perl 5.18.0.
Is it worth that? Current thoughts on *that*​: I really don't know. It's not
like the feature will not be useful anyway...

Anyway, the general question is the same as any other [EXPERIMENT] ticket​:

What are the criteria for accepting or rejecting this experiment?

--
rjbs

[p5pRT]

From @cpansprout

On Thu Aug 22 18​:47​:31 2013, rjbs wrote​:

What are the criteria for accepting or rejecting this experiment?

For acceptance​:
• Remove The Unicode Bug; i.e., that changes to the ‘type’ of the
  argument (for various meanings of ‘type’, such as blessedness
  [bliss?]) do not change what the operator does. Without that, it
  causes compatibility problems for modules that want to start
  returning blessed array refs instead of the unblessed ones they used
  to return, etc.
• If necessary after fixing that, extend the prototype syntax so that
  it can describe the actual syntax of these functions, so that we can
  make &CORE​::shift work. At one point I suggested \. for a reference
  to any scalar expression, so shift would have (;\[.@​]).

--

Father Chrysostomos

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @nwc10

On Thu, Aug 22, 2013 at 06​:47​:31PM -0700, Ricardo SIGNES wrote​:

Specific questions​:

* does this feature stick around if postfix deref is added

On the third point, my current thought is that it seems like removing it just
because a maybe-better form came around is going to lead to the same kind of
groaning that we've gotten with the addition of warnings for ~~ in perl 5.18.0.
Is it worth that? Current thoughts on *that*​: I really don't know. It's not
like the feature will not be useful anyway...

Assuming that postfix deref works, and works out, I certainly think that we
should discourage the use of the autoderef syntax, because postfix deref
will work everywhere that arrays and hashes are expected. I find the
(necessarily) limited availability of postfix deref to be confusing - if I
can autoderef on push, how come I can't autoderef on foreach? [1]

I don't think that there will be anywhere near as much grousing

1) if a replacement is available
2) because it's not widely used, partly because unlike smartmatch it doesn't
  offer functionality available in any other form, but it costs you backwards
  compatibility. I can't find many uses, eg​:
  http​://grep.cpan.me/?q=%5E%5B%5E%23%5Cn%5D*%5Cbpush%5Cs*%5C%24
  (about half of which are false positives. That query has been refined -
  it also likely drops a couple of real uses, but strips out a lot more false
  positives)

My assumption is that if a more general, more powerful [2] replacement is
available, maintained code would migrate to it.

Nicholas Clark

1​: Yes, I know the answer. But I have to *think* about it.
  "This should work here too? Oh wait, not it doesn't, because no it can't"
2​: Doesn't fall foul of the bugs that sprout notes, some of which I think
  might be intractable.

[p5pRT]

From @rjbs

* Nicholas Clark <nick@​ccl4.org> [2013-08-23T03​:26​:26]

Assuming that postfix deref works, and works out, I certainly think that we
should discourage the use of the autoderef syntax, because postfix deref
will work everywhere that arrays and hashes are expected. I find the
(necessarily) limited availability of postfix deref to be confusing - if I

  ^^^^^^^- I think you mean auto-

--
rjbs

[p5pRT]

From @nwc10

On Fri, Aug 23, 2013 at 06​:57​:10AM -0400, Ricardo Signes wrote​:

* Nicholas Clark <nick@​ccl4.org> [2013-08-23T03​:26​:26]

Assuming that postfix deref works, and works out, I certainly think that we
should discourage the use of the autoderef syntax, because postfix deref
will work everywhere that arrays and hashes are expected. I find the
(necessarily) limited availability of postfix deref to be confusing - if I

                                    ^^^^^^^\- I think you mean auto\-

I think I did too. Thanks for spotting this.

use more 'coffee'; # or something like that*

Nicholas Clark

* says the hypocrite who is usually drinking decaf.
  Although not *right* now, as this afternoon my coffee is provided by
  http​://validad.com/ (whom, I think, are hiring)

[p5pRT]

From @ikegami

On Thu, Aug 22, 2013 at 9​:47 PM, Ricardo SIGNES
<perlbug-followup@​perl.org>wrote​:

What are the criteria for accepting or rejecting this experiment?

I'm still opposed to it on the grounds that it only works for some
references. C<push $ref, $a;> doesn't work for blessed arrays or objects
with overloaded @​{}, so using it isn't forward-compatible.

- Eric

[p5pRT]

From @cpansprout

On Fri Aug 23 09​:59​:45 2013, ikegami@​adaelis.com wrote​:

On Thu, Aug 22, 2013 at 9​:47 PM, Ricardo SIGNES
<perlbug-followup@​perl.org>wrote​:

What are the criteria for accepting or rejecting this experiment?

I'm still opposed to it on the grounds that it only works for some
references. C<push $ref, $a;> doesn't work for blessed arrays or objects
with overloaded @​{}, so using it isn't forward-compatible.

As I’ve been saying from the outset​: If push $scalar always implies
push @​{ $scalar } (without the scope) and keys $scalar always implies %{
$scalar }, this problem goes away. It also makes ‘keys foo’ consistent
with ‘keys $scalar’, so there is no longer any reason to deprecate what
will no longer be a special case (which strict subs catches anyway).

--

Father Chrysostomos

[p5pRT]

From @ap

* Father Chrysostomos via RT <perlbug-followup@​perl.org> [2013-08-23 20​:15]​:

As I’ve been saying from the outset​: If push $scalar always implies
push @​{ $scalar } (without the scope) and keys $scalar always implies
%{ $scalar }, this problem goes away. It also makes ‘keys
foo’ consistent with ‘keys $scalar’, so there is no longer any reason
to deprecate what will no longer be a special case (which strict subs
catches anyway).

That is the only form in which I find this feature viable. The current
form is value-polymorphic which just never ever works in Perl 5, but if
it were monomorphic it could stay. If that gets no votes, by next choice
is to chuck it out – even now, but especially after postfix deref.

--
Aristotle Pagaltzis // <http​://plasmasturm.org/>

[p5pRT]

From @xdg

On Thu, Aug 22, 2013 at 9​:47 PM, Ricardo SIGNES
<perlbug-followup@​perl.org> wrote​:

* do we revert the behavior changes, at least in some cases?
* do we add an 'experimental' warning for perl 5.20 if the experiment isn't
"accepted" yet by then?
* does this feature stick around if postfix deref is added

There are mixed solutions as well​:

* make pure array functions (push/pop/shift/unshift/splice) auto-deref
(even on blessed objects) because they are unambiguous
* eliminate auto-deref of any kind for mixed hash/array functions
because of the ambiguity

Because of the ambiguity of keys/values/each, we're always going to
disappoint/surprise somebody. The current approach of allowing only
unblessed objects seems a decent compromise, as does changing the
behavior of keys/values/each on a reference — blessed or unblessed —
to always be %{} which is probably 99% of cases. But not matter what,
someone is going to grumble about whatever choice we make, even if
it's "works for push, but not for keys".

In hindsight, I wish we had said keys($ref) is always keys(%$ref). I
think that might annoy people but is more useful and less likely to
surprise people than having keys($object) fail.

(I assume at this point that deprecating keys/values/each on arrays
and replacing them with array-specific functions is off the table.)

Ultimately, I'd hate to see the baby thrown out with the bathwater
because of an edge case.

I think this decision is somewhat orthogonal to postfix dereference,
but I wouldn't cry if we got that and lost auto-dereference. It was
our original inability to converge on postfix dereference that
inspired me to figure out auto-dereference in the first place.

David

--
David Golden <xdg@​xdg.me>
Take back your inbox! → http​://www.bunchmail.com/
Twitter/IRC​: @​xdg

[p5pRT]

From @b2gills

On Fri, Aug 23, 2013 at 7​:39 PM, David Golden <xdg@​xdg.me> wrote​:

On Thu, Aug 22, 2013 at 9​:47 PM, Ricardo SIGNES
<perlbug-followup@​perl.org> wrote​:

* do we revert the behavior changes, at least in some cases?
* do we add an 'experimental' warning for perl 5.20 if the experiment isn't
"accepted" yet by then?
* does this feature stick around if postfix deref is added

There are mixed solutions as well​:

* make pure array functions (push/pop/shift/unshift/splice) auto-deref
(even on blessed objects) because they are unambiguous
* eliminate auto-deref of any kind for mixed hash/array functions
because of the ambiguity

Because of the ambiguity of keys/values/each, we're always going to
disappoint/surprise somebody. The current approach of allowing only
unblessed objects seems a decent compromise, as does changing the
behavior of keys/values/each on a reference — blessed or unblessed —
to always be %{} which is probably 99% of cases. But not matter what,
someone is going to grumble about whatever choice we make, even if
it's "works for push, but not for keys".

In hindsight, I wish we had said keys($ref) is always keys(%$ref). I
think that might annoy people but is more useful and less likely to
surprise people than having keys($object) fail.

(I assume at this point that deprecating keys/values/each on arrays
and replacing them with array-specific functions is off the table.)

Ultimately, I'd hate to see the baby thrown out with the bathwater
because of an edge case.

I think this decision is somewhat orthogonal to postfix dereference,
but I wouldn't cry if we got that and lost auto-dereference. It was
our original inability to converge on postfix dereference that
inspired me to figure out auto-dereference in the first place.

I kind of like that I can have a subroutine like this that will
work whether someone gave it an array ref, or a hash ref

  sub say_kv{
  my($ref) = @​_;
  my @​keys = keys $ref;
  my @​values = values $ref;
  while( @​keys ){
  say shift(@​keys), "\t", shift(@​values);
  }
  }

That being said, I have yet to come up with an example
that isn't a solution looking for a problem.

[p5pRT]

From @Hugmeir

On Sat, Aug 24, 2013 at 1​:03 AM, Brad Gilbert <b2gills@​gmail.com> wrote​:

On Fri, Aug 23, 2013 at 7​:39 PM, David Golden <xdg@​xdg.me> wrote​:

On Thu, Aug 22, 2013 at 9​:47 PM, Ricardo SIGNES
<perlbug-followup@​perl.org> wrote​:

* do we revert the behavior changes, at least in some cases?
* do we add an 'experimental' warning for perl 5.20 if the experiment
isn't
"accepted" yet by then?
* does this feature stick around if postfix deref is added

There are mixed solutions as well​:

* make pure array functions (push/pop/shift/unshift/splice) auto-deref
(even on blessed objects) because they are unambiguous
* eliminate auto-deref of any kind for mixed hash/array functions
because of the ambiguity

Because of the ambiguity of keys/values/each, we're always going to
disappoint/surprise somebody. The current approach of allowing only
unblessed objects seems a decent compromise, as does changing the
behavior of keys/values/each on a reference — blessed or unblessed —
to always be %{} which is probably 99% of cases. But not matter what,
someone is going to grumble about whatever choice we make, even if
it's "works for push, but not for keys".

In hindsight, I wish we had said keys($ref) is always keys(%$ref). I
think that might annoy people but is more useful and less likely to
surprise people than having keys($object) fail.

(I assume at this point that deprecating keys/values/each on arrays
and replacing them with array-specific functions is off the table.)

Ultimately, I'd hate to see the baby thrown out with the bathwater
because of an edge case.

I think this decision is somewhat orthogonal to postfix dereference,
but I wouldn't cry if we got that and lost auto-dereference. It was
our original inability to converge on postfix dereference that
inspired me to figure out auto-dereference in the first place.

I kind of like that I can have a subroutine like this that will
work whether someone gave it an array ref, or a hash ref

sub say\_kv\{
  my\($ref\) = @​\_;
  my @​keys = keys $ref;
  my @​values = values $ref;
  while\( @​keys \)\{
    say shift\(@​keys\)\, "\\t"\, shift\(@​values\);
  \}
\}

Or even better​:

  sub say_kv{
  my($ref) = @​_;
  while( my ($k,$v) = each $ref ){
  say "$k\t$v"
  }
  }

each $arrayref is the only thing I would miss if this were yanked out.

[p5pRT]

From @arc

This experiment has been deemed unsuccessful, and was removed as of commit 2623090 (first released as Perl 5.23.1 on 20 July 2015).

--
Aaron Crane ** http​://aaroncrane.co.uk/

[p5pRT]

@arc - Status changed from 'open' to 'resolved'