segmentation fault while concurrently updating SetHash

[p6rt]

Migrated from rt.perl.org#132225 (status was 'open')

Searchable as RT132225$

[p6rt]

From j.david.lowe@gmail.com

This short program crashes reliably (with a segmentation fault) on my
system​:

```
#!/usr/bin/env perl6

use v6.c;

my $lock = Lock.new;
my $set = SetHash.new;
await (^12).map​: {
  start {
  for (^1000) {
  $lock.protect​: { $set<1> = True }
  $lock.protect​: { $set<1> = False }
  }
  }
}
```

More information​:

```
$ perl6 --version
This is Rakudo version 2017.09 built on MoarVM version 2017.09.1
implementing Perl 6.c.
```

[p6rt]

From @AlexDaniel

FWIW it always did that, not a regression.

On 2017-10-05 21​:10​:39, j.david.lowe@​gmail.com wrote​:

This short program crashes reliably (with a segmentation fault) on my
system​:

```
#!/usr/bin/env perl6

use v6.c;

my $lock = Lock.new;
my $set = SetHash.new;
await (^12).map​: {
start {
for (^1000) {
$lock.protect​: { $set<1> = True }
$lock.protect​: { $set<1> = False }
}
}
}
```

More information​:

```
$ perl6 --version
This is Rakudo version 2017.09 built on MoarVM version 2017.09.1
implementing Perl 6.c.
```

[p6rt]

The RT System itself - Status changed from 'new' to 'open'

[p6rt]

From @timo

This runs reliably when you let the lock-protected block return
something unrelated to the hash​:

    #!/usr/bin/env perl6

    use v6.c;

    my $lock = Lock.new;
    my $set = SetHash.new;
    await (^12).map​: {
    start {
    for (^1000) {
    $lock.protect​: { $set<1> = True }
    $lock.protect​: { $set<1> = False }
    }
    }
    }

My assumption is that it has something to do with sinking the Bool we're
assigning outside the lock-protected block, but printing the .VAR of
what it returns is really just a Bool object ...

[p6rt]

From @timo

It's a bit tricky to get a proper backtrace dump out of gdb for this,
but what I'm seeing is this​:

this source file calls sink on a value from outside the protected lock,
which runs the FETCH of the SetHash, which calls nqp​::existskey on the
internal elems hash.

That part is explosive, presumably because setting a value to False in
SetHash removes it from the underlying hash.

Not sure how to fix this bug properly. Could be "just" called a trap,
but it's a real nasty one.

Once hashes have been ported over to a non-crashy version (i.e. not
immediately freeing storage when the hash is resized) this will probably
just get inconsistent results in whether it'll sink a true or a false.

On 11/10/17 08​:06, Timo Paulssen via RT wrote​:

This runs reliably when you let the lock-protected block return
something unrelated to the hash​:

    #!/usr/bin/env perl6

    use v6.c;

    my $lock = Lock.new;
    my $set = SetHash.new;
    await (^12).map​: {
    start {
    for (^1000) {
    $lock.protect​: { $set<1> = True }
    $lock.protect​: { $set<1> = False }
    }
    }
    }

My assumption is that it has something to do with sinking the Bool we're
assigning outside the lock-protected block, but printing the .VAR of
what it returns is really just a Bool object ...

[p6rt]

From j.david.lowe@gmail.com

This crash still occurs with rakudo 2017.10.

On Thu, Oct 5, 2017 at 9​:10 PM, perl6 via RT <perl6-bugs-followup@​perl.org>
wrote​:

Greetings,

This message has been automatically generated in response to the
creation of a trouble ticket regarding​:
"segmentation fault while concurrently updating SetHash",
a summary of which appears below.

There is no need to reply to this message right now. Your ticket has been
assigned an ID of [perl #​132225].

Please include the string​:

     \[perl #​132225\]

in the subject line of all future correspondence about this issue. To do
so,
you may reply to this message.

                    Thank you,
                    perl6\-bugs\-followup@​perl\.org

-------------------------------------------------------------------------
This short program crashes reliably (with a segmentation fault) on my
system​:

```
#!/usr/bin/env perl6

use v6.c;

my $lock = Lock.new;
my $set = SetHash.new;
await (^12).map​: {
start {
for (^1000) {
$lock.protect​: { $set<1> = True }
$lock.protect​: { $set<1> = False }
}
}
}
```

More information​:

```
$ perl6 --version
This is Rakudo version 2017.09 built on MoarVM version 2017.09.1
implementing Perl 6.c.
```

[p6rt]

From @lizmat

This does not seem to appear if you add at least one key to the set before the await. The segfault only appears to occur when adding a first or removing the last key from the SetHash.

On 10 Nov 2017, at 03​:28, David Lowe <j.david.lowe@​gmail.com> wrote​:

This crash still occurs with rakudo 2017.10.

On Thu, Oct 5, 2017 at 9​:10 PM, perl6 via RT <perl6-bugs-followup@​perl.org> wrote​:
Greetings,

This message has been automatically generated in response to the
creation of a trouble ticket regarding​:
"segmentation fault while concurrently updating SetHash",
a summary of which appears below.

There is no need to reply to this message right now. Your ticket has been
assigned an ID of [perl #​132225].

Please include the string​:

     \[perl #​132225\]

in the subject line of all future correspondence about this issue. To do so,
you may reply to this message.

                    Thank you,
                    perl6\-bugs\-followup@​perl\.org

-------------------------------------------------------------------------
This short program crashes reliably (with a segmentation fault) on my
system​:

```
#!/usr/bin/env perl6

use v6.c;

my $lock = Lock.new;
my $set = SetHash.new;
await (^12).map​: {
start {
for (^1000) {
$lock.protect​: { $set<1> = True }
$lock.protect​: { $set<1> = False }
}
}
}
```

More information​:

```
$ perl6 --version
This is Rakudo version 2017.09 built on MoarVM version 2017.09.1
implementing Perl 6.c.
```

[p6rt]

From @lizmat

reducing the code to​:

use nqp;
my $lock = Lock.new;
my $hash := nqp​::hash;
await ^16 .map​: {
  start {
  for ^1000 {
  $lock.protect​: { nqp​::bindkey($hash,"a",1) }
  $lock.protect​: { nqp​::deletekey($hash,"a") }
  }
  }
}

does *not* make it crash. So it would appear that it’s not the lowlevel hash access that’s to blame. Looking at this with Telemetry.snapper, this *does* seem to eat a lot of memory very quickly. With a slightly bigger version, and much longer running, growth to about 490MB is observed (starting from 69MB) in 35 seconds, then remaining constant for about 125 seconds, after which a modest increase of only about 15KB per second was observed for the remainder of the run.

So the next maybe it’s something in handling Proxy.

On 10 Nov 2017, at 03​:28, David Lowe <j.david.lowe@​gmail.com> wrote​:

This crash still occurs with rakudo 2017.10.

On Thu, Oct 5, 2017 at 9​:10 PM, perl6 via RT <perl6-bugs-followup@​perl.org> wrote​:
Greetings,

This message has been automatically generated in response to the
creation of a trouble ticket regarding​:
"segmentation fault while concurrently updating SetHash",
a summary of which appears below.

There is no need to reply to this message right now. Your ticket has been
assigned an ID of [perl #​132225].

Please include the string​:

     \[perl #​132225\]

in the subject line of all future correspondence about this issue. To do so,
you may reply to this message.

                    Thank you,
                    perl6\-bugs\-followup@​perl\.org

-------------------------------------------------------------------------
This short program crashes reliably (with a segmentation fault) on my
system​:

```
#!/usr/bin/env perl6

use v6.c;

my $lock = Lock.new;
my $set = SetHash.new;
await (^12).map​: {
start {
for (^1000) {
$lock.protect​: { $set<1> = True }
$lock.protect​: { $set<1> = False }
}
}
}
```

More information​:

```
$ perl6 --version
This is Rakudo version 2017.09 built on MoarVM version 2017.09.1
implementing Perl 6.c.
```

[p6rt]

From @timo

I already figured out that it's about sinking the result of assigning to
the SetHash. When you access it you get a proxy, that is the return
value of the lock-protected block, and the proxy gets sunk outside of
it, thus causing concurrent access to the SetHash.

On 14/11/17 16​:03, Elizabeth Mattijsen wrote​:

reducing the code to​:

use nqp;
my $lock = Lock.new;
my $hash := nqp​::hash;
await ^16 .map​: {
start {
for ^1000 {
$lock.protect​: { nqp​::bindkey($hash,"a",1) }
$lock.protect​: { nqp​::deletekey($hash,"a") }
}
}
}

does *not* make it crash. So it would appear that it’s not the lowlevel hash access that’s to blame. Looking at this with Telemetry.snapper, this *does* seem to eat a lot of memory very quickly. With a slightly bigger version, and much longer running, growth to about 490MB is observed (starting from 69MB) in 35 seconds, then remaining constant for about 125 seconds, after which a modest increase of only about 15KB per second was observed for the remainder of the run.

So the next maybe it’s something in handling Proxy.

On 10 Nov 2017, at 03​:28, David Lowe <j.david.lowe@​gmail.com> wrote​:

This crash still occurs with rakudo 2017.10.

On Thu, Oct 5, 2017 at 9​:10 PM, perl6 via RT <perl6-bugs-followup@​perl.org> wrote​:
Greetings,

This message has been automatically generated in response to the
creation of a trouble ticket regarding​:
"segmentation fault while concurrently updating SetHash",
a summary of which appears below.

There is no need to reply to this message right now. Your ticket has been
assigned an ID of [perl #​132225].

Please include the string​:

     \[perl #​132225\]

in the subject line of all future correspondence about this issue. To do so,
you may reply to this message.

                    Thank you,
                    perl6\-bugs\-followup@​perl\.org

-------------------------------------------------------------------------
This short program crashes reliably (with a segmentation fault) on my
system​:

```
#!/usr/bin/env perl6

use v6.c;

my $lock = Lock.new;
my $set = SetHash.new;
await (^12).map​: {
start {
for (^1000) {
$lock.protect​: { $set<1> = True }
$lock.protect​: { $set<1> = False }
}
}
}
```

More information​:

```
$ perl6 --version
This is Rakudo version 2017.09 built on MoarVM version 2017.09.1
implementing Perl 6.c.
```

[p6rt]

From @lizmat

Ah, indeed, so a workaround would be​:

my $lock = Lock.new;
my $set = SetHash.new;
await ^16 .map​: {
  start {
  for ^10000 {
  $lock.protect​: { $set<1> = True; 1 }
  $lock.protect​: { $set<1> = False; 1 }
  }
  }
}

So maybe a solution would be to test for Proxy of the return value of the block, and then sink it?

On 14 Nov 2017, at 16​:59, Timo Paulssen via RT <perl6-bugs-followup@​perl.org> wrote​:

I already figured out that it's about sinking the result of assigning to
the SetHash. When you access it you get a proxy, that is the return
value of the lock-protected block, and the proxy gets sunk outside of
it, thus causing concurrent access to the SetHash.

On 14/11/17 16​:03, Elizabeth Mattijsen wrote​:

reducing the code to​:

use nqp;
my $lock = Lock.new;
my $hash := nqp​::hash;
await ^16 .map​: {
start {
for ^1000 {
$lock.protect​: { nqp​::bindkey($hash,"a",1) }
$lock.protect​: { nqp​::deletekey($hash,"a") }
}
}
}

does *not* make it crash. So it would appear that it’s not the lowlevel hash access that’s to blame. Looking at this with Telemetry.snapper, this *does* seem to eat a lot of memory very quickly. With a slightly bigger version, and much longer running, growth to about 490MB is observed (starting from 69MB) in 35 seconds, then remaining constant for about 125 seconds, after which a modest increase of only about 15KB per second was observed for the remainder of the run.

So the next maybe it’s something in handling Proxy.

On 10 Nov 2017, at 03​:28, David Lowe <j.david.lowe@​gmail.com> wrote​:

This crash still occurs with rakudo 2017.10.

On Thu, Oct 5, 2017 at 9​:10 PM, perl6 via RT <perl6-bugs-followup@​perl.org> wrote​:
Greetings,

This message has been automatically generated in response to the
creation of a trouble ticket regarding​:
"segmentation fault while concurrently updating SetHash",
a summary of which appears below.

There is no need to reply to this message right now. Your ticket has been
assigned an ID of [perl #​132225].

Please include the string​:

    \[perl #​132225\]

in the subject line of all future correspondence about this issue. To do so,
you may reply to this message.

                   Thank you,
                   perl6\-bugs\-followup@​perl\.org

-------------------------------------------------------------------------
This short program crashes reliably (with a segmentation fault) on my
system​:

```
#!/usr/bin/env perl6

use v6.c;

my $lock = Lock.new;
my $set = SetHash.new;
await (^12).map​: {
start {
for (^1000) {
$lock.protect​: { $set<1> = True }
$lock.protect​: { $set<1> = False }
}
}
}
```

More information​:

```
$ perl6 --version
This is Rakudo version 2017.09 built on MoarVM version 2017.09.1
implementing Perl 6.c.
```

[p6rt]

From @lizmat

Fixed with be9e19efd97755cfd , tests needed!

On 10 Nov 2017, at 03​:28, David Lowe <j.david.lowe@​gmail.com> wrote​:

This crash still occurs with rakudo 2017.10.

On Thu, Oct 5, 2017 at 9​:10 PM, perl6 via RT <perl6-bugs-followup@​perl.org> wrote​:
Greetings,

This message has been automatically generated in response to the
creation of a trouble ticket regarding​:
"segmentation fault while concurrently updating SetHash",
a summary of which appears below.

There is no need to reply to this message right now. Your ticket has been
assigned an ID of [perl #​132225].

Please include the string​:

     \[perl #​132225\]

in the subject line of all future correspondence about this issue. To do so,
you may reply to this message.

                    Thank you,
                    perl6\-bugs\-followup@​perl\.org

-------------------------------------------------------------------------
This short program crashes reliably (with a segmentation fault) on my
system​:

```
#!/usr/bin/env perl6

use v6.c;

my $lock = Lock.new;
my $set = SetHash.new;
await (^12).map​: {
start {
for (^1000) {
$lock.protect​: { $set<1> = True }
$lock.protect​: { $set<1> = False }
}
}
}
```

More information​:

```
$ perl6 --version
This is Rakudo version 2017.09 built on MoarVM version 2017.09.1
implementing Perl 6.c.
```