Parallel processing is broken, even with --parallel=1

[coke]

Because we are using ".allof" to wait for Promises, we are losing the fact that some of the pod processing is erroring out. - allof just makes sure the promise ended, not that it worked.

So far I've found issues with files that have:

=head 3 C<thing>

and nested tags like:

X<C<a thing>>

Basically, there are several places in the code where we are assuming something is a Str, when in fact it's a Pod object.

[JJ]

But are the two things related?
I seem to remember that worked correctly... at least until some time ago.
And the problem is anyway that we only have htmlify.p6 as a reference of how htmlify.p6 should behave. So a test suite is essential...

[JJ]

If #1947 depends on the the --parallel flag, maybe we should eliminate that completely. Is it really making things faster?

[dogbert17]

Yes it’s faster, when/if it works, but unfortunately it’s broken. From: Juan Julián Merelo Guervós [mailto:notifications@github.com] Sent: Saturday, 21 April 2018 17:45 To: perl6/doc Cc: Subscribed Subject: Re: [perl6/doc] Parallel processing is broken, even with --parallel=1 (#1938) If #1947 <#1947> depends on the the --parallel flag, maybe we should eliminate that completely. Is it really making things faster? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1938 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/ARklMgFWcU4CTUR8Bxxur-6LH3KrCHlZks5tq1QBgaJpZM4TZBQc> . <https://github.com/notifications/beacon/ARklMtJPMlVzoUEaKM_vGqUUlF1w8Ddcks5tq1QBgaJpZM4TZBQc.gif>

[coke]

If it's broken, it doesn't matter that it's faster.

I've ripped it out in the coke/build branch, may add parallel processing back in later; the main issue here is the "Promise.allof", which hides failures encountered when processing some pages; if you just switch it, the build will fail on the first page. I have more changes in coke/build to work around the issues, but they are just workarounds.

[tisonkun]

make html takes over 30 minutes on my local environment and longer with --parallel=n. I suspect highly wrong parallel does harm to efficiency instead of speeds up.

[JJ]

2018-04-23 16:28 GMT+02:00 陈梓立 <notifications@github.com>:

make html takes over 30 minutes on my local environment and longer with --parallel=n. I suspect highly wrong parallel does harm to efficiency instead of speeds up.

That might effectively be the case.

[coke]

d3db402 by @zoffixznet may have resolved this issue.

[JJ]

We'll have to check and close the issue if needed.

[JJ]

It dies when you do --parallel=2.

 11/53: doc/Language/control.pod6                => language/control
An operation first awaited:
  in sub process-pod-dir at htmlify.p6 line 269
  in sub MAIN at htmlify.p6 line 199
  in block <unit> at htmlify.p6 line 1003

Died with the exception:
    No such method 'level' for invocant of type 'Pod::Block::Para'
      in block  at htmlify.p6 line 627
      in sub find-definitions at htmlify.p6 line 503
      in sub process-pod-source at htmlify.p6 line 318
      in block  at htmlify.p6 line 265

[JJ]

--parallel=1 and no parallel take pretty much the same in my system. ~ 190 seconds. No wonder, since it's the default value...
So maybe we should do #1823 and fix it and/or eliminate it.

[coke]

parallel=1 on the command line is exactly the same as passing no arg. You're running exactly the same code.

the fact that parallel with numbers > 1 is broken has been discussed elsewhere. This ticket was specifically to point out that the theoretically non-parallel case of using 1 (either explicitly or implicitly) is (was?) also broken.

[JJ]

I didn't know that. Thought was kind of a flag. Anyway, let's close it if it's no longer broken.

[JJ]

And closing...