Efficiency problems after using a full mustache template

[antoniogamiz]

After creating the default template, generation time has increased up to 30 minutes, which is a big problem. I have come up with the following solution:

• Use Template::Mustache cache capabilities (done in Improve efficiency, fix #76 Pod-To-HTML#77)
• Prepolute menus and submenus in Documentable::To::HTML::Wrapper.

After measuring (approximately) the generation times, we get:

Change	Time
No changes	~ 1500 seconds (25 minutes)
(sub)menu pre-population	~ 1000 seconds (16 minutes)
pre-population + template caching	~ 500 seconds (8-9 minutes)

• plus around 3 minutes to create the pod cache.

[antoniogamiz]

After the last commit I have realized:

• some tests are missing from Documentable::To::HTML::Wrapper.
• Prepopulation is not supported in Mustache (at the moment, discussion: Doubt about Mustache specification softmoth/raku-Template-Mustache#36)

[JJ]

We can also try and use a different templating system, like Template::Classic.

[antoniogamiz]

In Pod::To::HTML too?

[JJ]

We should detach this from Pod::To::HTML, maybe.

[antoniogamiz]

I have used Template::Classic here: https://github.com/Raku/Documentable/tree/feat/use-template-classic. Results:

Generating source files 👇 ...
[=======================================================================]100.00%
Generate source files has taken 228.8313913 seconds 
Generating Kind::Syntax files 👇 ...
[=======================================================================]100.00%
Generating Kind::Routine files 👇 ...
[=======================================================================]100.00%
Generating per kind files has taken 180.3566868 seconds 
Generating indexes...
Generating index files has taken 10.72521574 seconds 
Writing search file...
Generating entries
Writing all generated files 🙌 ...
[#######################################################################]100.00%
Writing generated files has taken 0.8260129 seconds 
Whole process has taken 429.9028253 seconds

[JJ]

It's a bit less than even pre-population + template caching, correct?

[vrurg]

I noticed Template::Mustache and performance mentioned under the same context and perhaps can give a useful tip. I don't know where exactly the performance suffers the most in this case, but the latest module implementation supports .hyper/.race on rows:

%data<rows> = (gather { ...; take { :$record } }).hyper;

I use it this way in Vikna in a tool which produces HTML out of a sqlite.

[antoniogamiz]

It's a bit less than even pre-population + template caching, correct?

Both develop and feat/use-template-classic implements pre-popultation + template caching. In the first one, pre-population is made by Template::Mustache (and by that reason, it does not work, see softmoth/raku-Template-Mustache#36). In the other, it's made by Template::Classic.

In that particular case, it seems faster, but it should take the same amount of time (approx) because once templates are pre-populated, all the work is made by Pod::To::HTML.

[antoniogamiz]

I noticed Template::Mustache and performance mentioned under the same context and perhaps can give a useful tip. I don't know where exactly the performance suffers the most in this case, but the latest module implementation supports .hyper/.race on rows:

%data<rows> = (gather { ...; take { :$record } }).hyper;

I use it this way in Vikna in a tool which produces HTML out of a sqlite.

I tried to use parallelism to generate the HTML pages but the problem is that Template::Mustache is used inside Pod::To::HTML, which does not support parallelism ¹.

¹ As a curiosity, today is the anniversary of that issue.

[antoniogamiz]

I noticed Template::Mustache and performance mentioned under the same context and perhaps can give a useful tip. I don't know where exactly the performance suffers the most in this case, but the latest module implementation supports .hyper/.race on rows:

%data<rows> = (gather { ...; take { :$record } }).hyper;

I use it this way in Vikna in a tool which produces HTML out of a sqlite.

I have tried once again, just in case, but it's still failing:

use Pod::Load;
use Pod::To::HTML;

my $pod = load("Endian.pod6")[0];

my @works = [];
for 1..255 { @works.push: $pod }

@works.race.map({
    render($pod)
})