0.2.598 parser is slower than 0.2.543 on large files

[dv1x3r]

Hello, I noticed that the template generation works slower with version 598 on large .templ files.

I have a templ file with ~1k rows, and ~30 components. templ generate is ~5x slower on this one.

Here is the comparison before the update:

╰─ templ version
v0.2.543

╰─ time templ generate
Processing path: /Users/dx/source/teh-go
Generating production code: /Users/dx/source/teh-go
(✓) Generated code for "/Users/dx/source/teh-go/internal/app/component/admin.templ" in 2.520125ms
(✓) Generated code for "/Users/dx/source/teh-go/internal/app/component/client.templ" in 46.07ms
(✓) Generated code for 2 templates with 0 errors in 75.794833ms
templ generate  0.09s user 0.06s system 163% cpu 0.091 total

And after the update:

╰─ templ version
v0.2.598

╰─ time templ generate -v
(✓) Starting post-generation handler
(✓) Walking directory [ path=/Users/dx/source/teh-go devMode=false ]
(✓) Starting event handler
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/admin_templ.go ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/client_templ.go ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/client.templ ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/admin.templ ]
(✓) Generated code [ file=/Users/dx/source/teh-go/internal/app/component/admin.templ in=2.148833ms ]
(✓) First post-generation event received, starting proxy
(✓) No proxy URL specified, not starting proxy
(✓) Dev mode not enabled, process can finish early
(✓) Generated code [ file=/Users/dx/source/teh-go/internal/app/component/client.templ in=4.215424s ]
(✓) Post-generation event channel closed, exiting
(✓) Waiting for push handler to complete
(✓) Waiting for event handler to complete
(✓) Waiting for post-generation handler to complete
(✓) Complete [ updates=2 duration=4.216013041s ]
templ generate -v  5.84s user 0.95s system 160% cpu 4.231 total

I can split the client.templ into several smaller pieces, and then it will become "acceptable".
But I do not want to do that for this specific project.

╰─ templ version
v0.2.598

╰─ time templ generate -v
(✓) Starting post-generation handler
(✓) Walking directory [ path=/Users/dx/source/teh-go devMode=false ]
(✓) Starting event handler
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/account_templ.go ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/account.templ ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/admin_templ.go ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/admin.templ ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/catalog_templ.go ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/client_templ.go ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/client.templ ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/catalog.templ ]
(✓) Generated code [ file=/Users/dx/source/teh-go/internal/app/component/admin.templ in=4.315625ms ]
(✓) First post-generation event received, starting proxy
(✓) No proxy URL specified, not starting proxy
(✓) Generated code [ file=/Users/dx/source/teh-go/internal/app/component/client.templ in=145.065375ms ]
(✓) Dev mode not enabled, process can finish early
(✓) Generated code [ file=/Users/dx/source/teh-go/internal/app/component/account.templ in=208.923875ms ]
(✓) Generated code [ file=/Users/dx/source/teh-go/internal/app/component/catalog.templ in=870.116125ms ]
(✓) Post-generation event channel closed, exiting
(✓) Waiting for push handler to complete
(✓) Waiting for event handler to complete
(✓) Waiting for post-generation handler to complete
(✓) Complete [ updates=4 duration=871.431375ms ]
templ generate -v  1.37s user 0.28s system 185% cpu 0.888 total

System: MacBook Pro M1, Sonoma 14.2.1, but I also tested this on Lenovo laptop under Windows 10 WSL.

[a-h]

Thanks, I suspect that the regression in performance is coming from the new Go expression parser which uses the Go AST.

It doesn't do it in a particularly smart way, it attempts to find the longest section of valid Go code by parsing from where it is, then extending to the next boundary token, and trying again. Previously, it used a heuristic of not parsing the Go code at all and looked for the closing } (taking into account strings and {} pairs in the Go code.), so it was doing a lot less work.

I'm sure I can improve that. The performance was acceptable on small files, but probably gets a lot worse the larger the file.

I'll come up with a test file, and rework that.

[joerdav]

I think a big win here would be to resolve the fact that the Inspect function of the go ast, continues to iterate even past the point where the function returns false, I've made a function which makes the API simpler, but it still iterates past a "found" node.

[a-h]

Got a PR in flight. I dropped down to the token scanner level instead.

Care to both review and try it out? #605

[joerdav]

500x faster is no small feat, will take a look!

[dv1x3r]

@a-h PR #605 works great with my chunky file, 500x thank you!

╰─ templ version
v0.2.613

╰─ time templ generate -v
(!) templ version check: generator v0.2.613 is newer than templ version v0.2.543 found in go.mod file, consider running `go get -u github.com/a-h/templ` to upgrade
(✓) Starting post-generation handler
(✓) Walking directory [ path=/Users/dx/source/teh-go devMode=false ]
(✓) Starting event handler
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/admin_templ.go ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/client_templ.go ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/client.templ ]
(✓) Processing file [ file=/Users/dx/source/teh-go/internal/app/component/admin.templ ]
(✓) Generated code [ file=/Users/dx/source/teh-go/internal/app/component/admin.templ in=1.898375ms ]
(✓) First post-generation event received, starting proxy
(✓) No proxy URL specified, not starting proxy
(✓) Dev mode not enabled, process can finish early
(✓) Generated code [ file=/Users/dx/source/teh-go/internal/app/component/client.templ in=179.932625ms ]
(✓) Post-generation event channel closed, exiting
(✓) Waiting for push handler to complete
(✓) Waiting for event handler to complete
(✓) Waiting for post-generation handler to complete
(✓) Complete [ updates=2 duration=180.303208ms ]
templ generate -v  0.25s user 0.11s system 183% cpu 0.196 total

[a-h]

PR merged. This change will be in the next release.