Skip to content

Speed up builds through efficient loading of .ty headers#2503

Merged
plajjan merged 7 commits into
mainfrom
faster
Oct 31, 2025
Merged

Speed up builds through efficient loading of .ty headers#2503
plajjan merged 7 commits into
mainfrom
faster

Conversation

@plajjan

@plajjan plajjan commented Oct 30, 2025

Copy link
Copy Markdown
Contributor

We now store a bit more information in the .ty file and rearrange the fields so that it is possible to read these "header" fields quickly and efficiently, from the start up to end of headers. Freshness checks, import chasing and root generation now use the header to avoid decoding large sections. This reduces I/O and decoding on no-op and partial builds, making rebuilds faster.

Zlib compression has been removed in the process, but more as a temporary measure. It makes random lazy reading hard, but now that the header fields are moved to the start of the .ty file, we could probably still compress it. Nonetheless, I left it out. I want to switch to zstd which is faster and offers higher compression ratio at similar speeds (or higher speeds at same ratio) and I think it would also be interesting to apply compression to individual fields instead of the whole file. Meanwhile, the removal of zlib increases size by 10x, but .ty files are still smaller than the generated .h or .c files.

Fixes #2502.

@plajjan plajjan force-pushed the faster branch 2 times, most recently from 9eaa4c4 to c66d5d2 Compare October 30, 2025 20:13
Kristian Larsson added 5 commits October 30, 2025 21:26
We now store a bit more information in the .ty file and rearrange the
fields so that it is possible to read these "header" fields quickly and
efficiently, from the start up to end of headers. Freshness checks,
import chasing and root generation now use the header to avoid decoding
large sections. This reduces I/O and decoding on no-op and partial
builds, making rebuilds faster.
compilesFiles might be invoked for a small set of files, perhaps a
single file, but clearly generating an index for a project is only valid
for when we are working with the whole project, thus moving it to
buildProject.

It is a whole that the user could invoke `acton doc` before the project
is built, but that is not a new problem, it existed before.
Make sanitizer force exactly three spaces before the canonical "0.000 s"
token to avoid platform-specific padding in macOS 13 x86_64 CI logs.
This stabilizes project build goldens without changing the expected
content. Really really weird to see something like this, but hey..
@plajjan plajjan merged commit 64c7ddc into main Oct 31, 2025
35 checks passed
@plajjan plajjan deleted the faster branch October 31, 2025 06:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

YANG model change results in zig error

1 participant