Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cache test level config in logger::entry #1044

Merged
merged 1 commit into from
May 5, 2022

Conversation

hannaeko
Copy link
Member

@hannaeko hannaeko commented Mar 3, 2022

Purpose

Reduce the amount of time used cloning the test levels config.
Each call to Zonemaster::Engine::Profile->effective->get clone the result if it is not a scalar. For the test levels, this means that there will be a lot of calls to clone a fairly big hash. Here are the profiling info for a single test measured with PERL5OPT=-d:NYTProf ./zonemaster-cli --level info zonemaster.net --show-module --show-testcase.

calls exclusive time inclusive time
Zonemaster::Engine::Logger::Entry::level
before 5841 411ms 3390ms
after 5842 102ms 122ms
Zonemaster::Engine::Profile::get
before 35873 361ms 3570ms
after 24838 245ms 726ms

Context

Ported from https://gitlab.rd.nic.fr/zonemaster/zonemaster-engine/-/blob/nightly-ronde/lib/Zonemaster/Engine/Logger/Entry.pm

Changes

  • Cache the test levels for the time of the test

How to test this PR

Testing a domain should work as usual: ./zonemaster-cli --level info zonemaster.net --show-module --show-testcase

@hannaeko hannaeko added this to the v2022.1 milestone Mar 3, 2022
@hannaeko hannaeko requested review from mattias-p, matsduf, a user and tgreenx March 3, 2022 15:58
ghost
ghost previously approved these changes Mar 14, 2022
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvement. Just a thought, do we really need to clone the data even once? Can't we just pass a ref around?

matsduf
matsduf previously approved these changes Mar 25, 2022
@mattias-p
Copy link
Member

Just a thought, do we really need to clone the data even once? Can't we just pass a ref around?

I designed the Profile module interface to avoid leaking references to internal mutable data structures. The idea is that it'll make our lives easier.

@mattias-p
Copy link
Member

Won't this break the ability to switch between profiles in Backend? Maybe reset_config() could be called from the "run" methods in Zonemaster::Engine::Test or something?

@matsduf
Copy link
Contributor

matsduf commented Mar 25, 2022

Won't this break the ability to switch between profiles in Backend?

@blacksponge, has switching been tested?

@matsduf matsduf dismissed their stale review March 25, 2022 22:08

See comment by @mattias-p.

@hannaeko
Copy link
Member Author

hannaeko commented Apr 7, 2022

Won't this break the ability to switch between profiles in Backend? Maybe reset_config() could be called from the "run" methods in Zonemaster::Engine::Test or something?

As the profile is only set in the child process this is not breaking profile switching. To be sure I tried to run two consecutive tests with different profile and it was correctly executed.

@matsduf
Copy link
Contributor

matsduf commented Apr 8, 2022

I will re-review when the conflict has been resolved.

@hannaeko
Copy link
Member Author

I have rebased on top of develop, please re-review

Copy link
Contributor

@matsduf matsduf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve.

@hannaeko hannaeko merged commit 5df479b into zonemaster:develop May 5, 2022
@marc-vanderwal marc-vanderwal added the S-ReleaseTested Status: The PR has been successfully tested in release testing label Jun 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-ReleaseTested Status: The PR has been successfully tested in release testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants