Skip to content

[chore](workflow) Improve the cache hit ratio in BE UT (Clang)#20871

Merged
adonis0147 merged 1 commit intoapache:masterfrom
adonis0147:master
Jun 16, 2023
Merged

[chore](workflow) Improve the cache hit ratio in BE UT (Clang)#20871
adonis0147 merged 1 commit intoapache:masterfrom
adonis0147:master

Conversation

@adonis0147
Copy link
Contributor

@adonis0147 adonis0147 commented Jun 15, 2023

Proposed changes

In workflow BE UT (Clang), we set up the ldb_toolchain before we build the UT. During generating the toolchain tools, the script modifies the RPATH and dynamic linker of the executables which making the mtime of compilers change in every build.

By default, Ccache computes the hash of the compilers' mtime and size to check the compilers whether they are consistent with the ones which were used to generate the caches. If the compilers change, the caches can not be hit. We should change the default behavior of Ccache by setting the configuration compiler_check (CCACHE_COMPILERCHECK) to increase the cache hit ratio in the workflow.

Reference: https://ccache.dev/manual/latest.html#_configuration_options

compiler_check (CCACHE_COMPILERCHECK)
By default, ccache includes the modification time (“mtime”) and size of the compiler in the hash to ensure that results retrieved from the cache are accurate. If compiler plugins are used, these plugins will also be added to the hash. This option can be used to select another strategy. Possible values are:

content
Hash the content of the compiler binary. This makes ccache very slightly slower compared to mtime, but makes it cope better with compiler upgrades during a build bootstrapping process.

mtime
Hash the compiler’s mtime and size, which is fast. This is the default.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@adonis0147
Copy link
Contributor Author

run buildall

Copy link
Contributor

@BiteTheDDDDt BiteTheDDDDt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 16, 2023
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@adonis0147 adonis0147 merged commit a52fc3e into apache:master Jun 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants