Skip to content

Add a dir-entry setting to volumes.config#11838

Merged
zwoop merged 1 commit into
apache:masterfrom
zwoop:DirEntries
Nov 14, 2024
Merged

Add a dir-entry setting to volumes.config#11838
zwoop merged 1 commit into
apache:masterfrom
zwoop:DirEntries

Conversation

@zwoop
Copy link
Copy Markdown
Contributor

@zwoop zwoop commented Nov 1, 2024

The idea / purpose of this is to allow for creating a volume which has a different configuration for the number of directory entries. The use case would be in a mixed environment where you have a mix of very large and very small objects, and don't want to waste a lot of memory for unused directory entries.

Example use:

volume=1 scheme=http size=20%
volume=2 scheme=http size=20%
volume=3 scheme=http size=20%
volume=4 scheme=http size=20%
volume=5 scheme=http size=20% avg_obj_size=1024

And then assign volume5 to one or many hosts / domains via hosting.config.

Before using avg_obj_size, we'd get the same number of directories per volume:

root@frigg:/opt/ats-10# ./bin/traffic_ctl  metric match 'direntries.total'
proxy.process.cache.direntries.total 4183592
proxy.process.cache.direntries.used 0
proxy.process.cache.volume_1.direntries.total 835072
proxy.process.cache.volume_2.direntries.total 835072
proxy.process.cache.volume_3.direntries.total 835072
proxy.process.cache.volume_4.direntries.total 839188
proxy.process.cache.volume_5.direntries.total 839188

after enabling the new configuration on volume5:

root@frigg:/opt/ats-10# ./bin/traffic_ctl  metric match 'direntries.total'
proxy.process.cache.direntries.total 29700544
proxy.process.cache.volume_1.direntries.total 835072
proxy.process.cache.volume_2.direntries.total 835072
proxy.process.cache.volume_3.direntries.total 835072
proxy.process.cache.volume_4.direntries.total 839188
proxy.process.cache.volume_5.direntries.total 26356140

@zwoop zwoop added this to the 10.1.0 milestone Nov 1, 2024
@zwoop zwoop requested a review from bryancall November 1, 2024 01:06
@zwoop zwoop self-assigned this Nov 1, 2024
@zwoop zwoop force-pushed the DirEntries branch 4 times, most recently from 9c347c6 to 2846cae Compare November 1, 2024 02:08
bryancall
bryancall previously approved these changes Nov 4, 2024
@zwoop zwoop merged commit b38ef43 into apache:master Nov 14, 2024
@zwoop zwoop deleted the DirEntries branch November 14, 2024 23:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants