Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specified logical memory size #3188

Open
fengjiachun opened this issue Jan 18, 2024 · 2 comments
Open

Specified logical memory size #3188

fengjiachun opened this issue Jan 18, 2024 · 2 comments

Comments

@fengjiachun
Copy link
Collaborator

fengjiachun commented Jan 18, 2024

Can we obtain a logical memory size here, which should be smaller than the system memory.
Consider the following scenario:
The user deploys our DB on a machine with a large amount of memory, and this machine is also running other processes.
They want our DB to occupy less memory. It would be better if we can provide a parameter to specify the memory size, for example, by specifying it through startup parameters.
Then in the get_sys_total_memory method, we return the minimum value between it and sys_total_memory: Min(logical_total_memory, sys_total_memory).

The same idea for the get_cpus()

Originally posted by @fengjiachun in #3165 (comment)

@waynexia
Copy link
Member

This idea looks cool. We already have many configs related to memory, and will have more in the future. They are useful when one wants to control them precisely to suit different workloads. But are sometimes frustrated if I do not care about that.

Those configs have default values, but absolute memory size can hardly be used in various environments. A more flexible way might be providing the default value in a relative way. E.g.: CacheA uses 10%, BufferB uses 25% etc.

BTW, sys_total_memory might be misleading -- it looks like configuring a hard limit. but we don't have those limits at present. Do you mean to provide a config to config the total memory used by cache and buffer? And with this and the above relative coefficient we can get the absolute memory size for each component. Like for an environment that has 10GB in total for cache and buffer, we can get CacheA is configured to 1GB and BufferB is 2.5GB

@zhongzc
Copy link
Contributor

zhongzc commented Jan 18, 2024

Can't we let users utilize the resource limitation mechanisms that come with the OS? I specifically asked the PR author to make get_sys_total_memory cgroup aware, so users could bind the process to a cgroup, or use containerization tools (like Docker) more directly to achieve the desired effect, even though it might not be as convenient as configuring the database directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants