Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization updating memory usage via mincore syscall is expensive #1234

Open
zhuangel opened this issue Nov 19, 2019 · 3 comments
Open

Optimization updating memory usage via mincore syscall is expensive #1234

zhuangel opened this issue Nov 19, 2019 · 3 comments

Comments

@zhuangel
Copy link
Contributor

@zhuangel zhuangel commented Nov 19, 2019

When sentry update memory usage, mincore syscall will be used to check whether pages are resident in memory, this is expensive for CPU utilization, because syscall in sentry kernel will invoke a vmexit and kernel syscall.

Mincore syscall will cause about 1%~%5 CPU utilization in all Guest ring 0 and Host ring 0 CPU utilization of our Java / Go application container.

@zhuangel

This comment has been minimized.

Copy link
Contributor Author

@zhuangel zhuangel commented Nov 19, 2019

The issue could be emulated with following case, a memory allocate and free program, a meminfo read program, run this two program.

memory allocate and free program

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>

#define MAX_SIZE (102410241024)
#define PAGE_SIZE 4096
#define PAGE_NUM (MAX_SIZE/PAGE_SIZE)

void main(void)
{
unsigned char *ptr[PAGE_NUM];
unsigned long idx;

while (1) {
	for (idx=0;idx<PAGE_NUM;idx++)
		ptr[idx] = (unsigned char *) malloc(4*1024);
	usleep(10);
	for (idx=0;idx<PAGE_NUM;idx++) {
		ptr[idx][idx%4096] = idx;
	}
	usleep(10);
	for (idx=0;idx<PAGE_NUM;idx++)
		free(ptr[idx]);
	usleep(10);
}

}

/proc/meminfo reading program

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>

int main(void){
int fd;
size_t size, total;
unsigned char buf[4096];

while (1) {
	fd = open("/proc/meminfo",O_RDONLY);
	size = read(fd, buf, 4096);
	close(fd);
	total += size;
	usleep(10);
}
return total;

}

@amscanne

This comment has been minimized.

Copy link
Collaborator

@amscanne amscanne commented Nov 19, 2019

There are likely other implications (e.g. huge pages) but I’d be curious what happens if the MapUnit in kvm is set to usermem.PageSize.

In theory, we should be able to actually mark this as used on the fault itself, and there should be no mincore required.

@zhuangel

This comment has been minimized.

Copy link
Contributor Author

@zhuangel zhuangel commented Nov 20, 2019

usermem.PageSize

Thanks @amscanne , I tried to change MapUnit into usermem.PageSize, test in my test environment, the mincore stil has the same cost of CPU.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.