Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker doesn't use memory or CPU options #145

Closed
gupta-ak opened this Issue Sep 22, 2017 · 15 comments

Comments

Projects
None yet
@gupta-ak
Copy link
Member

gupta-ak commented Sep 22, 2017

For example, if I try running mssql-server-linux image with -m=4g, I get this error sqlservr: This program requires a machine with at least 3250 megabytes of memory.

@PatrickLang

This comment has been minimized.

Copy link
Member

PatrickLang commented Oct 6, 2017

This needs an issue in moby/moby. Will probably require changes to moby/moby as well as opengcs

@jhowardmsft jhowardmsft self-assigned this Oct 6, 2017

@jhowardmsft

This comment has been minimized.

Copy link
Member

jhowardmsft commented Oct 6, 2017

Leave it here for now. It's already on my list.

@edrevo

This comment has been minimized.

Copy link

edrevo commented Jan 18, 2018

It would be great if the move to LCOW allowed enabling dynamic memory for Linux containers.

@PatrickLang

This comment has been minimized.

Copy link
Member

PatrickLang commented Jan 22, 2018

Good news :) Containers with Hyper-V isolation already allocate memory on-demand so it's more or less comparable to dynamic memory. This option puts a limit on the max memory size which is enforced by the hypervisor. You can see this behavior by looking at the memory usage of vmmem processes on the host. Those are a wrapper to hold the memory used by the container

@PatrickLang

This comment has been minimized.

Copy link
Member

PatrickLang commented Mar 6, 2018

ping @jhowardmsft - is this still in the works? do you have an issue in moby/moby yet?

@yusuf-gunaydin

This comment has been minimized.

Copy link

yusuf-gunaydin commented Mar 26, 2018

Is this change as simple as adding the the following code snippet to createLinux function in this file? Or, is there another limitation/option in HCS to be considered?

if spec.Windows.Resources != nil {
	if spec.Windows.Resources.CPU != nil {
		if spec.Windows.Resources.CPU.Count != nil {
			// This check is being done here rather than in adaptContainerSettings
			// because we don't want to update the HostConfig in case this container
			// is moved to a host with more CPUs than this one.
			cpuCount := *spec.Windows.Resources.CPU.Count
			hostCPUCount := uint64(sysinfo.NumCPU())
			if cpuCount > hostCPUCount {
				c.logger.Warnf("Changing requested CPUCount of %d to current number of processors, %d", cpuCount, hostCPUCount)
				cpuCount = hostCPUCount
			}
			configuration.ProcessorCount = uint32(cpuCount)
		}
		if spec.Windows.Resources.CPU.Shares != nil {
			configuration.ProcessorWeight = uint64(*spec.Windows.Resources.CPU.Shares)
		}
		if spec.Windows.Resources.CPU.Maximum != nil {
			configuration.ProcessorMaximum = int64(*spec.Windows.Resources.CPU.Maximum)
		}
	}
	if spec.Windows.Resources.Memory != nil {
		if spec.Windows.Resources.Memory.Limit != nil {
			configuration.MemoryMaximumInMB = int64(*spec.Windows.Resources.Memory.Limit) / 1024 / 1024
		}
	}
	if spec.Windows.Resources.Storage != nil {
		if spec.Windows.Resources.Storage.Bps != nil {
			configuration.StorageBandwidthMaximum = *spec.Windows.Resources.Storage.Bps
		}
		if spec.Windows.Resources.Storage.Iops != nil {
			configuration.StorageIOPSMaximum = *spec.Windows.Resources.Storage.Iops
		}
	}
}
@yusuf-gunaydin

This comment has been minimized.

Copy link

yusuf-gunaydin commented Mar 27, 2018

OK, the previous code does not work. However, if anyone is in need of this, adding a hardcoded value to the docker source code works. Add MemoryMaximumInMB: 4096 to configuration := &hcsshim.ContainerConfig in createLinux function inside client_local_windows.go. Then you can compile and use the patched dockerd.exe

@fizxmike

This comment has been minimized.

Copy link

fizxmike commented Apr 25, 2018

@yusuf-gunaydin I'm going to try this out... have you looked into possibly grabbing the -m parameter from inside createLinux function and applying that number to MemoryMaximumInMB? Maybe this PR does it!?

@brianweet

This comment has been minimized.

Copy link

brianweet commented Apr 26, 2018

@yusuf-gunaydin thanks for the tip, seems to work :)

@yusuf-gunaydin

This comment has been minimized.

Copy link

yusuf-gunaydin commented Apr 26, 2018

@fizxmike If I recall correctly I have tried the code I posted above, but it didn't work either for CPU or memory or neither. As I needed them both, I hardcoded the CPU and memory values. By the way CPU is also limited to 2 cores, if anyone needs the information.

I couldn't find a good way to debug docker sources. Println debugging and recompiling the whole source every time a small change is made prevents me from trying more.

@kethahel99

This comment has been minimized.

Copy link

kethahel99 commented May 22, 2018

Is this still being implemented? I'm running Docker Edge

PS C:\Users\C740763> docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 19
Server Version: 18.05.0-ce
Storage Driver: windowsfilter (windows) lcow (linux)
Windows:
LCOW:
Logging Driver: json-file
Plugins:
Volume: local
Network: ics l2bridge l2tunnel nat null overlay transparent
Log: awslogs etwlogs fluentd gelf json-file logentries splunk syslog
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 16299 (16299.431.amd64fre.rs3_release_svc_escrow.180502-1908)
Operating System: Windows 10 Enterprise Version 1709 (OS Build 16299.431)
OSType: windows
Architecture: x86_64
CPUs: 8
Total Memory: 15.38GiB
Name: 23360jnbsis126l
ID: OMBQ:JNTL:FTZF:UZSI:72I3:OXGQ:CYD7:VEIT:F5LJ:ZFDO:CFK5:Y3G2
Docker Root Dir: D:\Docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: -1
Goroutines: 28
System Time: 2018-05-23T00:33:26.6804568+02:00
EventsListeners: 2
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

@LukeRH

This comment has been minimized.

Copy link

LukeRH commented Jul 26, 2018

Hi, Is there an estimate on when this will be fixed?
Thanks

@Iristyle

This comment has been minimized.

Copy link

Iristyle commented Nov 29, 2018

There is a PR up in Moby at moby/moby#37296 ... but it's unclear how close to approval it is. Some folks have built from source with success.

@jterry75

This comment has been minimized.

Copy link
Member

jterry75 commented Nov 29, 2018

@jhowardmsft - None of these asks are OpenGCS specific and indeed the platform already supports them. Its just a config issue in Docker correct? Can we close this to avoid confusion? Or am I missing something fundamental here?

@jhowardmsft

This comment has been minimized.

Copy link
Member

jhowardmsft commented Dec 3, 2018

@jterry75 Yes, I'm closing it here. The fix needs to go in moby, although maintainers there are not accepting it. There have been several PR attempts in moby to fix.

@jhowardmsft jhowardmsft closed this Dec 3, 2018

@Microsoft Microsoft locked and limited conversation to collaborators Dec 3, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.