-
Notifications
You must be signed in to change notification settings - Fork 281
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot run container with more than 3GB memory #1094
Comments
cc @darrenstahlmsft |
What is the output of This failure is in the Windows platform, not Docker, which is why Docker is not able to provide a better error message. As a first step, run Debug-ContainerHost.ps1 from here Hopefully the real error is in the event logs (The invalid handle is just a red herring) and that script will pick it up, otherwise I can provide more detailed logging instructions. |
Checked the HyperV event logs, unfortunately they were not helpful:
Hyper-V-Compute:
I ran Debug-ContainerHost.ps1, all tests passed. The ouput (includes
|
Sorry for the delay, lots going on. Fortunately I work on the Hyper-V team, so these error codes are useful to me 😄 The utility VM is crashing during first boot due to a memory access violation. I'm working on getting debug steps for this from some internal folks, but it's failing in a place that's tough to get useful logs from without a debugger attached. First, I have a hunch this may be due to processor address widths. Could you reply with the output of the following Powershell command on both the working machine, as well as the failing machine:
I'll get back with more complete debugging instructions soon. I'm very busy at the moment, so it might be a bit before I have full debug instructions. |
The output of the command above on both machines: https://www.diffchecker.com/xf7hUTVq |
Nothing obvious from those. I'll provide more detailed debugging instructions as soon as I have time to test them myself. |
@darrenstahlmsft thank you for looking into this issue! I'm looking forward for the debug instructions. |
I ran into the memory limit on another machine, but in this case it was 2GB (it fails when creating a container with 3GB+ memory). Also the error message is different: On what does the amount of memory that can be allocated depend? |
The timeout due to memory limits is new to me. Is this a different Windows build? What is the output from I don't have full details on what the memory limit depends on right now, but I'll get those. I'll see if I can find some docs. I have some steps to get a crash dump from a teammate here at Microsoft, but neither of us are able to cause a crash in the UtilityVM (We've tried a number of ways, no dice) in order to confirm that the crash dump is correctly written, so it might take a few attempts to get this to work.
The parent template is located at : Send both of those vhdx files to me at darst@microsoft.com and I can see if the crash produced a memory dump. Don't forget to undo the registry change to enable preserving crashed VMs or your disk might fill up fast if they keep crashing
Let me know if you have any issues with these steps. |
Thanks for the instructions! You are right, Windows version is the key. After upgrading the first machine to version 16299 (it had 14393 before) I got the same error message. More exactly, now I have two machines with the same hardware and the same Windows version. When I try run a container with 5GB memory I get the timeout issue mentioned above. I had no issues following your steps, although note that the PreserveVmOnErrorDestruct registry setting did not work on build 14393 (VHDX files were not kept). I sent the two VHDX files to the address provided by you. |
Got them. I'll let you know what we find. Ah, yes, I understand what the new error is from. The new error is the real error, as the failure is that the VM fails to start, so we timeout while waiting for it to connect. The |
@apetres I sent updated instructions via email, just updating here in case you missed them 😄 |
Hi, I have the same problem, my machine has 23gb of memory as soon as I try to start a container with 3 gb or more I got the timeout message. |
@SergeBerwert Can you post the output of Powershell command |
|
@apetres @SergeBerwert Just to confirm, the machines that this is failing on is a physical machine, not a VM or similar, correct? Also, if you could post your current output of |
Yes, in my case both machines are physical machines.
|
Yes this war the date from the Host Computer not the container. My Docker Info is:
|
Just a small update. We're tracking this issue internally. This occurs on certain processors that don't support 1GB pages. We are not aware of any mitigations (other than using a different processor), and I don't have an ETA on a fix yet. I'll update as I know more. |
Having same problem (max 2GB usable, more fail). Physical machine with 16GB RAM. Procesor info:
|
I'm also experiencing this issue with a Windows 10 host running inside VMware Fusion for macOS on an iMac (21.5-inch, Mid 2011) Intel(R) Core(TM) i5-2400S CPU @ 2.50GHz. I was getting the same error message as OP for values > 3GB, but between 1.5 and 3 I got a different error:
It ended up working with 1.5 gb, which is marginally better than the default 1GB, but the next largest I tried was 2.5, so maybe there's a higher limit for me that I can get to. As far as mitigation is concerned, if it ultimately is a hardware restriction, it would be nice if docker could recognize this and throw a helpful error message (or warn and fallback to the highest possible supported memory size) when it catches itself being sent a -m that is beyond what the host can support. Of course fixing it in Hyper-V would be better. |
Thank you @darrenstahlmsft for the update, this is important information to me. I feared this issue is a general instability, which would kill my use-case. Now theres at least a way to work around it by choosing the right Host system. I ran into the same issue on a Windows 10 physical machine when trying to set
I assume it doesn't support those hugepages. Edit: Indeed, with the help of Coreinfo and the |
Would you please provide more technical info on this? |
Hi! We have a sopport-contract with MS. But MS does not support Docker. We were not able to figure out, how we could start the Hyper-V with these VHDX Files. This would help us a lot, because in that case MS would help us to solve this problem. Thanks |
But it does support Windows Docker implementation. This bug looks like specific to implementation. |
same for me. Interesting part, it worked end of last year for a presentation, but after reinstallation of Windows 10 this year, i wasn't able to run any docker image with +3gb, anymore. Get-WmiObject win32_Processor | fl Coreinfo -f SMX - Supports Intel trusted execution NX * Supports no-execute page protection docker info |
I have related to docker memory limit issue and I would like to know if any Intel cpu model/generation requirements such as supported cpu flags(Ex: "PAGE1GB - Supports 1 GB large pages") is required to run this nav container? I am running docker on Microsoft Windows 10 Enterprise edition Windows Event Application log: According my understanding and reading this link these are memory issues: |
Just used another PC with more modern cpu(2015) architecture and now docker ce(windows) can create and run nav/bc container without any issue. |
Issues go stale after 90d of inactivity. Prevent issues from auto-closing with an If this issue is safe to close now please do so. Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. |
/remove-lifecycle stale |
@timotei Not a workaround, but just FYI: with this change the current issue might go away. |
It seems that this issue has been fixed in the latest Windows 10 release. The combination that worked for me:
|
Closed issues are locked after 30 days of inactivity. If you have found a problem that seems similar to this, please open a new issue. Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. |
Expected behavior
docker run -m 5GB microsoft/windowsservercore
creates a container with 5GB memory (the amount can be 4GB or greater)Actual behavior
An error message is displayed:
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: container f6d27506ece6b1135a8d92b142f0e81802046dc3a8c05c2e99c8dfde54679a0e encountered an error during CreateContainer: failure in a Windows system call: The handle is invalid. (0x6).
Information
If the memory specified is less or equal then 3GB, the container runs successfully, so I don't think this is a network-related issue.
I tried to reproduce the issue on another machine with the same version of Docker, without success (meaning that I could create a container with more than 3GB memory).
Could you recommend a way of debugging this issue?
(Side question: could you recommend a way to obtain more detailed error message from docker? In this case it would help if I know what kind of handle is invalid)
Steps to reproduce the behavior
docker run -m 5GB microsoft/windowsservercore
The text was updated successfully, but these errors were encountered: