New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to change memory limitation of Ubuntu sub system. #92

Closed
umurgdk opened this Issue Apr 9, 2016 · 33 comments

Comments

Projects
None yet
@umurgdk

umurgdk commented Apr 9, 2016

I was trying to compile crystal programming language on ubuntu subsystem lately, unfortunately it fails in a way which doesn't makes sense. After some research crystal devs worried about the available ram. Then I thought maybe ubuntu sub system has some memory limitation and bam! there it is. Ubuntu sub systems sees my machine has 1Gb of total ram, instead of 4Gb.

So how can I change/remove this memory limitation?

@mchristen

This comment has been minimized.

Show comment
Hide comment
@mchristen

mchristen Apr 9, 2016

I also ran into this while trying to compile the ruby runtime through rbenv/ruby-install.

It would be nice if this subsystem had access to the full amount of RAM that Windows has, I'm not sure I could even fit my entire Ruby development environment in memory with only 1GB available.

./tool/runruby.rb:99:in `exec': Cannot allocate memory - ./ruby (Errno::ENOMEM)
        from ./tool/runruby.rb:99:in `<main>'

mchristen commented Apr 9, 2016

I also ran into this while trying to compile the ruby runtime through rbenv/ruby-install.

It would be nice if this subsystem had access to the full amount of RAM that Windows has, I'm not sure I could even fit my entire Ruby development environment in memory with only 1GB available.

./tool/runruby.rb:99:in `exec': Cannot allocate memory - ./ruby (Errno::ENOMEM)
        from ./tool/runruby.rb:99:in `<main>'
@Talvish

This comment has been minimized.

Show comment
Hide comment
@Talvish

Talvish Apr 11, 2016

No clue how to change it but if you want see how much memory is available use free -m. Currently shows 1006 total with 664 free for me ... nothing running at the moment though oddly when I had redis running free showed the exact same values.

I'm personally having issues with PostgreSQL running due to lack of shared memory (though I'll file a separate bug for that).

Talvish commented Apr 11, 2016

No clue how to change it but if you want see how much memory is available use free -m. Currently shows 1006 total with 664 free for me ... nothing running at the moment though oddly when I had redis running free showed the exact same values.

I'm personally having issues with PostgreSQL running due to lack of shared memory (though I'll file a separate bug for that).

@hut8

This comment has been minimized.

Show comment
Hide comment
@hut8

hut8 Apr 12, 2016

Looks like there's no swap implemented either:

root@localhost:~# dd if=/dev/zero of=/swapfile count=1024 bs=1M
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 0.474613 s, 2.3 GB/s
root@localhost:~# mkswap /swapfile
Setting up swapspace version 1, size = 1048572 KiB
no label, UUID=6e26185d-0408-496c-ad8f-1ee78deb64af
root@localhost:~# swapon /swapfile
swapon: /swapfile: swapon failed: Invalid argument

hut8 commented Apr 12, 2016

Looks like there's no swap implemented either:

root@localhost:~# dd if=/dev/zero of=/swapfile count=1024 bs=1M
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 0.474613 s, 2.3 GB/s
root@localhost:~# mkswap /swapfile
Setting up swapspace version 1, size = 1048572 KiB
no label, UUID=6e26185d-0408-496c-ad8f-1ee78deb64af
root@localhost:~# swapon /swapfile
swapon: /swapfile: swapon failed: Invalid argument
@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis Apr 13, 2016

Member

Thanks for reporting this, I will dig around and see what I can find. We are probably missing a /proc file that those tools are using he query available memory.

Member

benhillis commented Apr 13, 2016

Thanks for reporting this, I will dig around and see what I can find. We are probably missing a /proc file that those tools are using he query available memory.

@avens19

This comment has been minimized.

Show comment
Hide comment
@avens19

avens19 Apr 13, 2016

Bump. Can't install ruby as-is

avens19 commented Apr 13, 2016

Bump. Can't install ruby as-is

@hut8

This comment has been minimized.

Show comment
Hide comment
@hut8

hut8 commented Apr 13, 2016

@avens19 you can also just install a package, such as https://launchpad.net/~brightbox/+archive/ubuntu/ruby-ng

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis Apr 14, 2016

Member

I looked into this and this is due to some hardcoded values that we have in procfs (files in /proc). I'd suggest creating a post on our uservoice page that mentions the apps and scenarios that this is currently blocking.

https://wpdev.uservoice.com/forums/266908-command-prompt-console/category/161892-bash

Member

benhillis commented Apr 14, 2016

I looked into this and this is due to some hardcoded values that we have in procfs (files in /proc). I'd suggest creating a post on our uservoice page that mentions the apps and scenarios that this is currently blocking.

https://wpdev.uservoice.com/forums/266908-command-prompt-console/category/161892-bash

@smurawski

This comment has been minimized.

Show comment
Hide comment
@smurawski

smurawski Apr 15, 2016

Member

Hey @benhillis Thanks for the acknowledgement of the issue. I filed a user voice issue on my specific blockers - https://wpdev.uservoice.com/forums/266908-command-prompt-console-bash-on-ubuntu-on-windo/suggestions/13444257-unable-to-perform-simple-development-tasks-in-bash

Unfortunately, that memory cap makes the bash experience pointless for my workflow. Hopefully we'll see that as a configurable option in the near future. I know that would help the workflow of a number of the devs at my company.

Member

smurawski commented Apr 15, 2016

Hey @benhillis Thanks for the acknowledgement of the issue. I filed a user voice issue on my specific blockers - https://wpdev.uservoice.com/forums/266908-command-prompt-console-bash-on-ubuntu-on-windo/suggestions/13444257-unable-to-perform-simple-development-tasks-in-bash

Unfortunately, that memory cap makes the bash experience pointless for my workflow. Hopefully we'll see that as a configurable option in the near future. I know that would help the workflow of a number of the devs at my company.

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis Apr 15, 2016

Member

Thanks for filing the user voice. Do you happen to know the specific /proc files that the tools are accessing? Taking an strace and looking just above where the failure is occurring should shed some light. That would greatly help us prioritize this.

Member

benhillis commented Apr 15, 2016

Thanks for filing the user voice. Do you happen to know the specific /proc files that the tools are accessing? Taking an strace and looking just above where the failure is occurring should shed some light. That would greatly help us prioritize this.

@smurawski

This comment has been minimized.

Show comment
Hide comment
@smurawski

smurawski Apr 18, 2016

Member

@benhillis Thanks for the response - I grabbed strace logs and posted them to this gist https://gist.github.com/smurawski/25c7d0279a84951a133e0b3f4bdf22a6 . Let me know if I can help in any other way.

Member

smurawski commented Apr 18, 2016

@benhillis Thanks for the response - I grabbed strace logs and posted them to this gist https://gist.github.com/smurawski/25c7d0279a84951a133e0b3f4bdf22a6 . Let me know if I can help in any other way.

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis Apr 18, 2016

Member

@smurawski Thanks, that was very useful. It looks like these two procfs files need to be implemented. I've also added this info to your uservoice page. I'll bring this up with the team and see if we can get somebody working on this ASAP.

/proc/meminfo
/proc/self/status
Member

benhillis commented Apr 18, 2016

@smurawski Thanks, that was very useful. It looks like these two procfs files need to be implemented. I've also added this info to your uservoice page. I'll bring this up with the team and see if we can get somebody working on this ASAP.

/proc/meminfo
/proc/self/status
@mondus

This comment has been minimized.

Show comment
Hide comment
@mondus

mondus Apr 19, 2016

Hi @benhillis, another confirmation of the memory limitation. This prevents gcc from building many larger applications. For example 'pip install Image' will fail ass gcc is unable to allocate enough memory. Similarly I have found that creating swap space is not possible. Having a configuration for the available memory in the Linux Subsystem would be incredibly useful. I have added my specific example to @smurawski voice issue.

mondus commented Apr 19, 2016

Hi @benhillis, another confirmation of the memory limitation. This prevents gcc from building many larger applications. For example 'pip install Image' will fail ass gcc is unable to allocate enough memory. Similarly I have found that creating swap space is not possible. Having a configuration for the available memory in the Linux Subsystem would be incredibly useful. I have added my specific example to @smurawski voice issue.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Apr 25, 2016

The same issue exists while trying to install some specific versions of ruby with rvm.

ghost commented Apr 25, 2016

The same issue exists while trying to install some specific versions of ruby with rvm.

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis Apr 25, 2016

Member

I'm testing a fix for reporting the actual amount of free memory. I'd like to validate that some of the scenarios in this thread will be fixed by this change, could somebody share with me the command line to install a specific ruby gem that was failing previously? (I'm a total ruby noob)

Member

benhillis commented Apr 25, 2016

I'm testing a fix for reporting the actual amount of free memory. I'd like to validate that some of the scenarios in this thread will be fixed by this change, could somebody share with me the command line to install a specific ruby gem that was failing previously? (I'm a total ruby noob)

@russalex russalex added the bug label Apr 25, 2016

@hut8

This comment has been minimized.

Show comment
Hide comment
@hut8

hut8 Apr 25, 2016

I didn't see any gems installations that failed in this thread, it was the programming language itself. With Ruby, most people use some "ruby manager" like rvm, and often rvm will download the ruby source, patch it and compile it. So, if you install rvm and ruby with:

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
\curl -sSL https://get.rvm.io | bash -s stable --ruby=2.3.0

that should cause a failure.

hut8 commented Apr 25, 2016

I didn't see any gems installations that failed in this thread, it was the programming language itself. With Ruby, most people use some "ruby manager" like rvm, and often rvm will download the ruby source, patch it and compile it. So, if you install rvm and ruby with:

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
\curl -sSL https://get.rvm.io | bash -s stable --ruby=2.3.0

that should cause a failure.

@mondus

This comment has been minimized.

Show comment
Hide comment
@mondus

mondus Apr 25, 2016

Not a ruby gem but

pip install image

Fails with a memory allocation error from gcc during the build.

On Monday, 25 April 2016, Liam notifications@github.com wrote:

I didn't see any gems installations that failed in this thread, it was the
programming language itself. With Ruby, most people use some "ruby manager"
like rvm, and often rvm will download the ruby source, patch it and
compile it. So, if you install rvm and ruby with:

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
\curl -sSL https://get.rvm.io | bash -s stable --ruby=2.3.0

that should cause a failure.


You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#92 (comment)

Dr Paul Richmond (Research Fellow) - University of Sheffield
http://www.paulrichmond.staff.shef.ac.uk
p.richmond@sheffield.ac.uk

Department of Computer Science http://www.sheffield.ac.uk/dcs
Sheffield NVIDIA CUDA Research Centre
http://gpucomputing.sites.sheffield.ac.uk/
Advanced Computing Research Centre http://www.acrc.com

mondus commented Apr 25, 2016

Not a ruby gem but

pip install image

Fails with a memory allocation error from gcc during the build.

On Monday, 25 April 2016, Liam notifications@github.com wrote:

I didn't see any gems installations that failed in this thread, it was the
programming language itself. With Ruby, most people use some "ruby manager"
like rvm, and often rvm will download the ruby source, patch it and
compile it. So, if you install rvm and ruby with:

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
\curl -sSL https://get.rvm.io | bash -s stable --ruby=2.3.0

that should cause a failure.


You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#92 (comment)

Dr Paul Richmond (Research Fellow) - University of Sheffield
http://www.paulrichmond.staff.shef.ac.uk
p.richmond@sheffield.ac.uk

Department of Computer Science http://www.sheffield.ac.uk/dcs
Sheffield NVIDIA CUDA Research Centre
http://gpucomputing.sites.sheffield.ac.uk/
Advanced Computing Research Centre http://www.acrc.com

@smurawski

This comment has been minimized.

Show comment
Hide comment
@smurawski

smurawski Apr 25, 2016

Member

@benhillis while I didn't have a specific gem build fail, if you install the chefdk and run chef verify, you'll get an out of memory error.

curl -L https://omnitruck.chef.io/install.sh | sudo bash -s -- -P chefdk
chef verify
Member

smurawski commented Apr 25, 2016

@benhillis while I didn't have a specific gem build fail, if you install the chefdk and run chef verify, you'll get an out of memory error.

curl -L https://omnitruck.chef.io/install.sh | sudo bash -s -- -P chefdk
chef verify
@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Apr 26, 2016

Is there a fix for this issue any time soon? I've experienced the problem with ruby, postgres and npm.

ghost commented Apr 26, 2016

Is there a fix for this issue any time soon? I've experienced the problem with ruby, postgres and npm.

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis Apr 26, 2016

Member

Unfortunately it looks like rmv is failing due to some missing execve syscall support and not the incorrect memory reporting.

I will try more of the commands in this thread and see if others are actually failing due to the incorrect proc meminfo file.

Member

benhillis commented Apr 26, 2016

Unfortunately it looks like rmv is failing due to some missing execve syscall support and not the incorrect memory reporting.

I will try more of the commands in this thread and see if others are actually failing due to the incorrect proc meminfo file.

@mchristen

This comment has been minimized.

Show comment
Hide comment
@mchristen

mchristen Apr 26, 2016

I was attempting to install ruby 2.3 from source using ruby-install as a plugin inside of rbenv.

mchristen commented Apr 26, 2016

I was attempting to install ruby 2.3 from source using ruby-install as a plugin inside of rbenv.

@smurawski

This comment has been minimized.

Show comment
Hide comment
@smurawski

smurawski Apr 26, 2016

Member

@benhillis are there related ETW traces I can grab that would help? I can replicate the memory error with just about any step in my development workflow working on Chef via the ChefDK and its omnibus ruby install (ChefDK packages up ruby and a bunch of gems out of the box to keep things self contained)

Member

smurawski commented Apr 26, 2016

@benhillis are there related ETW traces I can grab that would help? I can replicate the memory error with just about any step in my development workflow working on Chef via the ChefDK and its omnibus ruby install (ChefDK packages up ruby and a bunch of gems out of the box to keep things self contained)

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis Apr 26, 2016

Member

@mondus - just tried pip install image on a build with my change that accurately reflects the amount of memory on the system and it installed just fine.

@smurawski - I tried the chef commands locally and they're failing due to some missing execve functionality that we're missing. I've filed an internal bug to track this.

Member

benhillis commented Apr 26, 2016

@mondus - just tried pip install image on a build with my change that accurately reflects the amount of memory on the system and it installed just fine.

@smurawski - I tried the chef commands locally and they're failing due to some missing execve functionality that we're missing. I've filed an internal bug to track this.

This was referenced Apr 27, 2016

@mondus

This comment has been minimized.

Show comment
Hide comment
@mondus

mondus Apr 29, 2016

@benhillis Superb is this build being rolled out via insider preview any time soon?

mondus commented Apr 29, 2016

@benhillis Superb is this build being rolled out via insider preview any time soon?

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis Apr 29, 2016

Member

@mondus Unfortunately I can't give a solid estimate because our team doesn't have a lot of control over when flights get released to the fast ring. These fixes were made recently so I'd guess a couple of weeks. Keep an eye out on our release notes.

Member

benhillis commented Apr 29, 2016

@mondus Unfortunately I can't give a solid estimate because our team doesn't have a lot of control over when flights get released to the fast ring. These fixes were made recently so I'd guess a couple of weeks. Keep an eye out on our release notes.

@yyq

This comment has been minimized.

Show comment
Hide comment
@yyq

yyq May 9, 2016

I also ran into this issue while trying to compile ruby from source code.

yyq commented May 9, 2016

I also ran into this issue while trying to compile ruby from source code.

@benhillis benhillis added the fixinbound label May 9, 2016

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis May 9, 2016

Member

@yyq - Are you using rvm to download and compile a specific version of ruby? I've verified the fix I discuss above allows ruby to compile and build through rvm.

Member

benhillis commented May 9, 2016

@yyq - Are you using rvm to download and compile a specific version of ruby? I've verified the fix I discuss above allows ruby to compile and build through rvm.

@umurgdk

This comment has been minimized.

Show comment
Hide comment
@umurgdk

umurgdk May 25, 2016

@benhillis is rvm building from source or using prebuilt ubuntu packages. if rvm can build it from source you should be able to build without it too. since the ram usage for the compiler doesn't change.

umurgdk commented May 25, 2016

@benhillis is rvm building from source or using prebuilt ubuntu packages. if rvm can build it from source you should be able to build without it too. since the ram usage for the compiler doesn't change.

@benhillis

This comment has been minimized.

Show comment
Hide comment
@benhillis

benhillis May 28, 2016

Member

Build 14352 will now report the actual amount of memory your system has. Swap file is reported as Windows pagefile size / usage. Please reopen if you see any issues on the new build, thanks!

Member

benhillis commented May 28, 2016

Build 14352 will now report the actual amount of memory your system has. Swap file is reported as Windows pagefile size / usage. Please reopen if you see any issues on the new build, thanks!

@benhillis benhillis closed this May 28, 2016

@benhillis benhillis added fixed and removed fixinbound labels May 28, 2016

@hach-que

This comment has been minimized.

Show comment
Hide comment
@hach-que

hach-que Nov 27, 2016

I'm still hitting out of memory errors on Build 14971, with only 72% memory used according to Task Manager:

image

image

hach-que commented Nov 27, 2016

I'm still hitting out of memory errors on Build 14971, with only 72% memory used according to Task Manager:

image

image

@webczat

This comment has been minimized.

Show comment
Hide comment
@webczat

webczat Nov 27, 2016

well, I do not know if I have a newest build, but just updated, trying to connect with weechat does not work still due to oom.

webczat commented Nov 27, 2016

well, I do not know if I have a newest build, but just updated, trying to connect with weechat does not work still due to oom.

@aurimus

This comment has been minimized.

Show comment
Hide comment
@aurimus

aurimus Mar 22, 2017

Why is this issue closed? I'm still having the same problem.

/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/bundler-1.14.6/lib/bundler/source/git/git_proxy.rb:162:in ``': Cannot allocate memory - git (Errno::ENOMEM)

aurimus commented Mar 22, 2017

Why is this issue closed? I'm still having the same problem.

/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/bundler-1.14.6/lib/bundler/source/git/git_proxy.rb:162:in ``': Cannot allocate memory - git (Errno::ENOMEM)
@aseering

This comment has been minimized.

Show comment
Hide comment
@aseering

aseering Mar 22, 2017

Contributor

Hi @aurimus -- what Windows build are you running? (Type ver at a regular Windows command prompt.)

Issues in this ticket tracker get closed when they have been addressed in Insider builds. New Insider builds are released every week or two, so it's possible to discuss and iterate on bugfixes in this bugtracker. However, the cost of rapid progress is potential instability -- sometimes a build may break things; then you have to revert and wait for the next release.

Those changes are batched and rolled out to regular Windows users roughly twice per year. (They are not released via regular Windows Update because Windows Update is only for stable Windows features and WSL is still very much in beta.)

Of course, this problem might also occur if your machine just plain doesn't have enough memory. If that's the case, there's not much that WSL can do :-)

Contributor

aseering commented Mar 22, 2017

Hi @aurimus -- what Windows build are you running? (Type ver at a regular Windows command prompt.)

Issues in this ticket tracker get closed when they have been addressed in Insider builds. New Insider builds are released every week or two, so it's possible to discuss and iterate on bugfixes in this bugtracker. However, the cost of rapid progress is potential instability -- sometimes a build may break things; then you have to revert and wait for the next release.

Those changes are batched and rolled out to regular Windows users roughly twice per year. (They are not released via regular Windows Update because Windows Update is only for stable Windows features and WSL is still very much in beta.)

Of course, this problem might also occur if your machine just plain doesn't have enough memory. If that's the case, there's not much that WSL can do :-)

@aurimus

This comment has been minimized.

Show comment
Hide comment
@aurimus

aurimus Mar 23, 2017

@aseering thanks, it worked after windows update.

aurimus commented Mar 23, 2017

@aseering thanks, it worked after windows update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment