New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some way to increase disk space from Vagrantfile #2339

Closed
peterbraden opened this Issue Oct 7, 2013 · 39 comments

Comments

Projects
None yet
@peterbraden
Copy link

peterbraden commented Oct 7, 2013

At the moment you can increase memory in the vagrantfile, but if you want to increase disk space you have to use the VBoxManage tool to enlarge the image. It would be great to be able to specify this in the vagrantfile.

@mitchellh

This comment has been minimized.

Copy link
Member

mitchellh commented Oct 15, 2013

You can do this already.

config.vm.customize "virtualbox" do |vb|
  vb.customize "pre-boot", ["..."]
end

It just isn't documented yet. But will be soon. Sorry!

@mitchellh mitchellh closed this Oct 15, 2013

@peterbraden

This comment has been minimized.

Copy link
Author

peterbraden commented Oct 15, 2013

what would I put in the pre-boot list - does it take the same commands as the VBoxManage tool?

@mitchellh

This comment has been minimized.

Copy link
Member

mitchellh commented Oct 15, 2013

Yes, it is VBoxManage.

@peterbraden

This comment has been minimized.

Copy link
Author

peterbraden commented Nov 13, 2013

This gives me:

"Bringing machine 'default' up with 'virtualbox' provider...
There are errors in the configuration of this machine. Please fix
the following errors and try again:

VirtualBox Provider:

  • virtualbox is not a valid event for customization. Valid events
    are: pre-import, pre-boot, post-boot"
@tmatilai

This comment has been minimized.

Copy link
Collaborator

tmatilai commented Nov 14, 2013

The example should be:

config.vm.provider "virtualbox" do |vb|
  vb.customize "pre-boot", ["..."]
end
@haf

This comment has been minimized.

Copy link

haf commented Dec 7, 2013

What are you supposed to put in there?

I tried

v.customize 'pre-boot', ['modifyhd', :id, '--resize', '15360']

Giving

[app] Running 'pre-boot' VM customizations...
A customization command failed:

["modifyhd", "f7ee3447-5748-4562-9ca2-87639ec39515", "--resize", "15360"]

The following error was experienced:

VBoxManage: error: The given path 'f7ee3447-5748-4562-9ca2-87639ec39515' is not fully qualified
VBoxManage: error: Details: code VBOX_E_FILE_ERROR (0x80bb0004), component Medium, interface IMedium, callee nsISupports
VBoxManage: error: Context: "OpenMedium(Bstr(pszFilenameOrUuid).raw(), enmDevType, enmAccessMode, fForceNewUuidOnOpen, pMedium.asOutParam())" at line 178 of file VBoxManageDisk.cpp


Please fix this customization and try again.
@odino

This comment has been minimized.

Copy link

odino commented Jan 9, 2014

hey @haf did you find a way out? :)

@haf

This comment has been minimized.

Copy link

haf commented Jan 9, 2014

Hi @odino, no shortly afterwards VMWare blue-screened my Mac and corrupted the VM anyway so I had to redo all the work. This time I chose a larger size initially.

@tmatilai

This comment has been minimized.

Copy link
Collaborator

tmatilai commented Jan 9, 2014

That :id will expand to the machine ID, while modifyhd expects ID or file system path of the disk image.
But I don't know how to get access to those in Vagrantfile context.

@snowch

This comment has been minimized.

Copy link

snowch commented Jan 21, 2014

You can use vboxmanage showvminfo UUID to find the disk UUID

vboxmanage showvminfo e47cb0df-0799-4de7-8073-58239da86216
...
SATA Controller (0, 0): /home/me/VirtualBox VMs/mybox_1389356990435_59717/box-disk1.vmdk (UUID: 6eb5858a-a486-4b41-a1fb-1cb9f0eb7e5f)
...

I was hoping that the showvminfo command could be executed by vb.customize and the response parsed.

vb.customize [ "showvminfo", :id ]

Howver, vb.customize does not return a response.

@kikitux

This comment has been minimized.

Copy link
Contributor

kikitux commented Jan 22, 2014

for virtualbox, if the disk is not VDI, then is not easy to expand the disk.

you should turn the guest off, clone the disk from vmdk to vdi, dettach
vmdk, attach vdi, expand vdi, then turn the machine on, and expand the
filesystem on the disk.

What OS you are using ?? I think will be way easier create a new VM with
the desired disk with packer and then use it in vagrant.

Alvaro.

On Wed, Jan 22, 2014 at 11:50 AM, snowch notifications@github.com wrote:

You can use vboxmanage showvminfo UUID to find the disk UUID

vboxmanage showvminfo e47cb0df-0799-4de7-8073-58239da86216
...
SATA Controller (0, 0): /home/me/VirtualBox VMs/mybox_1389356990435_59717/box-disk1.vmdk (UUID: 6eb5858a-a486-4b41-a1fb-1cb9f0eb7e5f)
...

I was hoping that the showvminfo command could be executed by
vb.customize and the response parsed.

vb.customize [ "showvminfo", :id ]

Howver, vb.customize does not return a response.


Reply to this email directly or view it on GitHubhttps://github.com//issues/2339#issuecomment-32973372
.

@snowch

This comment has been minimized.

Copy link

snowch commented Jan 22, 2014

@kikitux it's not easy, but it should be possible to automate shouldn't it?

The guest OS is CentOS 6.4. The host OS could be any OS supported by Vagrant.

The driver behind this requirement to increase the disk size of an existing box is that many security teams will only the use a 'trusted' third party boxes. It is better for me to expand a trusted image than create and make available my own non trusted image.

I've just taken a look at packer - thanks for the tip! I'll take a look and see if that solves the problem.

@kikitux

This comment has been minimized.

Copy link
Contributor

kikitux commented Jan 22, 2014

When you say my box, then i understand you are creating in some way the box.

I will say the easiest will be when you create the box, create a BIG disk,
say 50GB, as the disk will be an sparse disk, it will use only the size
that is actually used.

I do custom boxes for my test and I use 4 disks of 32GB, so should use
128GB, but the vagrant box is just 700MB for the OS, or couple of gigs when
I install the software.

If this approach work for you, then I will suggest import the box to
vagrant, do the expansion manually (or by some script outside vagrant) and
then export the box and you will get the new machine.

If, the boxes are not yours, and are 3rd party (created, managed by
others), then I will suggest teach the users how to do it, (teach them to
catch fish), or you will be the man in the middle all the time.

I don't think this is an Vagrant issue, so I think the best is to talk in
the mailing list..

I am more than happy to assist if you need any help with the
commands/order/etc, in the mailing list.

(vagrantup.com -> about -> support -> check around for the mailing list)

Alvaro.

On Wed, Jan 22, 2014 at 7:34 PM, snowch notifications@github.com wrote:

@kikitux https://github.com/kikitux it's not easy, but it is possible
to automate isn't it?

The guest OS is CentOS 6.4. The host OS could be any OS supported by
Vagrant.

The driver behind this requirement is that many security teams will only
the use a 'trusted' third party boxes. It is better for me to expand a
trusted image than create and make available my own non trusted image.

I've just taken a look at packer - thanks for the tip! Packer may overcome
the issue of automating the image, but it looks as though it adds another
installation step for users of my box. Asking my users to install vagrant,
virtualbox, cygwin/putty (windows users), 3 vagrant plugins, and packer is
a big barrier to people using my box.


Reply to this email directly or view it on GitHubhttps://github.com//issues/2339#issuecomment-32996903
.

@snowch

This comment has been minimized.

Copy link

snowch commented Jan 22, 2014

@kikitux I'm not creating a box. I don't want to do this because my security team don't trust boxes that are available in places like dropbox.

I'm using a box from a trusted place (the specific box is http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210.box). Someone raised an issue against them 4 months ago to increase the box size:

puppetlabs/puppet-vagrant-boxes#33

The pull request seems to still be open :(

Thanks for your feedback and kind offer of assistance.

@kikitux

This comment has been minimized.

Copy link
Contributor

kikitux commented Jan 22, 2014

Hello, I will give you my best answer based on my expertise on linux I have.

The box came out of the box like this:

[root@localhost ~]# df -Ph
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root 8.4G 1.1G 6.9G 14% /
tmpfs 230M 0 230M 0% /dev/shm
/dev/sda1 485M 33M 427M 8% /boot
/vagrant 725G 475G 251G 66% /vagrant
[root@localhost ~]#

Plenty of space for the OS.

So, my suggestion for your problem will be:

  • Start with this machine as came from the vendor
  • Download it, import it into vagrant.
  • Start it up, add a 2nd hard disk, expand the VG, expand the LV
  • Update to the latest (yum -y update)
  • Package and make available internally

as the box upstream is not suitable, and the vendor is performing slow on
this, you should do what I suggest.

If there is any linux admin there or any linux standard, then the 2nd disk
may be used with a new volume group and create a new mount point for your
applications, say /opt /srv /usr/local , etc.

Hope this helps.

Alvaro.

On Wed, Jan 22, 2014 at 8:15 PM, snowch notifications@github.com wrote:

@kikitux https://github.com/kikitux I'm not creating a box. I don't
want to do this because my security team don't trust boxes that are
available in places like dropbox.

I'm using a box from a trusted place (the specific box is
http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210.box).
Someone raised an issue against them 4 months ago to increase the box size:

puppetlabs/puppet-vagrant-boxes#33puppetlabs/puppet-vagrant-boxes#33

The pull request seems to still be open :(

Thanks for your feedback and kind offer of assistance.


Reply to this email directly or view it on GitHubhttps://github.com//issues/2339#issuecomment-32998454
.

@snowch

This comment has been minimized.

Copy link

snowch commented Jan 22, 2014

@kikitux your recommendation is pretty much what I am doing at the moment until automated disk resizing is figured out:

In may vagrantfile:

file_to_disk = File.realpath( "." ).to_s + "/disk.vdi"

        if ARGV[0] == "up" && ! File.exist?(file_to_disk) 
           puts "Creating 30GB disk #{file_to_disk}."
           vb.customize [
                'createhd', 
                '--filename', file_to_disk, 
                '--format', 'VDI', 
                '--size', 300000 * 1024 # 30 GB
                ] 
           vb.customize [
                'storageattach', :id, 
                '--storagectl', 'SATA Controller', 
                '--port', 1, '--device', 0, 
                '--type', 'hdd', '--medium', 
                file_to_disk
                ]
        end

and then:

carbon.vm.provision "shell", path: "scripts/add_new_disk.sh"

https://github.com/snowch/carbon-products-development-environment/blob/master/scripts/add_new_disk.sh

@kikitux

This comment has been minimized.

Copy link
Contributor

kikitux commented Jan 22, 2014

couple of suggestion.

If you change the script to:

pvcreate /dev/sdb
vgextend VolGroup /dev/sdb

as the new disk is VDI, you avoid creating partitions on it, and then
will be able to add more space in the future if needed.

you could expand the VDI, rescan the base disk, pvscan, pvresize, and
the vg will grow on the fly.

other tip, if you do this:

lvextend -l +$(pvdisplay /dev/sdb -c | cut -d: -f11) /dev/VolGroup/lv_root

you will extend for the exact number of FREE extents that are
available in the new disk.

note is +number to add the number..

Other than that, i think your scripts are fantastic good.

Alvaro.

On Thu, Jan 23, 2014 at 9:36 AM, snowch notifications@github.com wrote:

@kikitux https://github.com/kikitux your recommendation is pretty much
what I have done so far:

In may vagrantfile:

file_to_disk = File.realpath( "." ).to_s + "/disk.vdi"

    if ARGV[0] == "up" && ! File.exist?(file_to_disk)
       puts "Creating 30GB disk #{file_to_disk}."
       vb.customize [
            'createhd',
            '--filename', file_to_disk,
            '--format', 'VDI',
            '--size', 300000 * 1024 # 30 GB
            ]
       vb.customize [
            'storageattach', :id,
            '--storagectl', 'SATA Controller',
            '--port', 1, '--device', 0,
            '--type', 'hdd', '--medium',
            file_to_disk
            ]
    end

and then:

carbon.vm.provision "shell", path: "scripts/add_new_disk.sh"

https://github.com/snowch/carbon-products-development-environment/blob/master/scripts/add_new_disk.sh


Reply to this email directly or view it on GitHubhttps://github.com//issues/2339#issuecomment-33064917
.

@snowch

This comment has been minimized.

Copy link

snowch commented Jan 23, 2014

Very nice. Thank you!

@bechampion

This comment has been minimized.

Copy link

bechampion commented Mar 15, 2014

no news about increasing the size on pre-boot yet ? or some what to get the full path to the vdi from the :id (UUID) ?

@kikitux

This comment has been minimized.

Copy link
Contributor

kikitux commented Mar 17, 2014

that won't work, as when the machine is created from the base box, the disk are not VDI, hence can't be extended.

@xacaxulu

This comment has been minimized.

Copy link

xacaxulu commented May 6, 2015

Does anyone have the remotest idea about how to put this into a .kitchen.yml format?

@renan

This comment has been minimized.

Copy link

renan commented Jun 16, 2015

My solution was to resize the base box itself, like follows:

vagrant box add ubuntu/trusty64 --box-version 20150609.0.9
cd ~/.vagrant.d/boxes/ubuntu-VAGRANTSLASH-trusty64/20150609.0.9/virtualbox/
VBoxManage clonehd box-disk1.vmdk tmp-disk.vdi --format vdi
VBoxManage modifyhd tmp-disk.vdi --resize 61440
VBoxManage clonehd tmp-disk.vdi resized-disk.vmdk --format vmdk
rm tmp-disk.vdi box-disk1.vmdk
mv resized-disk.vmdk box-disk1.vmdk
# df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        60G  1.8G   55G   4% /

With 60GB instead of the base 40GB.

@hbrls

This comment has been minimized.

Copy link

hbrls commented Jun 19, 2015

I followed this blog post, and it works well.

http://blog.lenss.nl/2012/09/resize-a-vagrant-vmdk-drive/

@BenMorganIO

This comment has been minimized.

Copy link

BenMorganIO commented Jun 2, 2016

@mitchellh Do you know what's supposed to go into those ellipses?

@dvfabbri

This comment has been minimized.

Copy link

dvfabbri commented Jun 13, 2016

@BenMorganIO did you determine what goes in those ellipses to set disk size?

@BenMorganIO

This comment has been minimized.

Copy link

BenMorganIO commented Jun 13, 2016

Nope...

@dvfabbri

This comment has been minimized.

Copy link

dvfabbri commented Jun 14, 2016

@mitchellh Can you provide an example of the commands required to create a VM with 500GB disk in the Vagrant file? I have been looking around to set this up, but I currently am just stuck using Gparted to resize post creation.

config.vm.customize "virtualbox" do |vb|
  vb.customize "pre-boot", ["..."]
end
@dimasdanz

This comment has been minimized.

Copy link

dimasdanz commented Jun 28, 2016

The command should be something like this

config.vm.customize "virtualbox" do |vb|
  vb.customize "pre-boot", ["modifyhd", :id, "--resize", "32678"] #32678 = 32GiB
end

But, this would not work because:

  1. UUID passed by Vagrant to VBoxManage is not hdd's UUID listed by VBoxManage list hdds so modifyhd is not possible. It displays error like this

    Stderr: VBoxManage.exe: error: The given path <UUID-here> is not fully qualified 
    
  2. Even if we can get hdd UUID to execute modifyhd, storage format of boxes downloaded by Vagrant is VMDK. modifyhd command cannot be performed on VMDK format

So, as of now, I think the only way to increase disk space is @renan's way
You can also modify it post creation but you have to remove attached hdd and reattach the new one.

@jhgorse

This comment has been minimized.

Copy link

jhgorse commented Jul 6, 2016

so this what we need?

config.vm.provider "virtualbox" do |v|
  vmdk_path = get_vagrant_box_path()
  tmp1 = vmdk_path."tmp1"
  tmp2 = vmdk_path."tmp2"

  v.customize "pre-boot", ["clonehd",  vmdk_path,  tmp1, "--format vdi"]
  v.customize "pre-boot", ["modifyhd", tmp1, "--resize", "32678"]
  v.customize "pre-boot", ["clonehd",  tmp1,       tmp2, "--format vmdk"]

  rm vmdk_path tmp1
  mv tmp2 vmdk_path
end

@Svieg Svieg referenced this issue Aug 6, 2016

Closed

Specify disk size #14

@mleopold

This comment has been minimized.

Copy link

mleopold commented Aug 16, 2016

Hi,
@mitchellh The issue was closed in 2013, is there any chance of reopening it? it seems this issue is still around and the suggestions are inconvenient at best. A option for the disk-size at the first creation or more guided resize later sounds like it would be appreciated.

Br,
Martin

@Shuliyey

This comment has been minimized.

Copy link

Shuliyey commented Aug 23, 2016

@jhgorse thanks for the script.

Just a question I got the below error around the below two lines

  tmp1 = vmdk_path."tmp1"
  tmp2 = vmdk_path."tmp2"
There is a syntax error in the following Vagrantfile. The syntax error
message is reproduced below for convenience:


/mnt/sdb/VirtualBox/dye/Vagrantfile:55: syntax error, unexpected tSTRING_BEG, expecting '('
    tmp1 = vmdk_path."tmp1"
                      ^
/mnt/sdb/VirtualBox/dye/Vagrantfile:56: syntax error, unexpected tSTRING_BEG, expecting '('
    tmp2 = vmdk_path."tmp2"
@skortchmark9

This comment has been minimized.

Copy link

skortchmark9 commented Oct 11, 2016

@renan's solution seems to be working for me - However, I also needed to edit the box.ovf file, where it says ovf:capacity

@snowiow

This comment has been minimized.

Copy link

snowiow commented Oct 26, 2016

I had the same problem and thanks to @renan 's solution I got started. But it wasn't the complete solution for me, because I had an UUID conflict. I have written up my complete solution path here. I hope it is of any help for anybody.

@BenMorganIO

This comment has been minimized.

Copy link

BenMorganIO commented Oct 26, 2016

@snowiow Thankyou ❤️

@ogero

This comment has been minimized.

Copy link

ogero commented Jan 18, 2017

I've just made a Python script that does the VMDK resize and deal with UUID all the way..
But regrettably, it can't be automatically used inside Vagrantfile.. the steps are something like this:

>vagrant up --no-provision
>vagrant halt
>wget https://gist.githubusercontent.com/ogero/14692edf0702bfefe0455d6e7a9dc6de/raw/849c39404b649abaf6985901505170cc2f366c46/expand_disk.py
>chmod +x expand_disk.py
>python expand_disk.py
>vagrant up --no-provision
>vagrant ssh --command "sudo resize2fs /dev/sda1"
>vagrant provision
@Doume3

This comment has been minimized.

Copy link

Doume3 commented May 24, 2017

Hi, I saw this contribution which help me a lot 👍 https://github.com/sprotheroe/vagrant-disksize

@steveswinsburg

This comment has been minimized.

Copy link

steveswinsburg commented Aug 10, 2017

With the vagrant-disksize plugin I get a resized disk but the partition underneath is still the original so has to be resized within the guest. This makes it hard to automate.

@afturner

This comment has been minimized.

Copy link

afturner commented Feb 6, 2018

If you'd like an automated solution, I did this:

config.disksize.size = '10GB'
config.vm.provision "shell", inline: <<-SHELL
    sudo resize2fs /dev/sda1
SHELL

And install vagrant-disksize:

vagrant plugin install vagrant-disksize

YMMV. sda1 was the right one for me, but you may need to ssh and double check.

@CristianCantoro

This comment has been minimized.

Copy link

CristianCantoro commented Feb 27, 2018

@Doume3 said:

Hi, I saw this contribution which help me a lot +1 https://github.com/sprotheroe/vagrant-disksize

I confirm that using the vagrant-disksize plugin with a Ubuntu base box solved the issue for me. It also seems that I had nothing else to do (no resize2fs, no lvextend, etc.) and it worked "automagically".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment