New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust qcow2 image files to where 10GB really equals 10GB. #595

Closed
claflico opened this Issue Feb 20, 2015 · 3 comments

Comments

Projects
None yet
2 participants
@claflico

claflico commented Feb 20, 2015

Don't know if it is supported or not but I download the qcow2 images for memcached & redis and import them into Openstack. They show to be <100MB on the dashboard however attempting to launch an instance on a 10GB flavor fails to due to the hard-drive being too small.

View the details of the qcow file to see:
qemu-img info osv-redis-memonly-v0.17.qemu.qcow2
image: osv-redis-memonly-v0.17.qemu.qcow2
file format: qcow2
virtual size: 10G (10842275840 bytes)
disk size: 30M
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false

10842275840 bytes actually equals 10.8423GB which if I adjust my flavors to be 11GB I am able to launch the instance but then I don't have consistency with the rest of my flavors.

I also don't see why if the qcow2 files is only 30M that why the virtual size needs to be 10GB.

Thanks.

@nyh

This comment has been minimized.

Show comment
Hide comment
@nyh

nyh Feb 24, 2015

Contributor

On Sat, Feb 21, 2015 at 1:48 AM, Cory Claflin notifications@github.com
wrote:

Don't know if it is supported or not but I download the qcow2 images for
memcached & redis and import them into Openstack. They show to be <100MB on
the dashboard however attempting to launch an instance on a 10GB flavor
fails to due to the hard-drive being too small.

View the details of the qcow file to see:
qemu-img info osv-redis-memonly-v0.17.qemu.qcow2
image: osv-redis-memonly-v0.17.qemu.qcow2
file format: qcow2
virtual size: 10G (10842275840 bytes)
disk size: 30M
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false

10842275840 bytes actually equals 10.8423GB which if I adjust my flavors
to be 11GB I am able to launch the instance but then I don't have
consistency with the rest of my flavors.

I also don't see why if the qcow2 files is only 30M that why the virtual
size needs to be 10GB.

This 10GB or 11GB is a completely arbitrary size that can be set in build.mk's
fs_size_mb (you can also set it in the "make" command line).
If you need your virtual disk to contain empty space for data - beyond the
30 MB taken by the application, then obviously you need the virtual disk to
be able to grow to more than 30 MB - and 10 GB was arbitrarily chosen.

But why is our 10GB bigger than 10GB?

First of all, our code in build.mk work in powers of 2, not powers of 10,
so our default size isn't 10,000,000,000 bytes, it is 10_1024_1024*1024
bytes.

This almost explains the size you're seeing, except for about 100MB. That
could further be explained, I think, by the "+" in the following command in
build.mk:

    $(call quiet, qemu-img resize $@ +$(fs_size_mb)M > /dev/null 2>&1)

I don't know why this "+" appears here (and I already noted that in a
comment in my proposed rewrite of the makefile).

I believe that to get exactly 10,000,000,000 bytes, you can modify the
above line to drop the "M" (so exact bytes will be specified) and the "+".

Of course, without any changes you can also use fs_size_mb=9000, and it
will be smaller than 10 GB :-)

Nadav Har'El
nyh@cloudius-systems.com

Contributor

nyh commented Feb 24, 2015

On Sat, Feb 21, 2015 at 1:48 AM, Cory Claflin notifications@github.com
wrote:

Don't know if it is supported or not but I download the qcow2 images for
memcached & redis and import them into Openstack. They show to be <100MB on
the dashboard however attempting to launch an instance on a 10GB flavor
fails to due to the hard-drive being too small.

View the details of the qcow file to see:
qemu-img info osv-redis-memonly-v0.17.qemu.qcow2
image: osv-redis-memonly-v0.17.qemu.qcow2
file format: qcow2
virtual size: 10G (10842275840 bytes)
disk size: 30M
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false

10842275840 bytes actually equals 10.8423GB which if I adjust my flavors
to be 11GB I am able to launch the instance but then I don't have
consistency with the rest of my flavors.

I also don't see why if the qcow2 files is only 30M that why the virtual
size needs to be 10GB.

This 10GB or 11GB is a completely arbitrary size that can be set in build.mk's
fs_size_mb (you can also set it in the "make" command line).
If you need your virtual disk to contain empty space for data - beyond the
30 MB taken by the application, then obviously you need the virtual disk to
be able to grow to more than 30 MB - and 10 GB was arbitrarily chosen.

But why is our 10GB bigger than 10GB?

First of all, our code in build.mk work in powers of 2, not powers of 10,
so our default size isn't 10,000,000,000 bytes, it is 10_1024_1024*1024
bytes.

This almost explains the size you're seeing, except for about 100MB. That
could further be explained, I think, by the "+" in the following command in
build.mk:

    $(call quiet, qemu-img resize $@ +$(fs_size_mb)M > /dev/null 2>&1)

I don't know why this "+" appears here (and I already noted that in a
comment in my proposed rewrite of the makefile).

I believe that to get exactly 10,000,000,000 bytes, you can modify the
above line to drop the "M" (so exact bytes will be specified) and the "+".

Of course, without any changes you can also use fs_size_mb=9000, and it
will be smaller than 10 GB :-)

Nadav Har'El
nyh@cloudius-systems.com

@nyh

This comment has been minimized.

Show comment
Hide comment
@nyh

nyh Feb 24, 2015

Contributor

Fixed on my makefile rewrite branch (see issue #571), removing the superfluous "+" as described above.

On that branch, we now get:

$ scripts/build image=rogue fs_size_mb=1024
$ qemu-img info build/release.x64/usr.img
...
virtual size: 1.0G (1073741824 bytes)

(note how 1024_1024_1024=1073741824, exactly).

Also added an option to set the size in bytes exactly. However, this number is rounded down to the nearest multiple of 512:

$ scripts/build image=rogue fs_size=100000000
$ qemu-img info build/release.x64/usr.img
...
virtual size: 95M (99999744 bytes)

(Note 99999744 is 100000000 rounded down to the nearest multiple of 512)

Contributor

nyh commented Feb 24, 2015

Fixed on my makefile rewrite branch (see issue #571), removing the superfluous "+" as described above.

On that branch, we now get:

$ scripts/build image=rogue fs_size_mb=1024
$ qemu-img info build/release.x64/usr.img
...
virtual size: 1.0G (1073741824 bytes)

(note how 1024_1024_1024=1073741824, exactly).

Also added an option to set the size in bytes exactly. However, this number is rounded down to the nearest multiple of 512:

$ scripts/build image=rogue fs_size=100000000
$ qemu-img info build/release.x64/usr.img
...
virtual size: 95M (99999744 bytes)

(Note 99999744 is 100000000 rounded down to the nearest multiple of 512)

@claflico

This comment has been minimized.

Show comment
Hide comment
@claflico

claflico Feb 24, 2015

Thanks.....I think...lol If what you did makes the file size smaller then I'll give it another try when the next qcow2 file gets released.

Not meaning to sound lazy and realize that this probably isn't the intended use of the project but I'm not wanting to have to build/compile anything. I would like to be able to just download the qcow2 files for memcache & redis, import them into OpenStack, and have ready to go instances that work with my existing 10GB flavors. If not doable, then thanks for replying and I'll keep building them by hand.

claflico commented Feb 24, 2015

Thanks.....I think...lol If what you did makes the file size smaller then I'll give it another try when the next qcow2 file gets released.

Not meaning to sound lazy and realize that this probably isn't the intended use of the project but I'm not wanting to have to build/compile anything. I would like to be able to just download the qcow2 files for memcache & redis, import them into OpenStack, and have ready to go instances that work with my existing 10GB flavors. If not doable, then thanks for replying and I'll keep building them by hand.

@avikivity avikivity closed this in f37cef7 Mar 29, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment