Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it possible to store images and backups on a storage pool #5515

Open
stgraber opened this issue Feb 20, 2019 · 2 comments

Comments

2 participants
@stgraber
Copy link
Member

commented Feb 20, 2019

Right now, the original image artifacts and the container backups are stored on the main system storage, not on any of LXD's storage pools.

This is surprising to users who may then run themselves out of disk space despite having a lot of storage directly assigned to LXD.

I believe we should introduce two new configuration options (daemon level):

  • storage.images_volume
  • storage.backups_volume

Syntax being <pool>/<volume> where <volume> is an existing, unused custom volume.

When not set, we store things directly in the directory as we used to.
If set, we create a lxd_images or lxd_backups directory on the selected storage pool and turn the images or backups directory into a symlink to the relevant location.

Those settings may be changed and data will be moved around as needed.

Those storage volumes must be adequately protected so that they may not get attached to a container and may not be deleted or renamed while in use.

We also need to disallow the use of CEPH for this as CEPH+cluster would fail miserably.
We may re-visit this with CEPHfs when we add it but for now, we should just ban CEPH.

@stgraber stgraber added this to the soon milestone Feb 20, 2019

@stgraber stgraber added the Feature label Feb 20, 2019

@gpatel-fr

This comment has been minimized.

Copy link

commented Feb 24, 2019

Right now, the original image artifacts and the container backups are stored on the main system storage, not
on any of LXD's storage pools.
This is surprising to users who may then run themselves out of disk space despite having a lot of storage directly assigned to LXD.

Indeed this has happened to me. It's nice to see a project manager so responsive that he see the need to solve my problem before I have it :-)

But is it a real problem for images ? Images are usually 150-200 mb and only one at a time is stored temporarily on the file system
For backup, yes, the actual container data can be quite larger (see below). But is is necessary to actually manage new special volume classes ? Add new objects is nice when you have a still growing project, but later you can consider them as making it to heavy, in short overengineering.
Why not just add the possibility to enter a specific path for storing backups ? Simpler, lighter, leaving the actual management to system administrators (that's what I did once I understood the reason for the crash; BTW a man page for lxc-export could be useful, explaining what happens and what to do with the remaining data hogging the available space, this data that seems to be automatically cleaned when restarting a successful export, something not totally obvious and I have no idea how about doing it safely otherwise).

There is the problem of snap adding 'protection' to file accesses for people using lxd with snapcraft, though. Maybe that's the reason you want to add all this stuff.

From what I see, export does on the file system a full copy of all container data AND a tarball AND the gzipped tarball, needing so about 2,5 times the actual container size. Probably for the best, but lxc-export doc (if it actually existed) should say clearly that you need at least that on the temporary workspace. Many people would assume that 50% over the biggest container should be more than enough. Possibly the container copy could be zapped before creating the gzipped tar from the tarball. I don't quite see why it's needed at this point. I'd dare to say that efforts to make this more efficient could be actually useful.

@stgraber

This comment has been minimized.

Copy link
Member Author

commented Feb 25, 2019

@monstermunchkin can you check why we'd be keeping the temporary backup directory after we've tared it (and before we compress it)? Seems like an obvious waste of space.

monstermunchkin added a commit to monstermunchkin/lxd that referenced this issue Feb 25, 2019

lxd: Remove backup directory after creating tarball
Remove the backup backup directory as soon as the tarball has been
created. That way one will hopefully not run out of disk space when
doing backups.

See lxc#5515

Signed-off-by: Thomas Hipp <thomas.hipp@canonical.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.