Skip to content

Commit

Permalink
Merge pull request #313 from ajkavanagh/bug/274/storage-pool-methods
Browse files Browse the repository at this point in the history
Implementation of storage pools for pylxd
  • Loading branch information
javacruft committed Jul 30, 2018
2 parents 8ba096c + 34e0ed3 commit 0d5c227
Show file tree
Hide file tree
Showing 22 changed files with 1,582 additions and 179 deletions.
2 changes: 1 addition & 1 deletion doc/source/networks.rst
Expand Up @@ -5,7 +5,7 @@ Networks

:class:`Network` objects show the current networks available to LXD. Creation
and / or modification of networks is possible only if 'network' LXD API
extension is present (see :func:`~Network.network_extension_available`)
extension is present.


Manager methods
Expand Down
131 changes: 117 additions & 14 deletions doc/source/storage-pools.rst
Expand Up @@ -6,12 +6,37 @@ keys are top-level. Driver specific keys are namespaced by driver name. Volume
keys apply to any volume created in the pool unless the value is overridden on
a per-volume basis.

`Storage Pool` objects represent the json object that is returned from
`GET /1.0/storage-pools/<name>` and then the associated methods that are then
available at the same endpoint.
Storage Pool objects
--------------------

Manager methods
---------------
:py:class:`~pylxd.models.storage_pool.StoragePool` objects represent the json
object that is returned from `GET /1.0/storage-pools/<name>` and then the
associated methods that are then available at the same endpoint.

There are also :py:class:`~pylxd.models.storage_pool.StorageResource` and
:py:class:`~pylxd.models.storage_pool.StorageVolume` objects that represent the
storage resources endpoint for a pool at `GET
/1.0/storage-pools/<pool>/resources` and a storage volume on a pool at `GET
/1.0/storage-pools/<pool>/volumes/<type>/<name>`. Note that these should be
accessed from the storage pool object. For example:

.. code:: python
client = pylxd.Client()
storage_pool = client.storage_pools.get('poolname')
storage_volume = storage_pool.volumes.get('custom', 'volumename')
.. note:: For more details of the LXD documentation concerning storage pools
please see `LXD Storage Pools REST API`_ Documentation and `LXD Storage Pools`_
Documentation. This provides information on the parameters and attributes in
the following methods.

.. note:: Please see the pylxd API documentation for more information on
storage pool methods and parameters. The following is a summary.

Storage Pool Manager methods
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Storage-pools can be queried through the following client manager methods:

Expand All @@ -24,22 +49,100 @@ Storage-pools can be queried through the following client manager methods:
passed to this method.


Storage-pool attributes
-----------------------
Storage-pool Object attributes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For more information about the specifics of these attributes, please see
the `LXD documentation`_.
the `LXD Storage Pools REST API`_ documentation.

- `name` - the name of the storage pool
- `driver` - the driver (or type of storage pool). e.g. 'zfs' or 'btrfs', etc.
- `used_by` - which containers (by API endpoint `/1.0/containers/<name>`) are
using this storage-pool.
- `config` - a string (json encoded) with some information about the
storage-pool. e.g. size, source (path), volume.size, etc.
- `config` - a dictionary with some information about the storage-pool. e.g.
size, source (path), volume.size, etc.
- `managed` -- Boolean that indicates whether LXD manages the pool or not.

.. _LXD documentation: https://github.com/lxc/lxd/blob/master/doc/rest-api.md#10storage-pools

Storage-pool methods
--------------------
Storage-pool Object methods
^^^^^^^^^^^^^^^^^^^^^^^^^^^

The following methods are available on a Storage Pool object:

- `save` - save a modified storage pool. This saves the `config` attribute
in it's entirety.
- `delete` - delete the storage pool.
- `put` - Change the LXD storage object with a passed parameter. The object
is then synced back to the storage pool object.
- `patch` - A more fine grained patch of the object. Note that the object is
then synced back after a successful patch.

.. note:: `raw_put` and `raw_patch` are availble (but not documented) to allow
putting and patching without syncing the object back.


Storage Resources
-----------------

Storage Resources are accessed from the storage pool object:

.. code:: python
resources = storage_pool.resources.get()
Resources are read-only and there are no further methods available on them.

Storage Volumes
---------------

Storage Volumes are stored in storage pools. On the `pylxd` API they are
accessed from a storage pool object:

... code:: Python

storage_pool = client.storage_pools.get('pool1')
volumes = storage_pool.volumes.all()
named_volume = storage_pool.volumes.get('custom', 'vol1')

Methods available on `<storage_pool_object>.volumes`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The following methods are accessed from the `volumes` attribute on the storage
pool object.

- `all` - get all the volumes on the pool.
- `get` - a get a single, type + name volume on the pool.
- `create` - create a volume on the storage pool.

.. note:: Note that storage volumes have a tuple of `type` and `name` to uniquely
identify them. At present LXD recognises three types (but this may change),
and these are `container`, `image` and `custom`. LXD uses `container` and
`image` for containers and images respectively. Thus, for user applications,
`custom` seems like the type of choice. Please see the `LXD Storage Pools`_
documentation for further details.

Methods available on the storage volume object
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Once in possession of a storage volume object from the `pylxd` API, the
following methods are available:

- `rename` - Rename a volume. This can also be used to migrate a volume from
one pool to the other, as well as migrating to a different LXD instance.
- `put` - Put an object to the LXD server using the storage volume details
and then re-sync the object.
- `patch` - Patch the object on the LXD server, and then re-sync the object
back.
- `save` - after modifying the object in place, use a PUT to push those
changes to the LXD server.
- `delete` - delete a storage volume object. Note that the object is,
therefore, stale after this action.

.. note:: `raw_put` and `raw_patch` are availble (but not documented) to allow
putting and patching without syncing the object back.

.. links
The are no storage pool methods defined yet.
.. _LXD Storage Pools: https://lxd.readthedocs.io/en/latest/storage/
.. _LXD REST API: https://github.com/lxc/lxd/blob/master/doc/rest-api.md
.. _LXD Storage Pools REST API: https://github.com/lxc/lxd/blob/master/doc/rest-api.md#10storage-pools
5 changes: 2 additions & 3 deletions integration/test_containers.py
Expand Up @@ -136,9 +136,8 @@ def test_put_get_file(self):
filepath = '/tmp/an_file'
data = b'abcdef'

retval = self.container.files.put(filepath, data)

self.assertTrue(retval)
# raises an exception if this fails.
self.container.files.put(filepath, data)

contents = self.container.files.get(filepath)

Expand Down
3 changes: 1 addition & 2 deletions integration/test_networks.py
Expand Up @@ -14,15 +14,14 @@

from integration.testing import IntegrationTestCase
from pylxd import exceptions
from pylxd.models import Network


class NetworkTestCase(IntegrationTestCase):

def setUp(self):
super(NetworkTestCase, self).setUp()

if not Network.network_extension_available(self.client):
if not self.client.has_api_extension('network'):
self.skipTest('Required LXD API extension not available!')


Expand Down
175 changes: 175 additions & 0 deletions integration/test_storage.py
@@ -0,0 +1,175 @@
# Copyright (c) 2016 Canonical Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

import random
import string
import unittest

import six

from integration.testing import IntegrationTestCase
import pylxd.exceptions as exceptions


class StorageTestCase(IntegrationTestCase):

def setUp(self):
super(StorageTestCase, self).setUp()

if not self.client.has_api_extension('storage'):
self.skipTest('Required LXD API extension not available!')

def create_storage_pool(self):
# create a storage pool in the form of 'xxx1' as a dir.
name = ''.join(random.sample(string.ascii_lowercase, 3)) + '1'
self.lxd.storage_pools.post(json={
"config": {},
"driver": "dir",
"name": name,
})
return name

def delete_storage_pool(self, name):
# delete the named storage pool
try:
self.lxd.storage_pools[name].delete()
except exceptions.NotFound:
pass


class TestStoragePools(StorageTestCase):
"""Tests for :py:class:`pylxd.models.storage_pools.StoragePools"""
# note create and delete are tested in every method

def test_get(self):
name = self.create_storage_pool()
self.addCleanup(self.delete_storage_pool, name)

storage_pool = self.client.storage_pools.get(name)
self.assertEqual(name, storage_pool.name)

def test_all(self):
name = self.create_storage_pool()
self.addCleanup(self.delete_storage_pool, name)

storage_pools = self.client.storage_pools.all()
self.assertIn(name, [p.name for p in storage_pools])

def test_save(self):
name = self.create_storage_pool()
self.addCleanup(self.delete_storage_pool, name)

storage_pool = self.client.storage_pools.get(name)
storage_pool.description = "My storage pool"
storage_pool.save()

p = self.client.storage_pools.get(name)
self.assertEqual(p.description, "My storage pool")

def test_put(self):
name = self.create_storage_pool()
self.addCleanup(self.delete_storage_pool, name)

storage_pool = self.client.storage_pools.get(name)
new_desc = "new description"
put_object = {
"description": new_desc,
"config": storage_pool.config,
}
storage_pool.put(put_object)
self.assertEqual(storage_pool.description, new_desc)
p = self.client.storage_pools.get(name)
self.assertEqual(p.description, new_desc)

# can't test this as patch doesn't seem to work for storage pools.
# Need to wait until bug: https://github.com/lxc/lxd/issues/4709
# fix is released.
@unittest.skip("Can't test until fix to lxd bug #4709 is released")
def test_patch(self):
name = self.create_storage_pool()
self.addCleanup(self.delete_storage_pool, name)

desc = "My storage pool"
storage_pool = self.client.storage_pools.get(name)
patch = {"description": "hello world"}
storage_pool.patch(patch)
self.assertEqual(storage_pool.description, desc)

p = self.client.storage_pools.get(name)
self.assertEqual(p.description, "hello world")


class TestStorageResources(StorageTestCase):
"""Tests for :py:class:`pylxd.models.storage_pools.StorageResources"""

def test_get(self):
name = self.create_storage_pool()
self.addCleanup(self.delete_storage_pool, name)

storage_pool = self.client.storage_pools.get(name)
# just assert that it can be fetched and that a key exists
resources = storage_pool.resources.get()
self.assertIsInstance(resources.space, dict)
self.assertIsInstance(resources.inodes, dict)


class TestStorageVolume(StorageTestCase):
"""Tests for :py:class:`pylxd.models.storage_pools.StorageVolume"""
# note create and delete are tested in every method

def create_storage_volume(self, pool):
# note 'pool' needs to be storage_pool object or a string
if isinstance(pool, six.string_types):
pool = self.client.storage_pools.get(pool)
vol_input = {
"config": {},
"type": "custom",
# "pool": name,
"name": "vol1",
}
volume = pool.volumes.create(vol_input)
return volume

def delete_storage_volume(self, pool, volume):
# pool is either string or storage_pool object
# volume is either a string of storage_pool object
if isinstance(volume, six.string_types):
if isinstance(pool, six.string_types):
pool = self.client.storage_pools.get(pool)
volume = pool.volumes.get('custom', volume)
volume.delete()

def test_create_and_get_and_delete(self):
pool_name = self.create_storage_pool()
self.addCleanup(self.delete_storage_pool, pool_name)

storage_pool = self.client.storage_pools.get(pool_name)
volume = self.create_storage_volume(storage_pool)
vol_copy = storage_pool.volumes.get("custom", "vol1")
self.assertEqual(vol_copy.name, volume.name)
volume.delete()

@unittest.skip("Can't test PUT on volumes as it doesn't make sense yet")
def test_put(self):
pass

@unittest.skip("Can't test .save() on volumes yet - doesn't make sense")
def test_save(self):
pass

@unittest.skip("Can't test PATCH on volumes yet - doesn't make sense")
def test_patch(self):
# as we're not using ZFS (and can't in these integration tests) we
# can't really patch anything on a dir volume.
pass

0 comments on commit 0d5c227

Please sign in to comment.