Skip to content

Commit

Permalink
merge
Browse files Browse the repository at this point in the history
  • Loading branch information
advornic committed Aug 6, 2015
2 parents e88d468 + 92577d4 commit 560adc9
Show file tree
Hide file tree
Showing 3 changed files with 277 additions and 7 deletions.
78 changes: 71 additions & 7 deletions docs/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,8 @@ The values of the attributes can be either strings, numbers, lists, dictionaries
Plugins can be used to allocate resources on the server side and then pass the result of the allocation back to the client via the definition. The supported plugins are:

- **allocate(resource\_pool)** - allocates an available resource from a file-based resource pool
- **sqlite(resource\_pool)** - allocates an available resource from a sqlite database


.. note::

Expand Down Expand Up @@ -656,7 +658,7 @@ Actions
``[data_root]/actions/`` contains the set of all actions available for use in
definitions.

New custom actions to-be referenced from definitions can be added to
New custom actions to-be referenced from definitions can be added to
``[data_root]/actions/``. These will be loaded on-demand and do not require
a restart of the ZTPServer. See ``[data_root]/actions`` for examples.

Expand Down Expand Up @@ -765,22 +767,22 @@ just standard EOS configuration blocks.
Plugins for allocating resources
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Plugins for allocating resources from resource pools
are located in ``[data_root]/plugins/`` and are referenced
Plugins for allocating resources from resource pools
are located in ``[data_root]/plugins/`` and are referenced
by ``<filename>(<resource_pool>)``.

Each plugin contains a ``main`` function with the following signature:

def main(node_id, pool):
def main(node_id, pool):
...

where:
- ``node_id`` is the unique_id of the node being provisioned
- ``pool`` is the name of the resource pool from which an attribute is being allocated

New custom plugins to-be referenced from definitions can be added to
New custom plugins to-be referenced from definitions can be added to
``[data_root]/plugins/``. These will be loaded on-demand and do not require
a restart of the ZTPServer. See ``[data_root]/plugins/test`` for a very basic
a restart of the ZTPServer. See ``[data_root]/plugins/test`` for a very basic
example.

**allocate(resource_pool)**
Expand Down Expand Up @@ -839,6 +841,68 @@ associated to it back to ``null``, by editing the resource file.
Alternatively, ``$ztps --clear-resources`` can be used in order to free
all resources in all file-based resource files.

**sqlite(resource_pool)**

Allocates a resource from a pre-filled sqlite database. The database
is defined by the global variable, 'DB_URL' within the plugin. The database
can include multiple tables, but the value passed into the
'sqlite(resource_pool)' function will be used to look for an available resource.

Table structure should be as follows with the exact column names:

=============== ========
node_id key
=============== ========
NULL 1.1.1.1
NULL 1.1.1.2
NULL 1.1.1.3
=============== ========


Which can be created with statements like:

.. code-block:: mysql
CREATE TABLE `mgmt_subnet`(key TEXT, node_id TEXT)
and add entries with:

.. code-block:: mysql
INSERT INTO `mgmt_subnet` VALUES('1.1.1.1', NULL)
When a resource is added, the node_id row will be updated
to include the System ID from the switch.

=============== ========
node_id key
=============== ========
001122334455 1.1.1.1
NULL 1.1.1.2
NULL 1.1.1.3
=============== ========

On subsequent attempts to allocate the resource to the same node,
ztpserver will first check to see whether the node has already been
allocated a resource from the pool. If it has, it will reuse the
resource instead of allocating a new one.

Definition example:

.. code-block:: yaml
actions:
-
action: add_config
attributes:
url: files/templates/ma1.templates
variables:
ipaddress: sqlite('mgmt_subnet')
name: "configure ma1"
.. tip::
Check out `create_db.py <https://raw.githubusercontent.com/arista-eosplus/ztpserver/develop/utils/create_db.py>`_ for an example script to create a sqlite database.

Config-handlers
~~~~~~~~~~~~~~~

Expand Down
166 changes: 166 additions & 0 deletions plugins/sqlite
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
# Copyright (c) 2015, Arista Networks, Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# Neither the name of Arista Networks nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL ARISTA NETWORKS
# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
# BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
# OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
# IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
'''
Allocates a resource from a prefilled sqlite database. The database
is defined by the global variable, 'DB_URL' below. The database
can include multiple tables, but the value passed into the
'sqlite(table)' will be used to look for an available resource.

Table structure should be as follows:

---------------------
| node_id | key |
---------------------
| NULL | 1.1.1.1 |
| NULL | 1.1.1.2 |
| NULL | 1.1.1.3 |
---------------------

Which can be created with statements like:

CREATE TABLE `mgmt_subnet`(key TEXT, node_id TEXT)

and add entries with:

INSERT INTO `mgmt_subnet` VALUES('1.1.1.1', NULL)

When a resource is added, the node_id row will be updated
to include the System ID from the switch.

--------------------------
| node_id | key |
--------------------------
| 001122334455 | 1.1.1.1 |
| NULL | 1.1.1.2 |
| NULL | 1.1.1.3 |
--------------------------

On subsequent attempts to allocate the resource to the same node,
ztpserver will first check to see whether the node has already been
allocated a resource from the pool. If it has, it will reuse the
resource instead of allocating a new one.

Definition example:

actions:
-
action: add_config
attributes:
url: files/templates/ma1.templates
variables:
ipaddress: sqlite('mgmt_subnet')
name: "configure ma1"
'''

import logging
import os
import sqlite3 as lite

log = logging.getLogger('ztpserver') #pylint: disable=C0103

# SQLITE VARIABLES
DB_URL = "/usr/share/ztpserver/db/resources.db"


def check_url_valid(url):

if not url.startswith('http'):
log.info('checking sqlite db (%s) exists...' % DB_URL)
if not os.path.isfile(url):
raise Exception('Specified DB file %s does not exists.'
% url)


def assign_resource(node_id, table):

# Proactively check if the db file exists
check_url_valid(DB_URL)

log.info('%s: looking for resources in sqlite DB(%s) in table(%s)'
% (node_id, DB_URL, table))

#Setup connection to local sqlite database
con = lite.connect(DB_URL)

with con:
cur = con.cursor()

query = "SELECT * FROM `%s` WHERE node_id='%s'" % (table, node_id)

log.debug('%s: executing sql query:%s' % (node_id, query))
match = cur.execute(query).fetchone()

if match:
log.debug('%s: already allocated:%s in table %s'
% (node_id, match[0], table))
return match[0]
else:
log.info('%s: no existing resources matches this node '
'in the db. Looking for new resource in %s'
% (node_id, table))

# The query must use subquery since sqlite is not
# typically compiled with LIMIT support for UPDATE
query = """UPDATE `%s`
SET node_id = '%s'
WHERE key IN (
SELECT key
FROM `%s`
WHERE node_id IS NULL
ORDER BY rowid ASC
LIMIT 1
)""" % (table, node_id, table)
log.debug('%s: executing query: %s' % (node_id, query))
assigned = cur.execute(query).fetchone()
log.debug('%s: number of rows affected:%s' %
(node_id, cur.rowcount))

if cur.rowcount == 1:
# Go get the resouce that was just allocated
query = "SELECT * FROM `%s` WHERE node_id='%s'" % (table, node_id)
log.debug('%s: executing sql query:%s' % (node_id, query))
match = cur.execute(query).fetchone()
return match[0]


def main(node_id, table):
try:
key = assign_resource(node_id, table)
log.debug('%s: assigned resource from \'%s\': %s' %
(node_id, table, key))

except Exception as exc:
msg = '%s: failed to allocate resource from \'%s\'' % \
(node_id, table)
log.error(msg)
raise Exception('%s : %s' % (msg, exc.message))

return str(key)
40 changes: 40 additions & 0 deletions utils/create_db.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-

import sqlite3 as lite
import sys

con = lite.connect('/usr/share/ztpserver/db/resources.db')

with con:

for table in ['mgmt_subnet', 'tor_hostnames', 'ip_vlan100', 'ip_loopback']:
print "Working on: ",table
cur = con.cursor()
sql = "DROP TABLE IF EXISTS `%s`" % table
cur.execute(sql)
sql = "CREATE TABLE `%s`(key TEXT, node_id TEXT)" % table
cur.execute(sql)
if table == "mgmt_subnet":
base = "172.16.130."
subnet = "/24"
elif table == "tor_hostnames":
base = "veos-dc1-pod1-tor"
subnet = ""
elif table == "ip_vlan100":
base = "10.100.1."
subnet = "/24"
elif table == "ip_loopback":
base = "1.1.1."
subnet = "/32"

for x in range(1,500):
sql = "INSERT INTO %s VALUES('%s%s%s',NULL)" % (table, base, str(x), subnet)
cur.execute(sql)

sql = "SELECT * FROM `%s`" % table
print sql
cur.execute(sql)
rows = cur.fetchall()
for row in rows:
print row

0 comments on commit 560adc9

Please sign in to comment.