Improved config validation

README.rst

@@ -46,7 +46,8 @@ Current features
 * **configuration editor** based on `JSON-Schema editor <https://github.com/jdorn/json-editor>`_
 * **advanced edit mode**: edit `NetJSON`_ *DeviceConfiguration* objects for maximum flexibility
 * **configuration templates**: reduce repetition to the minimum
-* **configuration context**: reference ansible-like variables in the configuration
+* `configuration variables <#how-to-use-configuration-variables>`_: reference
+  ansible-like variables in the configuration and templates
 * **template tags**: tag templates to automate different types of auto-configurations (eg: mesh, WDS, 4G)
 * **simple HTTP resources**: allow devices to automatically download configuration updates
 * **VPN management**: easily create VPN servers and clients
@@ -191,6 +192,127 @@ Run tests with:
 
     ./runtests.py
 
+How to use configuration variables
+----------------------------------
+
+Sometimes the configuration is not exactly equal on all the devices,
+some parameters are unique to each device or need to be changed
+by the user.
+
+In these cases it is possible to use configuration variables in conjunction
+with templates, this feature is also known as *configuration context*, think of
+it like a dictionary which is passed to the function which renders the
+configuration, so that it can fill variables according to the passed context.
+
+The different ways in which variables are defined are described below.
+
+Predefined device variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Each device gets the following attributes passed as configuration variables:
+
+* ``id``
+* ``key``
+* ``name``
+* ``mac_address``
+
+User defined device variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the device configuration section, you can access the context
+field by clicking on "Advanced Options (show)".
+
+.. image:: https://raw.githubusercontent.com/openwisp/django-netjsonconfig/master/docs/images/device-advanced.png
+   :alt: advanced options (show)
+
+Then you can define the variables as a key, value dictionary (JSON formatted)
+as shown below.
+
+.. image:: https://raw.githubusercontent.com/openwisp/django-netjsonconfig/master/docs/images/device-context.png
+   :alt: context
+
+Template default values
+~~~~~~~~~~~~~~~~~~~~~~~
+
+It's possible to specify the default values of variables defined in a template.
+
+This allows to achieve 2 goals:
+
+1. pass schema validation without errors (otherwise it would not be possible
+   to save the template in the first place)
+2. provide good default values that are valid in most cases but can be
+   overridden in the device if needed
+
+These default values will be overridden by the
+`User defined device variables <#user-defined-device-variables>`_.
+
+To do this, click on "Advanced Options (show)" in the edit template page:
+
+.. image:: https://raw.githubusercontent.com/openwisp/django-netjsonconfig/master/docs/images/template-advanced.png
+   :alt: advanced options (show)
+
+Then you can define the default values of the variables:
+
+.. image:: https://raw.githubusercontent.com/openwisp/django-netjsonconfig/master/docs/images/template-default-values.png
+  :alt: default values
+
+Global variables
+~~~~~~~~~~~~~~~~
+
+Variables can also be defined globally using the
+`NETJSONCONFIG_CONTEXT <#netjsonconfig_context>`_ setting.
+
+Example usage of variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Here's a typical use case, the WiFi SSID and WiFi password.
+You don't want to define this for every device, but you may want to
+allow operators to easily change the SSID or WiFi password for a
+specific device without having to re-define the whole wifi interface
+to avoid duplicating information.
+
+This would be the template:
+
+.. code-block:: json
+
+    {
+        "interfaces": [
+            {
+                "type": "wireless",
+                "name": "wlan0",
+                "wireless": {
+                    "mode": "access_point",
+                    "radio": "radio0",
+                    "ssid": "{{wlan0_ssid}}",
+                    "encryption": {
+                        "protocol": "wpa2_personal",
+                        "key": "{{wlan0_password}}",
+                        "cipher": "auto"
+                    }
+                }
+            }
+        ]
+    }
+
+These would be the default values in the template:
+
+.. code-block:: json
+
+    {
+        "wlan0_ssid": "SnakeOil PublicWiFi",
+        "wlan0_password": "Snakeoil_pwd!321654"
+    }
+
+The default values can then be overridden at
+`device level <#user-defined-device-variables>`_ if needed, eg:
+
+.. code-block:: json
+
+    {
+        "wlan0_ssid": "Room 23 ACME Hotel",
+        "wlan0_password": "room_23pwd!321654"
+    }
+
 Signals
 -------
 
@@ -407,18 +529,12 @@ an ``ImproperlyConfigured`` exception will be raised on startup.
 | **default**: | ``{}``           |
 +--------------+------------------+
 
-Additional context that is passed to the default context of each ``Config`` object.
-
-Each ``Config`` object gets the following attributes passed as configuration variables:
-
-* ``id``
-* ``key``
-* ``name``
-* ``mac_address``
+Additional context that is passed to the default context of each device object.
 
 ``NETJSONCONFIG_CONTEXT`` can be used to define system-wide configuration variables.
 
-For more information, see `netjsonconfig context: configuration variables
+For technical information about how variables are handled in the lower levels
+of OpenWISP, see `netjsonconfig context: configuration variables
 <http://netjsonconfig.openwisp.org/en/latest/general/basics.html#context-configuration-variables>`_.
 
 ``NETJSONCONFIG_DEFAULT_AUTO_CERT``

django_netjsonconfig/base/admin.py

@@ -363,18 +363,27 @@ class AbstractTemplateAdmin(BaseConfigAdmin):
     list_display = ['name', 'type', 'backend', 'default', 'created', 'modified']
     list_filter = ['backend', 'type', 'default', 'created']
     search_fields = ['name']
-    fields = [
-        'name',
-        'type',
-        'backend',
-        'vpn',
-        'auto_cert',
-        'tags',
-        'default',
-        'config',
-        'created',
-        'modified',
-    ]
+    fieldsets = (
+        (
+            None,
+            {
+                'fields': [
+                    'name',
+                    'type',
+                    'backend',
+                    'vpn',
+                    'auto_cert',
+                    'tags',
+                    'default',
+                ]
+            },
+        ),
+        (
+            _('Advanced options'),
+            {'classes': ('collapse',), 'fields': ('default_values',)},
+        ),
+        (None, {'fields': ('config', 'created', 'modified')}),
+    )
 
     def clone_selected_templates(self, request, queryset):
         for templates in queryset:

django_netjsonconfig/base/base.py

@@ -148,15 +148,21 @@ def get_backend_instance(self, template_instances=None):
         """
         backend = self.backend_class
         kwargs = {'config': self.get_config()}
+        context = {}
         # determine if we can pass templates
         # expecting a many2many relationship
         if hasattr(self, 'templates'):
             if template_instances is None:
                 template_instances = self.templates.all()
-            kwargs['templates'] = [t.config for t in template_instances]
+            templates_list = list()
+            for t in template_instances:
+                templates_list.append(t.config)
+                context.update(t.get_context())
+            kwargs['templates'] = templates_list
         # pass context to backend if get_context method is defined
         if hasattr(self, 'get_context'):
-            kwargs['context'] = self.get_context()
+            context.update(self.get_context())
+            kwargs['context'] = context
         backend_instance = backend(**kwargs)
         # remove accidentally duplicated files when combining config and templates
         # this may happen if a device uses multiple VPN client templates

django_netjsonconfig/base/template.py

@@ -1,10 +1,12 @@
+from collections import OrderedDict
 from copy import copy
 
 from django.contrib.admin.models import ADDITION, LogEntry
 from django.contrib.contenttypes.models import ContentType
 from django.core.exceptions import ValidationError
 from django.db import models
 from django.utils.translation import ugettext_lazy as _
+from jsonfield import JSONField
 from taggit.managers import TaggableManager
 
 from ..settings import DEFAULT_AUTO_CERT
@@ -73,6 +75,19 @@ class AbstractTemplate(BaseConfig):
             'valid only for the VPN type'
         ),
     )
+    default_values = JSONField(
+        _('Default Values'),
+        default=dict,
+        blank=True,
+        help_text=_(
+            'A dictionary containing the default '
+            'values for the variables used by this '
+            'template; these default variables will '
+            'be used during schema validation.'
+        ),
+        load_kwargs={'object_pairs_hook': OrderedDict},
+        dump_kwargs={'indent': 4},
+    )
     __template__ = True
 
     class Meta:
@@ -114,7 +129,6 @@ def clean(self, *args, **kwargs):
         * clears VPN specific fields if type is not VPN
         * automatically determines configuration if necessary
         """
-        super().clean(*args, **kwargs)
         if self.type == 'vpn' and not self.vpn:
             raise ValidationError(
                 {'vpn': _('A VPN must be selected when template type is "VPN"')}
@@ -124,14 +138,14 @@ def clean(self, *args, **kwargs):
             self.auto_cert = False
         if self.type == 'vpn' and not self.config:
             self.config = self.vpn.auto_client(auto_cert=self.auto_cert)
+        super().clean(*args, **kwargs)
 
     def get_context(self):
-        c = {
-            'id': str(self.id),
-            'name': self.name,
-        }
-        c.update(super().get_context())
-        return c
+        context = {}
+        if self.default_values:
+            context = copy(self.default_values)
+        context.update(super().get_context())
+        return context
 
     def clone(self, user):
         clone = copy(self)

django_netjsonconfig/migrations/0044_template_default_values.py

@@ -0,0 +1,27 @@
+# Generated by Django 3.0.4 on 2020-04-08 23:46
+
+import collections
+from django.db import migrations
+import jsonfield.fields
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('django_netjsonconfig', '0043_add_indexes_on_ip_fields'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='template',
+            name='default_values',
+            field=jsonfield.fields.JSONField(
+                blank=True,
+                default=dict,
+                dump_kwargs={'ensure_ascii': False, 'indent': 4},
+                help_text='A dictionary containing the default values for the variables used by this template; these default variables will be used during schema validation.',
+                load_kwargs={'object_pairs_hook': collections.OrderedDict},
+                verbose_name='Default Values',
+            ),
+        ),
+    ]