Merge pull request #82 from postatum/104069592_field_processors

Rework field processors

docs/source/event_handlers.rst

@@ -1,32 +1,35 @@
 Event Handlers
 ==============
 
-Ramses supports Nefertari event handlers. The following documentation describes how to define and connect them.
+Ramses supports Nefertari model event handlers. The following documentation describes how to define and connect them.
 
 
 Writing Event Handlers
 ----------------------
 
 You can write custom functions inside your ``__init__.py`` file, then add the ``@registry.add`` decorator before the functions that you'd like to turn into CRUD event handlers. Ramses CRUD event handlers has the same API as Nefertari CRUD event handlers. Check Nefertari CRUD Events doc for more details on events API.
 
-E.g.
+Example:
 
 .. code-block:: python
 
     @registry.add
-    def lowercase(event):
-        """ This processor lowercases the value of a field """
-        value = (event.field.new_value or '').lower().strip()
-        event.set_field_value(value)
+    def log_changed_fields(event):
+        import logging
+        logger = logging.getLogger('foo')
+        changed = ['{}: {}'.format(name, field.new_value)
+                   for name, field in event.fields.items()]
+        logger.debug('Changed fields: ' + ', '.join(changed))
 
 
 Connecting Event Handlers
 -------------------------
 
-When you define event handlers in your ``__init__.py`` as described above, you can apply them on either a per-model or a per-field basis. If multiple handlers are listed, they are executed in the order in which they are listed. Handlers are defined using the ``_event_handlers`` property. This property is an object, keys of which are called "event tags" and values are lists of handler names. Event tags are constructed of two parts: ``<type>_<action>`` whereby:
+When you define event handlers in your ``__init__.py`` as described above, you can apply them on per-model basis. If multiple handlers are listed, they are executed in the order in which they are listed. Handlers are defined in the root of JSON schema using ``_event_handlers`` property. This property is an object, keys of which are called "event tags" and values are lists of handler names. Event tags are constructed of two parts: ``<type>_<action>`` whereby:
 
 **type**
     Is either ``before`` or ``after``, depending on when handler should run - before view method call or after respectively.
+
 **action**
     Exact name of Nefertari view method that processes the request (action).
 
@@ -84,10 +87,10 @@ Before vs After
     * Perform notifications/logging
 
 
-Per-model Handlers
-------------------
+Registering Event Handlers
+--------------------------
 
-To register handlers on a per-model basis, you can define the ``_event_handlers`` property at the root of your model's JSON schema. For example, if we have a JSON schema for the model ``User`` and we want to log all collection GET requests to the ``User`` model after they were processed (using the ``log_request`` handler), we can register the handler in the JSON schema like this:
+To register event handlers, you can define the ``_event_handlers`` property at the root of your model's JSON schema. For example, if we have a JSON schema for the model ``User`` and we want to log all collection GET requests to the ``User`` model after they were processed (using the ``log_request`` handler), we can register the handler in the JSON schema like this:
 
 .. code-block:: json
 
@@ -102,43 +105,6 @@ To register handlers on a per-model basis, you can define the ``_event_handlers`
     }
 
 
-Per-field Handlers
-------------------
-
-To register handlers on a per-field basis, you can define the ``_event_handlers`` property inside the fields of your JSON schema (same level as ``_db_settings``).
-
-E.g. if our model ``User`` has a field ``username``, we might want to make sure that ``username`` is not a reserved name. If ``username`` is a reserved name, we want to raise an exception to interrupt the request.
-
-.. code-block:: python
-
-    @registry.add
-    def check_username(event):
-        reserved = ('admin', 'cat', 'system')
-        username = event.field.new_value
-        if username in reserved:
-            raise ValueError('Reserved username: {}'.format(username))
-
-
-The following JSON schema registers ``before_set`` on the field ``User.username``. When connected this way, the ``check_username`` handler will only be executed if the request has the field ``username`` passed to it:
-
-.. code-block:: json
-
-    {
-        "type": "object",
-        "title": "User schema",
-        "$schema": "http://json-schema.org/draft-04/schema",
-        "properties": {
-            "username": {
-                "_db_settings": {...},
-                "_event_handlers": {
-                    "before_set": ["check_username"]
-                }
-            }
-        }
-        ...
-    }
-
-
 Other Things You Can Do
 -----------------------
 
@@ -150,22 +116,7 @@ You can update another field's value, for example, increment a counter:
     def increment_count(event):
         counter = event.instance.counter
         incremented = counter + 1
-        event.set_field_value(incremented, 'counter')
-
-
-You can transform the value of a field, for example, encrypt a password before saving it:
-
-.. code-block:: python
-
-    @registry.add
-    def encrypt(event):
-        import cryptacular.bcrypt
-        crypt = cryptacular.bcrypt.BCRYPTPasswordManager()
-        password = event.field.new_value
-
-        if password and not crypt.match(password):
-            encrypted = str(crypt.encode(password))
-            event.set_field_value(encrypted)
+        event.set_field_value('counter', incremented)
 
 
 You can update other collections (or filtered collections), for example, mark sub-tasks as completed whenever a task is completed:
@@ -174,9 +125,11 @@ You can update other collections (or filtered collections), for example, mark su
 
     @registry.add
     def mark_subtasks_completed(event):
+        if 'task' not in event.fields:
+            return
 
         from nefertari import engine
-        completed = event.field.new_value
+        completed = event.fields['task'].new_value
         instance = event.instance
 
         if completed:
@@ -191,10 +144,12 @@ You can perform more complex queries using ElasticSearch:
 
     @registry.add
     def mark_subtasks_after_2015_completed(event):
+        if 'task' not in event.fields:
+            return
 
         from nefertari import engine
         from nefertari.elasticsearch import ES
-        completed = event.field.new_value
+        completed = event.fields['task'].new_value
         instance = event.instance
 
         if completed:

docs/source/field_validators.rst

@@ -0,0 +1,111 @@
+Field validators
+================
+
+Ramses allows users to define functions that accept field data and return modified field value, may perform validation or perform other actions related to field.
+
+These functions are called "field validators". They are set up per-field and are called when request comes into application that modifies the field for which validator is set up (when field is present in request JSON).
+
+
+Usage basics
+------------
+
+Field validators are defined in your target project just like event handlers:
+
+.. code-block:: python
+
+    @registry.add
+    def lowercase(**kwargs):
+        """ Make :new_value: lowercase (and stripped) """
+        return (kwargs['new_value'] or '').lower().strip()
+
+
+To use this field validator, define ``_validators`` attribute in your field definition (next to ``_db_settings``) which should be an array listing names of validators to apply. You can also use ``_backref_validators`` attribute defined the same way to specify validators for backref field. For backref validators to be set up, ``_db_settings`` must contain attributes ``document``, ``type=relationship`` and ``backref_name``.
+
+Field validators should expect following kwargs to be passed:
+
+**new_value**
+    New value of of field.
+
+**instance**
+    Instance affected by request. Is None when set of items is updated in bulk and when item is created.
+
+**field**
+    Instance of nefertari.utils.data.FieldData instance containing data of changed field.
+
+**request**
+    Current Pyramid Request instance.
+
+**model**
+    Model class affected by request.
+
+**event**
+    Underlying event object. Should be used to edit other fields of instance using ``event.set_field_value(field_name, value)``.
+
+Processors are called in order they are listed. Each validator must return processed value which is used a input for next validator if present.
+
+
+Examples
+--------
+
+If we had following validators defined:
+
+.. code-block:: python
+
+    from .my_helpers import get_stories_by_ids
+
+    @registry.add
+    def lowercase(**kwargs):
+        """ Make :new_value: lowercase (and stripped) """
+        return (kwargs['new_value'] or '').lower().strip()
+
+    @registry.add
+    def validate_stories_exist(**kwargs):
+        """ Make sure added stories exist. """
+        story_ids = kwargs['new_value']
+        if story_ids:
+            # Get stories by ids
+            stories = get_stories_by_ids(story_ids)
+            if not stories or len(stories) < len(story_ids):
+                raise Exception("Some of provided stories do not exist")
+        return story_ids
+
+
+.. code-block:: json
+
+    # User model json
+    {
+        "type": "object",
+        "title": "User schema",
+        "$schema": "http://json-schema.org/draft-04/schema",
+        "properties": {
+            "stories": {
+                "_db_settings": {
+                    "type": "relationship",
+                    "document": "Story",
+                    "backref_name": "owner"
+                },
+                "_validators": ["validate_stories_exist"],
+                "_backref_validators": ["lowercase"]
+            },
+            ...
+        }
+    }
+
+When connected like above:
+    * ``validate_stories_exist`` validator will be run when request changes ``User.stories`` value. The validator will make sure all of story IDs from request exist.
+    * ``lowercase`` validator will be run when request changes ``Story.owner`` field. The validator will lowercase new value of the ``Story.owner`` field.
+
+To edit other fields of instance, ``event.set_field_value`` method should be used. E.g. if we have fields ``due_date`` and ``days_left`` and we connect validator defined below to field ``due_date``, we can update ``days_left`` from it:
+
+.. code-block:: python
+
+    from .helpers import parse_data
+    from datetime import datetime
+
+    @registry.add
+    def calculate_days_left(**kwargs):
+        parsed_date = parse_data(kwargs['new_value'])
+        days_left = (parsed_date-datetime.now()).days
+        event = kwargs['event']
+        event.set_field_value('days_left', days_left)
+        return kwargs['new_value']

docs/source/index.rst

@@ -16,6 +16,7 @@ Table of Contents
    schemas
    fields
    event_handlers
+   field_validators
    relationships
    changelog
 

ramses/models.py

@@ -162,7 +162,8 @@ def generate_model_cls(config, schema, model_name, raml_resource,
 
     # Generate new model class
     model_cls = metaclass(model_name, tuple(bases), attrs)
-    setup_event_subscribers(config, model_cls, schema)
+    setup_model_event_subscribers(config, model_cls, schema)
+    setup_fields_validators(config, model_cls, schema)
     return model_cls, auth_model
 
 
@@ -208,16 +209,18 @@ def handle_model_generation(config, raml_resource, route_name):
         raise ValueError('{}: {}'.format(model_name, str(ex)))
 
 
-def _connect_subscribers(config, events_map, events_schema, event_kwargs):
-    """ Performs the actual subscribers set up.
+def setup_model_event_subscribers(config, model_cls, schema):
+    """ Set up model event subscribers.
 
     :param config: Pyramid Configurator instance.
-    :param events_map: Dict returned by `get_events_map`.
-    :param events_schema: Dict of {event_tag: [handler1, ...]}
-    :param event_kwargs: Dict of kwargs to be used when subscribing
-        to event.
+    :param model_cls: Model class for which handlers should be connected.
+    :param schema: Dict of model JSON schema.
     """
-    for event_tag, subscribers in events_schema.items():
+    events_map = get_events_map()
+    model_events = schema.get('_event_handlers', {})
+    event_kwargs = {'model': model_cls}
+
+    for event_tag, subscribers in model_events.items():
         type_, action = event_tag.split('_')
         event_objects = events_map[type_][action]
 
@@ -230,27 +233,40 @@ def _connect_subscribers(config, events_map, events_schema, event_kwargs):
                 sub_func, event_objects, **event_kwargs)
 
 
-def setup_event_subscribers(config, model_cls, schema):
-    """ High level function to set up event subscribers.
+def setup_fields_validators(config, model_cls, schema):
+    """ Set up model fields' validators.
 
     :param config: Pyramid Configurator instance.
-    :param model_cls: Model class for which handlers should be connected.
+    :param model_cls: Model class for field of which validators should be
+        set up.
     :param schema: Dict of model JSON schema.
     """
-    events_map = get_events_map()
-
-    # Model events
-    model_events = schema.get('_event_handlers', {})
-    event_kwargs = {'model': model_cls}
-    _connect_subscribers(config, events_map, model_events, event_kwargs)
-
-    # Field events
     properties = schema.get('properties', {})
     for field_name, props in properties.items():
-
-        if not props or '_event_handlers' not in props:
+        if not props:
             continue
 
-        field_events = props.get('_event_handlers', {})
-        event_kwargs = {'model': model_cls, 'field': field_name}
-        _connect_subscribers(config, events_map, field_events, event_kwargs)
+        validators = props.get('_validators')
+        backref_validators = props.get('_backref_validators')
+
+        if validators:
+            validators = [resolve_to_callable(val) for val in validators]
+            setup_kwargs = {'model': model_cls, 'field': field_name}
+            config.add_field_processors(validators, **setup_kwargs)
+
+        if backref_validators:
+            db_settings = props.get('_db_settings', {})
+            is_relationship = db_settings.get('type') == 'relationship'
+            document = db_settings.get('document')
+            backref_name = db_settings.get('backref_name')
+            if not (is_relationship and document and backref_name):
+                continue
+
+            backref_validators = [
+                resolve_to_callable(val) for val in backref_validators]
+            setup_kwargs = {
+                'model': engine.get_document_cls(document),
+                'field': backref_name
+            }
+            config.add_field_processors(
+                backref_validators, **setup_kwargs)