Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Clean basic nested serialization #749

Merged
merged 25 commits into from

4 participants

Tom Christie Alan Justino da Silva Mark Aaron Shirley Craig de Stigter
Tom Christie
Owner

Support for writable nested serializers (Serializer support, not include ModelSerializer/HyperLinkedModelSerializer)

Mark Aaron Shirley

Why isn't this a NestedValidationError as well?

Actually I ought to add an explanatory docstring to NestedValidationError.
All that class does is make sure that if the message doesn't get forcibly stringified.
It's needed only in the case of the nested serializer.errors being a list of dicts.

Ah yes, I remember that issue.

Let me know what direction you plan to take this in, and I can update the writable-nested-serializers branch to take it into account.

Yah will do. Should have a little time this eve to expand on this...

and others added some commits
Tom Christie Extra tests for nested serializers d8c5dca
Tom Christie Descriptive text for NestedValidationError 2f19519
Tom Christie Merge branch 'master' into basic-nested-serialization b6b686d
Mark Aaron Shirley maspwr One-to-one writable, nested serializer support 3006e38
Tom Christie one 2 one nested relationships fb3b578
Tom Christie Clean out ModelSerializer special casing from Serializer.field_from_n…
…ative
47492e3
Tom Christie Merge branch 'master' into basic-nested-serialization 1aedf57
Tom Christie Merge branch 'basic-nested-serialization' into one-to-one-nested-wip 5f5b22d
Tom Christie Remove erronous _delete attribute 32e0e5e
Tom Christie Remove unneeded arguments to save_object 5665311
Tom Christie Clean up and comment `restore_object` ccf5512
Tom Christie Fixes to save_object 3ff103a
Tom Christie Fixes to save_object 66bdd60
Tom Christie Merge pull request #735 from tomchristie/one-to-one-nested-wip
One to one writable nested model serializers (wip)
ee20cf8
Craig de Stigter craigds accept all WritableField kwargs for writable serializers (eg required…
…=True)
c8416df
Craig de Stigter craigds allow default values in writable serializer fields d6d5b1d
Craig de Stigter craigds use writablefield style for serializer handling of self.default 101fa26
Tom Christie Merge pull request #739 from craigds/basic-nested-serialization
tweaks to writable nested serializers
75fbfb5
Tom Christie Merge branch 'master' into basic-nested-serialization ad3ffe2
Tom Christie Added bulk create tests deb5e65
Tom Christie Merge master 9bf7c9b
Tom Christie Merge master fe82982
Tom Christie Tweaking 9cdf841
Tom Christie Defer the writable nested modelserializers work addf7e9
Tom Christie tomchristie merged commit 9fdb661 into from
Tom Christie tomchristie deleted the branch
Alan Justino da Silva

1) Is not this "all" test leaking Django ORM stuff at BaseSerializer, when should be at ModelSerializer ?
2) Asking because I need to filter a queryset to ModelSerializer used as a field, by something like this:

class CarSerializer(serializers.ModelSerializer):
    manufacturer = serializers.Field()
    model = serializers.Field()
    class Meta:
        model = Car

def _filter_manufacturer(queryset, instance):
    manufacturer = instance.context['request'].DATA.get('car_manufacturer', [])
    if manufacturer:
        return queryset.filter(manufacturer__in=manufacturer)

class CarOwnerSerializer(serializers.ModelSerializer):
    car_set = CarSerializer(read_only=True, filter_queryset=_filter_manufacturer)

So I would implement filter_queryset at ModelSerializer.__init__() but this would hit BaseSerializer code for something related to Django ORM.

3) Is this a good idea, or have you something cleaner in mind?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 12, 2013
Commits on Mar 14, 2013
  1. Mark Aaron Shirley
Commits on Mar 15, 2013
Commits on Mar 16, 2013
  1. Fixes to save_object

    authored
  2. Fixes to save_object

    authored
  3. Merge pull request #735 from tomchristie/one-to-one-nested-wip

    authored
    One to one writable nested model serializers (wip)
Commits on Mar 18, 2013
  1. Craig de Stigter
  2. Craig de Stigter
  3. Craig de Stigter
  4. Merge pull request #739 from craigds/basic-nested-serialization

    authored
    tweaks to writable nested serializers
  5. Added bulk create tests

    authored
Commits on Mar 22, 2013
  1. Merge master

    authored
  2. Merge master

    authored
  3. Tweaking

    authored
This page is out of date. Refresh to see the latest.
Showing with 351 additions and 39 deletions.
  1. +105 −39 rest_framework/serializers.py
  2. +246 −0 rest_framework/tests/serializer_nested.py
144 rest_framework/serializers.py
View
@@ -20,6 +20,25 @@
from rest_framework.fields import *
+class NestedValidationError(ValidationError):
+ """
+ The default ValidationError behavior is to stringify each item in the list
+ if the messages are a list of error messages.
+
+ In the case of nested serializers, where the parent has many children,
+ then the child's `serializer.errors` will be a list of dicts. In the case
+ of a single child, the `serializer.errors` will be a dict.
+
+ We need to override the default behavior to get properly nested error dicts.
+ """
+
+ def __init__(self, message):
+ if isinstance(message, dict):
+ self.messages = [message]
+ else:
+ self.messages = message
+
+
class DictWithMetadata(dict):
"""
A dict-like object, that can have additional properties attached.
@@ -98,7 +117,7 @@ def __init__(self, meta):
self.exclude = getattr(meta, 'exclude', ())
-class BaseSerializer(Field):
+class BaseSerializer(WritableField):
"""
This is the Serializer implementation.
We need to implement it as `BaseSerializer` due to metaclass magicks.
@@ -110,9 +129,9 @@ class Meta(object):
_dict_class = SortedDictWithMetadata
def __init__(self, instance=None, data=None, files=None,
- context=None, partial=False, many=None, source=None,
- allow_delete=False):
- super(BaseSerializer, self).__init__(source=source)
+ context=None, partial=False, many=None,
+ allow_delete=False, **kwargs):
+ super(BaseSerializer, self).__init__(**kwargs)
self.opts = self._options_class(self.Meta)
self.parent = None
self.root = None
@@ -305,40 +324,77 @@ def from_native(self, data, files):
def field_to_native(self, obj, field_name):
"""
- Override default so that we can apply ModelSerializer as a nested
- field to relationships.
+ Override default so that the serializer can be used as a nested field
+ across relationships.
"""
if self.source == '*':
return self.to_native(obj)
try:
- if self.source:
- for component in self.source.split('.'):
- obj = getattr(obj, component)
- if is_simple_callable(obj):
- obj = obj()
- else:
- obj = getattr(obj, field_name)
- if is_simple_callable(obj):
- obj = obj()
+ source = self.source or field_name
+ value = obj
+
+ for component in source.split('.'):
+ value = get_component(value, component)
+ if value is None:
+ break
except ObjectDoesNotExist:
return None
- # If the object has an "all" method, assume it's a relationship
- if is_simple_callable(getattr(obj, 'all', None)):
- return [self.to_native(item) for item in obj.all()]
+ if is_simple_callable(getattr(value, 'all', None)):
+ return [self.to_native(item) for item in value.all()]
- if obj is None:
+ if value is None:
return None
if self.many is not None:
many = self.many
else:
- many = hasattr(obj, '__iter__') and not isinstance(obj, (Page, dict, six.text_type))
+ many = hasattr(value, '__iter__') and not isinstance(value, (Page, dict, six.text_type))
if many:
- return [self.to_native(item) for item in obj]
- return self.to_native(obj)
+ return [self.to_native(item) for item in value]
+ return self.to_native(value)
+
+ def field_from_native(self, data, files, field_name, into):
+ """
+ Override default so that the serializer can be used as a writable
+ nested field across relationships.
+ """
+ if self.read_only:
+ return
+
+ try:
+ value = data[field_name]
+ except KeyError:
+ if self.default is not None and not self.partial:
+ # Note: partial updates shouldn't set defaults
+ value = copy.deepcopy(self.default)
+ else:
+ if self.required:
+ raise ValidationError(self.error_messages['required'])
+ return
+
+ # Set the serializer object if it exists
+ obj = getattr(self.parent.object, field_name) if self.parent.object else None
+
+ if value in (None, ''):
+ into[(self.source or field_name)] = None
+ else:
+ kwargs = {
+ 'instance': obj,
+ 'data': value,
+ 'context': self.context,
+ 'partial': self.partial,
+ 'many': self.many
+ }
+ serializer = self.__class__(**kwargs)
+
+ if serializer.is_valid():
+ into[self.source or field_name] = serializer.object
+ else:
+ # Propagate errors up to our parent
+ raise NestedValidationError(serializer.errors)
def get_identity(self, data):
"""
@@ -637,33 +693,43 @@ def restore_object(self, attrs, instance=None):
"""
Restore the model instance.
"""
- self.m2m_data = {}
- self.related_data = {}
+ m2m_data = {}
+ related_data = {}
+ meta = self.opts.model._meta
- # Reverse fk relations
- for (obj, model) in self.opts.model._meta.get_all_related_objects_with_model():
+ # Reverse fk or one-to-one relations
+ for (obj, model) in meta.get_all_related_objects_with_model():
field_name = obj.field.related_query_name()
if field_name in attrs:
- self.related_data[field_name] = attrs.pop(field_name)
+ related_data[field_name] = attrs.pop(field_name)
# Reverse m2m relations
- for (obj, model) in self.opts.model._meta.get_all_related_m2m_objects_with_model():
+ for (obj, model) in meta.get_all_related_m2m_objects_with_model():
field_name = obj.field.related_query_name()
if field_name in attrs:
- self.m2m_data[field_name] = attrs.pop(field_name)
+ m2m_data[field_name] = attrs.pop(field_name)
# Forward m2m relations
- for field in self.opts.model._meta.many_to_many:
+ for field in meta.many_to_many:
if field.name in attrs:
- self.m2m_data[field.name] = attrs.pop(field.name)
+ m2m_data[field.name] = attrs.pop(field.name)
+ # Update an existing instance...
if instance is not None:
for key, val in attrs.items():
setattr(instance, key, val)
+ # ...or create a new instance
else:
instance = self.opts.model(**attrs)
+ # Any relations that cannot be set until we've
+ # saved the model get hidden away on these
+ # private attributes, so we can deal with them
+ # at the point of save.
+ instance._related_data = related_data
+ instance._m2m_data = m2m_data
+
return instance
def from_native(self, data, files):
@@ -680,15 +746,15 @@ def save_object(self, obj, **kwargs):
"""
obj.save(**kwargs)
- if getattr(self, 'm2m_data', None):
- for accessor_name, object_list in self.m2m_data.items():
- setattr(self.object, accessor_name, object_list)
- self.m2m_data = {}
+ if getattr(obj, '_m2m_data', None):
+ for accessor_name, object_list in obj._m2m_data.items():
+ setattr(obj, accessor_name, object_list)
+ del(obj._m2m_data)
- if getattr(self, 'related_data', None):
- for accessor_name, object_list in self.related_data.items():
- setattr(self.object, accessor_name, object_list)
- self.related_data = {}
+ if getattr(obj, '_related_data', None):
+ for accessor_name, related in obj._related_data.items():
+ setattr(obj, accessor_name, related)
+ del(obj._related_data)
class HyperlinkedModelSerializerOptions(ModelSerializerOptions):
246 rest_framework/tests/serializer_nested.py
View
@@ -0,0 +1,246 @@
+"""
+Tests to cover nested serializers.
+
+Doesn't cover model serializers.
+"""
+from __future__ import unicode_literals
+from django.test import TestCase
+from rest_framework import serializers
+
+
+class WritableNestedSerializerBasicTests(TestCase):
+ """
+ Tests for deserializing nested entities.
+ Basic tests that use serializers that simply restore to dicts.
+ """
+
+ def setUp(self):
+ class TrackSerializer(serializers.Serializer):
+ order = serializers.IntegerField()
+ title = serializers.CharField(max_length=100)
+ duration = serializers.IntegerField()
+
+ class AlbumSerializer(serializers.Serializer):
+ album_name = serializers.CharField(max_length=100)
+ artist = serializers.CharField(max_length=100)
+ tracks = TrackSerializer(many=True)
+
+ self.AlbumSerializer = AlbumSerializer
+
+ def test_nested_validation_success(self):
+ """
+ Correct nested serialization should return the input data.
+ """
+
+ data = {
+ 'album_name': 'Discovery',
+ 'artist': 'Daft Punk',
+ 'tracks': [
+ {'order': 1, 'title': 'One More Time', 'duration': 235},
+ {'order': 2, 'title': 'Aerodynamic', 'duration': 184},
+ {'order': 3, 'title': 'Digital Love', 'duration': 239}
+ ]
+ }
+
+ serializer = self.AlbumSerializer(data=data)
+ self.assertEqual(serializer.is_valid(), True)
+ self.assertEqual(serializer.object, data)
+
+ def test_nested_validation_error(self):
+ """
+ Incorrect nested serialization should return appropriate error data.
+ """
+
+ data = {
+ 'album_name': 'Discovery',
+ 'artist': 'Daft Punk',
+ 'tracks': [
+ {'order': 1, 'title': 'One More Time', 'duration': 235},
+ {'order': 2, 'title': 'Aerodynamic', 'duration': 184},
+ {'order': 3, 'title': 'Digital Love', 'duration': 'foobar'}
+ ]
+ }
+ expected_errors = {
+ 'tracks': [
+ {},
+ {},
+ {'duration': ['Enter a whole number.']}
+ ]
+ }
+
+ serializer = self.AlbumSerializer(data=data)
+ self.assertEqual(serializer.is_valid(), False)
+ self.assertEqual(serializer.errors, expected_errors)
+
+ def test_many_nested_validation_error(self):
+ """
+ Incorrect nested serialization should return appropriate error data
+ when multiple entities are being deserialized.
+ """
+
+ data = [
+ {
+ 'album_name': 'Russian Red',
+ 'artist': 'I Love Your Glasses',
+ 'tracks': [
+ {'order': 1, 'title': 'Cigarettes', 'duration': 121},
+ {'order': 2, 'title': 'No Past Land', 'duration': 198},
+ {'order': 3, 'title': 'They Don\'t Believe', 'duration': 191}
+ ]
+ },
+ {
+ 'album_name': 'Discovery',
+ 'artist': 'Daft Punk',
+ 'tracks': [
+ {'order': 1, 'title': 'One More Time', 'duration': 235},
+ {'order': 2, 'title': 'Aerodynamic', 'duration': 184},
+ {'order': 3, 'title': 'Digital Love', 'duration': 'foobar'}
+ ]
+ }
+ ]
+ expected_errors = [
+ {},
+ {
+ 'tracks': [
+ {},
+ {},
+ {'duration': ['Enter a whole number.']}
+ ]
+ }
+ ]
+
+ serializer = self.AlbumSerializer(data=data)
+ self.assertEqual(serializer.is_valid(), False)
+ self.assertEqual(serializer.errors, expected_errors)
+
+
+class WritableNestedSerializerObjectTests(TestCase):
+ """
+ Tests for deserializing nested entities.
+ These tests use serializers that restore to concrete objects.
+ """
+
+ def setUp(self):
+ # Couple of concrete objects that we're going to deserialize into
+ class Track(object):
+ def __init__(self, order, title, duration):
+ self.order, self.title, self.duration = order, title, duration
+
+ def __eq__(self, other):
+ return (
+ self.order == other.order and
+ self.title == other.title and
+ self.duration == other.duration
+ )
+
+ class Album(object):
+ def __init__(self, album_name, artist, tracks):
+ self.album_name, self.artist, self.tracks = album_name, artist, tracks
+
+ def __eq__(self, other):
+ return (
+ self.album_name == other.album_name and
+ self.artist == other.artist and
+ self.tracks == other.tracks
+ )
+
+ # And their corresponding serializers
+ class TrackSerializer(serializers.Serializer):
+ order = serializers.IntegerField()
+ title = serializers.CharField(max_length=100)
+ duration = serializers.IntegerField()
+
+ def restore_object(self, attrs, instance=None):
+ return Track(attrs['order'], attrs['title'], attrs['duration'])
+
+ class AlbumSerializer(serializers.Serializer):
+ album_name = serializers.CharField(max_length=100)
+ artist = serializers.CharField(max_length=100)
+ tracks = TrackSerializer(many=True)
+
+ def restore_object(self, attrs, instance=None):
+ return Album(attrs['album_name'], attrs['artist'], attrs['tracks'])
+
+ self.Album, self.Track = Album, Track
+ self.AlbumSerializer = AlbumSerializer
+
+ def test_nested_validation_success(self):
+ """
+ Correct nested serialization should return a restored object
+ that corresponds to the input data.
+ """
+
+ data = {
+ 'album_name': 'Discovery',
+ 'artist': 'Daft Punk',
+ 'tracks': [
+ {'order': 1, 'title': 'One More Time', 'duration': 235},
+ {'order': 2, 'title': 'Aerodynamic', 'duration': 184},
+ {'order': 3, 'title': 'Digital Love', 'duration': 239}
+ ]
+ }
+ expected_object = self.Album(
+ album_name='Discovery',
+ artist='Daft Punk',
+ tracks=[
+ self.Track(order=1, title='One More Time', duration=235),
+ self.Track(order=2, title='Aerodynamic', duration=184),
+ self.Track(order=3, title='Digital Love', duration=239),
+ ]
+ )
+
+ serializer = self.AlbumSerializer(data=data)
+ self.assertEqual(serializer.is_valid(), True)
+ self.assertEqual(serializer.object, expected_object)
+
+ def test_many_nested_validation_success(self):
+ """
+ Correct nested serialization should return multiple restored objects
+ that corresponds to the input data when multiple objects are
+ being deserialized.
+ """
+
+ data = [
+ {
+ 'album_name': 'Russian Red',
+ 'artist': 'I Love Your Glasses',
+ 'tracks': [
+ {'order': 1, 'title': 'Cigarettes', 'duration': 121},
+ {'order': 2, 'title': 'No Past Land', 'duration': 198},
+ {'order': 3, 'title': 'They Don\'t Believe', 'duration': 191}
+ ]
+ },
+ {
+ 'album_name': 'Discovery',
+ 'artist': 'Daft Punk',
+ 'tracks': [
+ {'order': 1, 'title': 'One More Time', 'duration': 235},
+ {'order': 2, 'title': 'Aerodynamic', 'duration': 184},
+ {'order': 3, 'title': 'Digital Love', 'duration': 239}
+ ]
+ }
+ ]
+ expected_object = [
+ self.Album(
+ album_name='Russian Red',
+ artist='I Love Your Glasses',
+ tracks=[
+ self.Track(order=1, title='Cigarettes', duration=121),
+ self.Track(order=2, title='No Past Land', duration=198),
+ self.Track(order=3, title='They Don\'t Believe', duration=191),
+ ]
+ ),
+ self.Album(
+ album_name='Discovery',
+ artist='Daft Punk',
+ tracks=[
+ self.Track(order=1, title='One More Time', duration=235),
+ self.Track(order=2, title='Aerodynamic', duration=184),
+ self.Track(order=3, title='Digital Love', duration=239),
+ ]
+ )
+ ]
+
+ serializer = self.AlbumSerializer(data=data)
+ self.assertEqual(serializer.is_valid(), True)
+ self.assertEqual(serializer.object, expected_object)
Something went wrong with that request. Please try again.