Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Clean basic nested serialization #749

Merged
merged 25 commits into from

4 participants

@tomchristie
Owner

Support for writable nested serializers (Serializer support, not include ModelSerializer/HyperLinkedModelSerializer)

@maspwr

Why isn't this a NestedValidationError as well?

Actually I ought to add an explanatory docstring to NestedValidationError.
All that class does is make sure that if the message doesn't get forcibly stringified.
It's needed only in the case of the nested serializer.errors being a list of dicts.

Ah yes, I remember that issue.

Let me know what direction you plan to take this in, and I can update the writable-nested-serializers branch to take it into account.

Yah will do. Should have a little time this eve to expand on this...

and others added some commits
@tomchristie Extra tests for nested serializers d8c5dca
@tomchristie Descriptive text for NestedValidationError 2f19519
@tomchristie Merge branch 'master' into basic-nested-serialization b6b686d
@maspwr maspwr One-to-one writable, nested serializer support 3006e38
@tomchristie one 2 one nested relationships fb3b578
@tomchristie Clean out ModelSerializer special casing from Serializer.field_from_n…
…ative
47492e3
@tomchristie Merge branch 'master' into basic-nested-serialization 1aedf57
@tomchristie Merge branch 'basic-nested-serialization' into one-to-one-nested-wip 5f5b22d
@tomchristie Remove erronous _delete attribute 32e0e5e
@tomchristie Remove unneeded arguments to save_object 5665311
@tomchristie Clean up and comment `restore_object` ccf5512
@tomchristie Fixes to save_object 3ff103a
@tomchristie Fixes to save_object 66bdd60
@tomchristie Merge pull request #735 from tomchristie/one-to-one-nested-wip
One to one writable nested model serializers (wip)
ee20cf8
@craigds craigds accept all WritableField kwargs for writable serializers (eg required…
…=True)
c8416df
@craigds craigds allow default values in writable serializer fields d6d5b1d
@craigds craigds use writablefield style for serializer handling of self.default 101fa26
@tomchristie Merge pull request #739 from craigds/basic-nested-serialization
tweaks to writable nested serializers
75fbfb5
@tomchristie Merge branch 'master' into basic-nested-serialization ad3ffe2
@tomchristie Added bulk create tests deb5e65
@tomchristie Merge master 9bf7c9b
@tomchristie Merge master fe82982
@tomchristie Tweaking 9cdf841
@tomchristie Defer the writable nested modelserializers work addf7e9
@tomchristie tomchristie merged commit 9fdb661 into master
@tomchristie tomchristie deleted the clean-basic-nested-serialization branch
@alanjds

1) Is not this "all" test leaking Django ORM stuff at BaseSerializer, when should be at ModelSerializer ?
2) Asking because I need to filter a queryset to ModelSerializer used as a field, by something like this:

class CarSerializer(serializers.ModelSerializer):
    manufacturer = serializers.Field()
    model = serializers.Field()
    class Meta:
        model = Car

def _filter_manufacturer(queryset, instance):
    manufacturer = instance.context['request'].DATA.get('car_manufacturer', [])
    if manufacturer:
        return queryset.filter(manufacturer__in=manufacturer)

class CarOwnerSerializer(serializers.ModelSerializer):
    car_set = CarSerializer(read_only=True, filter_queryset=_filter_manufacturer)

So I would implement filter_queryset at ModelSerializer.__init__() but this would hit BaseSerializer code for something related to Django ORM.

3) Is this a good idea, or have you something cleaner in mind?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 12, 2013
Commits on Mar 14, 2013
  1. @maspwr
Commits on Mar 15, 2013
Commits on Mar 16, 2013
  1. Fixes to save_object

    authored
  2. Fixes to save_object

    authored
  3. Merge pull request #735 from tomchristie/one-to-one-nested-wip

    authored
    One to one writable nested model serializers (wip)
Commits on Mar 18, 2013
  1. @craigds
  2. @craigds
  3. @craigds
  4. Merge pull request #739 from craigds/basic-nested-serialization

    authored
    tweaks to writable nested serializers
  5. Added bulk create tests

    authored
Commits on Mar 22, 2013
  1. Merge master

    authored
  2. Merge master

    authored
  3. Tweaking

    authored
This page is out of date. Refresh to see the latest.
Showing with 351 additions and 39 deletions.
  1. +105 −39 rest_framework/serializers.py
  2. +246 −0 rest_framework/tests/serializer_nested.py
View
144 rest_framework/serializers.py
@@ -20,6 +20,25 @@
from rest_framework.fields import *
+class NestedValidationError(ValidationError):
+ """
+ The default ValidationError behavior is to stringify each item in the list
+ if the messages are a list of error messages.
+
+ In the case of nested serializers, where the parent has many children,
+ then the child's `serializer.errors` will be a list of dicts. In the case
+ of a single child, the `serializer.errors` will be a dict.
+
+ We need to override the default behavior to get properly nested error dicts.
+ """
+
+ def __init__(self, message):
+ if isinstance(message, dict):
+ self.messages = [message]
+ else:
+ self.messages = message
+
+
class DictWithMetadata(dict):
"""
A dict-like object, that can have additional properties attached.
@@ -98,7 +117,7 @@ def __init__(self, meta):
self.exclude = getattr(meta, 'exclude', ())
-class BaseSerializer(Field):
+class BaseSerializer(WritableField):
"""
This is the Serializer implementation.
We need to implement it as `BaseSerializer` due to metaclass magicks.
@@ -110,9 +129,9 @@ class Meta(object):
_dict_class = SortedDictWithMetadata
def __init__(self, instance=None, data=None, files=None,
- context=None, partial=False, many=None, source=None,
- allow_delete=False):
- super(BaseSerializer, self).__init__(source=source)
+ context=None, partial=False, many=None,
+ allow_delete=False, **kwargs):
+ super(BaseSerializer, self).__init__(**kwargs)
self.opts = self._options_class(self.Meta)
self.parent = None
self.root = None
@@ -305,40 +324,77 @@ def from_native(self, data, files):
def field_to_native(self, obj, field_name):
"""
- Override default so that we can apply ModelSerializer as a nested
- field to relationships.
+ Override default so that the serializer can be used as a nested field
+ across relationships.
"""
if self.source == '*':
return self.to_native(obj)
try:
- if self.source:
- for component in self.source.split('.'):
- obj = getattr(obj, component)
- if is_simple_callable(obj):
- obj = obj()
- else:
- obj = getattr(obj, field_name)
- if is_simple_callable(obj):
- obj = obj()
+ source = self.source or field_name
+ value = obj
+
+ for component in source.split('.'):
+ value = get_component(value, component)
+ if value is None:
+ break
except ObjectDoesNotExist:
return None
- # If the object has an "all" method, assume it's a relationship
- if is_simple_callable(getattr(obj, 'all', None)):
- return [self.to_native(item) for item in obj.all()]
+ if is_simple_callable(getattr(value, 'all', None)):
+ return [self.to_native(item) for item in value.all()]
- if obj is None:
+ if value is None:
return None
if self.many is not None:
many = self.many
else:
- many = hasattr(obj, '__iter__') and not isinstance(obj, (Page, dict, six.text_type))
+ many = hasattr(value, '__iter__') and not isinstance(value, (Page, dict, six.text_type))
if many:
- return [self.to_native(item) for item in obj]
- return self.to_native(obj)
+ return [self.to_native(item) for item in value]
+ return self.to_native(value)
+
+ def field_from_native(self, data, files, field_name, into):
+ """
+ Override default so that the serializer can be used as a writable
+ nested field across relationships.
+ """
+ if self.read_only:
+ return
+
+ try:
+ value = data[field_name]
+ except KeyError:
+ if self.default is not None and not self.partial:
+ # Note: partial updates shouldn't set defaults
+ value = copy.deepcopy(self.default)
+ else:
+ if self.required:
+ raise ValidationError(self.error_messages['required'])
+ return
+
+ # Set the serializer object if it exists
+ obj = getattr(self.parent.object, field_name) if self.parent.object else None
+
+ if value in (None, ''):
+ into[(self.source or field_name)] = None
+ else:
+ kwargs = {
+ 'instance': obj,
+ 'data': value,
+ 'context': self.context,
+ 'partial': self.partial,
+ 'many': self.many
+ }
+ serializer = self.__class__(**kwargs)
+
+ if serializer.is_valid():
+ into[self.source or field_name] = serializer.object
+ else:
+ # Propagate errors up to our parent
+ raise NestedValidationError(serializer.errors)
def get_identity(self, data):
"""
@@ -637,33 +693,43 @@ def restore_object(self, attrs, instance=None):
"""
Restore the model instance.
"""
- self.m2m_data = {}
- self.related_data = {}
+ m2m_data = {}
+ related_data = {}
+ meta = self.opts.model._meta
- # Reverse fk relations
- for (obj, model) in self.opts.model._meta.get_all_related_objects_with_model():
+ # Reverse fk or one-to-one relations
+ for (obj, model) in meta.get_all_related_objects_with_model():
field_name = obj.field.related_query_name()
if field_name in attrs:
- self.related_data[field_name] = attrs.pop(field_name)
+ related_data[field_name] = attrs.pop(field_name)
# Reverse m2m relations
- for (obj, model) in self.opts.model._meta.get_all_related_m2m_objects_with_model():
+ for (obj, model) in meta.get_all_related_m2m_objects_with_model():
field_name = obj.field.related_query_name()
if field_name in attrs:
- self.m2m_data[field_name] = attrs.pop(field_name)
+ m2m_data[field_name] = attrs.pop(field_name)
# Forward m2m relations
- for field in self.opts.model._meta.many_to_many:
+ for field in meta.many_to_many:
if field.name in attrs:
- self.m2m_data[field.name] = attrs.pop(field.name)
+ m2m_data[field.name] = attrs.pop(field.name)
+ # Update an existing instance...
if instance is not None:
for key, val in attrs.items():
setattr(instance, key, val)
+ # ...or create a new instance
else:
instance = self.opts.model(**attrs)
+ # Any relations that cannot be set until we've
+ # saved the model get hidden away on these
+ # private attributes, so we can deal with them
+ # at the point of save.
+ instance._related_data = related_data
+ instance._m2m_data = m2m_data
+
return instance
def from_native(self, data, files):
@@ -680,15 +746,15 @@ def save_object(self, obj, **kwargs):
"""
obj.save(**kwargs)
- if getattr(self, 'm2m_data', None):
- for accessor_name, object_list in self.m2m_data.items():
- setattr(self.object, accessor_name, object_list)
- self.m2m_data = {}
+ if getattr(obj, '_m2m_data', None):
+ for accessor_name, object_list in obj._m2m_data.items():
+ setattr(obj, accessor_name, object_list)
+ del(obj._m2m_data)
- if getattr(self, 'related_data', None):
- for accessor_name, object_list in self.related_data.items():
- setattr(self.object, accessor_name, object_list)
- self.related_data = {}
+ if getattr(obj, '_related_data', None):
+ for accessor_name, related in obj._related_data.items():
+ setattr(obj, accessor_name, related)
+ del(obj._related_data)
class HyperlinkedModelSerializerOptions(ModelSerializerOptions):
View
246 rest_framework/tests/serializer_nested.py
@@ -0,0 +1,246 @@
+"""
+Tests to cover nested serializers.
+
+Doesn't cover model serializers.
+"""
+from __future__ import unicode_literals
+from django.test import TestCase
+from rest_framework import serializers
+
+
+class WritableNestedSerializerBasicTests(TestCase):
+ """
+ Tests for deserializing nested entities.
+ Basic tests that use serializers that simply restore to dicts.
+ """
+
+ def setUp(self):
+ class TrackSerializer(serializers.Serializer):
+ order = serializers.IntegerField()
+ title = serializers.CharField(max_length=100)
+ duration = serializers.IntegerField()
+
+ class AlbumSerializer(serializers.Serializer):
+ album_name = serializers.CharField(max_length=100)
+ artist = serializers.CharField(max_length=100)
+ tracks = TrackSerializer(many=True)
+
+ self.AlbumSerializer = AlbumSerializer
+
+ def test_nested_validation_success(self):
+ """
+ Correct nested serialization should return the input data.
+ """
+
+ data = {
+ 'album_name': 'Discovery',
+ 'artist': 'Daft Punk',
+ 'tracks': [
+ {'order': 1, 'title': 'One More Time', 'duration': 235},
+ {'order': 2, 'title': 'Aerodynamic', 'duration': 184},
+ {'order': 3, 'title': 'Digital Love', 'duration': 239}
+ ]
+ }
+
+ serializer = self.AlbumSerializer(data=data)
+ self.assertEqual(serializer.is_valid(), True)
+ self.assertEqual(serializer.object, data)
+
+ def test_nested_validation_error(self):
+ """
+ Incorrect nested serialization should return appropriate error data.
+ """
+
+ data = {
+ 'album_name': 'Discovery',
+ 'artist': 'Daft Punk',
+ 'tracks': [
+ {'order': 1, 'title': 'One More Time', 'duration': 235},
+ {'order': 2, 'title': 'Aerodynamic', 'duration': 184},
+ {'order': 3, 'title': 'Digital Love', 'duration': 'foobar'}
+ ]
+ }
+ expected_errors = {
+ 'tracks': [
+ {},
+ {},
+ {'duration': ['Enter a whole number.']}
+ ]
+ }
+
+ serializer = self.AlbumSerializer(data=data)
+ self.assertEqual(serializer.is_valid(), False)
+ self.assertEqual(serializer.errors, expected_errors)
+
+ def test_many_nested_validation_error(self):
+ """
+ Incorrect nested serialization should return appropriate error data
+ when multiple entities are being deserialized.
+ """
+
+ data = [
+ {
+ 'album_name': 'Russian Red',
+ 'artist': 'I Love Your Glasses',
+ 'tracks': [
+ {'order': 1, 'title': 'Cigarettes', 'duration': 121},
+ {'order': 2, 'title': 'No Past Land', 'duration': 198},
+ {'order': 3, 'title': 'They Don\'t Believe', 'duration': 191}
+ ]
+ },
+ {
+ 'album_name': 'Discovery',
+ 'artist': 'Daft Punk',
+ 'tracks': [
+ {'order': 1, 'title': 'One More Time', 'duration': 235},
+ {'order': 2, 'title': 'Aerodynamic', 'duration': 184},
+ {'order': 3, 'title': 'Digital Love', 'duration': 'foobar'}
+ ]
+ }
+ ]
+ expected_errors = [
+ {},
+ {
+ 'tracks': [
+ {},
+ {},
+ {'duration': ['Enter a whole number.']}
+ ]
+ }
+ ]
+
+ serializer = self.AlbumSerializer(data=data)
+ self.assertEqual(serializer.is_valid(), False)
+ self.assertEqual(serializer.errors, expected_errors)
+
+
+class WritableNestedSerializerObjectTests(TestCase):
+ """
+ Tests for deserializing nested entities.
+ These tests use serializers that restore to concrete objects.
+ """
+
+ def setUp(self):
+ # Couple of concrete objects that we're going to deserialize into
+ class Track(object):
+ def __init__(self, order, title, duration):
+ self.order, self.title, self.duration = order, title, duration
+
+ def __eq__(self, other):
+ return (
+ self.order == other.order and
+ self.title == other.title and
+ self.duration == other.duration
+ )
+
+ class Album(object):
+ def __init__(self, album_name, artist, tracks):
+ self.album_name, self.artist, self.tracks = album_name, artist, tracks
+
+ def __eq__(self, other):
+ return (
+ self.album_name == other.album_name and
+ self.artist == other.artist and
+ self.tracks == other.tracks
+ )
+
+ # And their corresponding serializers
+ class TrackSerializer(serializers.Serializer):
+ order = serializers.IntegerField()
+ title = serializers.CharField(max_length=100)
+ duration = serializers.IntegerField()
+
+ def restore_object(self, attrs, instance=None):
+ return Track(attrs['order'], attrs['title'], attrs['duration'])
+
+ class AlbumSerializer(serializers.Serializer):
+ album_name = serializers.CharField(max_length=100)
+ artist = serializers.CharField(max_length=100)
+ tracks = TrackSerializer(many=True)
+
+ def restore_object(self, attrs, instance=None):
+ return Album(attrs['album_name'], attrs['artist'], attrs['tracks'])
+
+ self.Album, self.Track = Album, Track
+ self.AlbumSerializer = AlbumSerializer
+
+ def test_nested_validation_success(self):
+ """
+ Correct nested serialization should return a restored object
+ that corresponds to the input data.
+ """
+
+ data = {
+ 'album_name': 'Discovery',
+ 'artist': 'Daft Punk',
+ 'tracks': [
+ {'order': 1, 'title': 'One More Time', 'duration': 235},
+ {'order': 2, 'title': 'Aerodynamic', 'duration': 184},
+ {'order': 3, 'title': 'Digital Love', 'duration': 239}
+ ]
+ }
+ expected_object = self.Album(
+ album_name='Discovery',
+ artist='Daft Punk',
+ tracks=[
+ self.Track(order=1, title='One More Time', duration=235),
+ self.Track(order=2, title='Aerodynamic', duration=184),
+ self.Track(order=3, title='Digital Love', duration=239),
+ ]
+ )
+
+ serializer = self.AlbumSerializer(data=data)
+ self.assertEqual(serializer.is_valid(), True)
+ self.assertEqual(serializer.object, expected_object)
+
+ def test_many_nested_validation_success(self):
+ """
+ Correct nested serialization should return multiple restored objects
+ that corresponds to the input data when multiple objects are
+ being deserialized.
+ """
+
+ data = [
+ {
+ 'album_name': 'Russian Red',
+ 'artist': 'I Love Your Glasses',
+ 'tracks': [
+ {'order': 1, 'title': 'Cigarettes', 'duration': 121},
+ {'order': 2, 'title': 'No Past Land', 'duration': 198},
+ {'order': 3, 'title': 'They Don\'t Believe', 'duration': 191}
+ ]
+ },
+ {
+ 'album_name': 'Discovery',
+ 'artist': 'Daft Punk',
+ 'tracks': [
+ {'order': 1, 'title': 'One More Time', 'duration': 235},
+ {'order': 2, 'title': 'Aerodynamic', 'duration': 184},
+ {'order': 3, 'title': 'Digital Love', 'duration': 239}
+ ]
+ }
+ ]
+ expected_object = [
+ self.Album(
+ album_name='Russian Red',
+ artist='I Love Your Glasses',
+ tracks=[
+ self.Track(order=1, title='Cigarettes', duration=121),
+ self.Track(order=2, title='No Past Land', duration=198),
+ self.Track(order=3, title='They Don\'t Believe', duration=191),
+ ]
+ ),
+ self.Album(
+ album_name='Discovery',
+ artist='Daft Punk',
+ tracks=[
+ self.Track(order=1, title='One More Time', duration=235),
+ self.Track(order=2, title='Aerodynamic', duration=184),
+ self.Track(order=3, title='Digital Love', duration=239),
+ ]
+ )
+ ]
+
+ serializer = self.AlbumSerializer(data=data)
+ self.assertEqual(serializer.is_valid(), True)
+ self.assertEqual(serializer.object, expected_object)
Something went wrong with that request. Please try again.