New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support custom serialization function #167

Closed
klinkin opened this Issue Mar 19, 2013 · 15 comments

Comments

Projects
None yet
7 participants
@klinkin
Contributor

klinkin commented Mar 19, 2013

Hi Jeffrey

In #139 you are remove custom code for specifying include and exclude columns,
after that it is very inconvenient to include/exclude columns in postprocessor function (particularly in GET_MANY postprocessor).

My suggestion is add new parameter "serialization_function" to create_api_blueprint and if this param is not None call this function instead of the _to_dict

@klinkin

This comment has been minimized.

Contributor

klinkin commented Apr 11, 2013

In 6af2051 include/exclude columns bring back.

@jfinkels

This comment has been minimized.

Owner

jfinkels commented Feb 16, 2015

Since someone else created a new issue, #407, which is essentially the same as this, I'd like to restart the conversation on this issue.

I think this is a good idea now. Now that the code is more mature, this should be easy to implement, and should be made available as soon as possible.

@taion

This comment has been minimized.

taion commented Feb 16, 2015

I think one question is the right level of abstraction. I think something like a marshmallow schema that is specifically for serializing and deserializing objects is probably the right level to work at and will limit unnecessarily re-inventing the wheel or doing something hacky here that would better be done elsewhere.

@jfinkels

This comment has been minimized.

Owner

jfinkels commented Feb 16, 2015

Currently, the way a model gets to the client is like this:

model -> to_dict() function -> mimerender -> jsonpify() function -> client

Your suggestion is to replace the to_dict part of this sequence, right?

model -> custom serialization function -> mimerender -> jsonpify() function -> client

I don't see why Flask-Restless should know anything about the implementation of the custom serialization function, whether Marshmallow or otherwise.

@taion

This comment has been minimized.

taion commented Feb 16, 2015

One example where this comes up is actually in the reverse operation. If I have a custom serializer, it would be best if it were as easy as possible to set things up such that deserialization were guaranteed to behave as the inverse.

While it's possible to do so using arbitrary serialization and deserialization methods, typically it's going to be much more maintainable to use something like a marshmallow schema to do this. DRF uses this sort of framework: http://www.django-rest-framework.org/api-guide/serializers/

Custom serialization/deserialization functions can be set up to call into any of these as needed, but the benefit of the structured approach is that it makes it easier to implement things in a way that is more correct and maintainable for users of the library.

@jfinkels

This comment has been minimized.

Owner

jfinkels commented Feb 16, 2015

I'm still not following. Why can't we just do something like this:

# Create the model class and the serialization schema; Flask-Restless doesn't need to know.
class MyModel: ...
class MySchema: ...

# Assuming the schema has a serialize() and deserialize() function...
apimanager.create_api(MyModel, serializer=MySchema.serialize, deserializer=MySchema.deserialize)

I'm trying to avoid at all costs coupling Flask-Restless with a particular serizalization framework or style.

@taion

This comment has been minimized.

taion commented Feb 16, 2015

Fair enough - this may just be a matter of documentation, then. I think it's an anti-pattern to actually write a serializer as opposed to using a library that lets you define a schema, but how you connect it to the API can be flexible.

@jfinkels

This comment has been minimized.

Owner

jfinkels commented Feb 16, 2015

I think it's an anti-pattern to actually write a serializer as opposed to using a library that lets you define a schema [...]

I agree, but I think that's beyond the scope of Flask-Restless. My serialization function to_dict() is just giving a best-effort attempt at interpreting a pure SQLAlchemy model without any external information. Getting some examples in the documentation (once this feature is implemented) and putting some strong suggestions in there is a must.

It seems very aggressive to require users to provide not only a SQLAlchemy model, but also a Marshmallow schema. However, it would certainly make the world a safer and more structured place :).

@taion

This comment has been minimized.

taion commented Feb 16, 2015

Makes sense. I think the existing to_dict() is sufficient for the majority of use cases, and is a lot easier to get started with than defining a full schema. It's just a question of how to handle custom serialization of fields. I don't think anybody's going to make the mistake of trying to write their own custom serializer and deserializer if the documentation points them in the right direction.

@jfinkels jfinkels closed this in b736d28 Feb 18, 2015

@spight

This comment has been minimized.

spight commented Sep 8, 2015

As far as I can tell, GET_MANY operations don't respect the serializer function. Is this by design, or did it slip through the cracks?

@itsrifat

This comment has been minimized.

itsrifat commented Sep 22, 2015

@spight I'm having the same problem too. I'm using a postprocessor for now and modifying the result['objects'] there applying the deserializer.

person_schema = PersonSchema()

def person_deserializer(data):
    return person_schema.load(data).data

def person_after_get_many(result=None, search_params=None, **kw):
  result['objects'] = [person_deserializer(obj) for obj in result['objects']]

apimanager.create_api(Person, 
    methods=['GET', 'POST','PUT', 'DELETE'],
    postprocessors={
        'GET_MANY':[person_after_get_many]
    }
)
@mgberlin

This comment has been minimized.

mgberlin commented Sep 27, 2015

+1 to the question by @spight. Seems odd that the serializer is run for GET_SINGLE but not GET_MANY.

@spight

This comment has been minimized.

spight commented Oct 6, 2015

@itsrifat I'm doing something similar to that as well, but with Marshmallow

result['object'] = Schema.dump(result['objects'], many=True)

Unfortunately, this still has its limitations since to_dict alters types in some cases, (Example: DateTime objects becomes strings). I might try my hand at making a patch for this particular issue, but the project looks like it's been abandoned for a few months....

@spight

This comment has been minimized.

spight commented Oct 7, 2015

For posterity, another issue I ran into is that the default serializer was mangling the data presented to my Marshmallow serializers beyond usability. I ended up having to monkey patch the flask.ext.restless.helpers.to_dict function.

# Somewhere in my init code
from flask.ext.restless import views
def to_dict_hack(x, *args, **kwargs):
    return x
views.to_dict = to_dict_hack

Then I was able to apply @itsrifat's solution since the objects were still instances of my model.

@kopf

This comment has been minimized.

Contributor

kopf commented Jan 29, 2016

EDIT - Removed this comment as I see flask-restless works as expected in 1.0.0-dev.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment