New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Serializer] Injecting additional data during serialization #18904

Open
simshaun opened this Issue May 28, 2016 · 18 comments

Comments

Projects
None yet
8 participants
@simshaun
Contributor

simshaun commented May 28, 2016

I frequently have serialization use cases where the frontend needs more information about each entity than is available in the entities themselves. Just one example is using the router to provide URLs for each entry when serializing a collection.

From what I can see, there doesn't appear to be an easy way to do that. I propose allowing the optional use of an event dispatcher during the normalization process to modify the data that gets encoded.

@iltar

This comment has been minimized.

Show comment
Hide comment
@iltar

iltar May 28, 2016

Contributor

I frequently have serialization use cases where the frontend needs more information about each entity than is available in the entities themselves

What about serializing view/value objects instead? Imo it's not the best idea to expose your internal structure to the outside world. If you treat your entity as a simple typed array, why not just serialize an array?

Contributor

iltar commented May 28, 2016

I frequently have serialization use cases where the frontend needs more information about each entity than is available in the entities themselves

What about serializing view/value objects instead? Imo it's not the best idea to expose your internal structure to the outside world. If you treat your entity as a simple typed array, why not just serialize an array?

@ogizanagi

This comment has been minimized.

Show comment
Hide comment
@ogizanagi

ogizanagi May 29, 2016

Member

Why not creating your own normalizer in which you could inject the router ?

Member

ogizanagi commented May 29, 2016

Why not creating your own normalizer in which you could inject the router ?

@dunglas

This comment has been minimized.

Show comment
Hide comment
@dunglas

dunglas May 31, 2016

Member

@ogizanagi is right, creating a custom normalizer is the way to go (it's what we do in API Platform: https://github.com/api-platform/core/blob/master/src/JsonLd/Serializer/ItemNormalizer.php)

Member

dunglas commented May 31, 2016

@ogizanagi is right, creating a custom normalizer is the way to go (it's what we do in API Platform: https://github.com/api-platform/core/blob/master/src/JsonLd/Serializer/ItemNormalizer.php)

@simshaun

This comment has been minimized.

Show comment
Hide comment
@simshaun

simshaun May 31, 2016

Contributor

@ogizanagi @dunglas
Creating a custom normalizer works in this case, but I'm not fully convinced it's always the best solution. My main concern is being constrained to a single normalizer per object + format. That 1 normalizer has to know about every serialization use-case I may have for a class.

I think @iltar's recommendation of creating serializable view objects is the most flexible, but I'm wrestling with a tradeoff of convenience.

For example, let's pretend I have a Case entity object with associated Customer and many CaseItems, and I'm building a report that needs:

  • A few pieces of Case
  • A few pieces of Customer
  • A URL to view Customer
  • A few pieces of each CaseItem
  • A URL for each CaseItem

For this single use-case, I need:

  1. Create 3 use-case-specific view classes (CaseView, CustomerView, and CaseItemView) that have only the desired properties from each entity.
  2. Generate the needed URLs at some point (Inject router into CustomerView and CaseItemView constructor maybe?)
  3. Build a collection of CaseItemView objects based on the collection in Case entity.
  4. Build a CustomerView object from the Customer entity.
  5. Build a CaseView object from the Case entity.
  6. Store CaseItemView collection and CustomerView in the CaseView object.
  7. Serialize CaseView.

That gives the consumer access to:

  • case.id and case.....
  • case.customer.name, case.customer......, and case.customer.url
  • case.items.0.name, case.items.0......., and case.items.0.url
  • case.items.1.name, case.items.1......., and case.items.1.url

Am I off base? Over-complicating it? If so, how? It seems clean, but not very convenient when I could instead just use serializer groups on the entities and (if I had my way) build a couple simple listeners that append the URL to the normalized data when Customer or CaseItem are being normalized in a certain context.

Contributor

simshaun commented May 31, 2016

@ogizanagi @dunglas
Creating a custom normalizer works in this case, but I'm not fully convinced it's always the best solution. My main concern is being constrained to a single normalizer per object + format. That 1 normalizer has to know about every serialization use-case I may have for a class.

I think @iltar's recommendation of creating serializable view objects is the most flexible, but I'm wrestling with a tradeoff of convenience.

For example, let's pretend I have a Case entity object with associated Customer and many CaseItems, and I'm building a report that needs:

  • A few pieces of Case
  • A few pieces of Customer
  • A URL to view Customer
  • A few pieces of each CaseItem
  • A URL for each CaseItem

For this single use-case, I need:

  1. Create 3 use-case-specific view classes (CaseView, CustomerView, and CaseItemView) that have only the desired properties from each entity.
  2. Generate the needed URLs at some point (Inject router into CustomerView and CaseItemView constructor maybe?)
  3. Build a collection of CaseItemView objects based on the collection in Case entity.
  4. Build a CustomerView object from the Customer entity.
  5. Build a CaseView object from the Case entity.
  6. Store CaseItemView collection and CustomerView in the CaseView object.
  7. Serialize CaseView.

That gives the consumer access to:

  • case.id and case.....
  • case.customer.name, case.customer......, and case.customer.url
  • case.items.0.name, case.items.0......., and case.items.0.url
  • case.items.1.name, case.items.1......., and case.items.1.url

Am I off base? Over-complicating it? If so, how? It seems clean, but not very convenient when I could instead just use serializer groups on the entities and (if I had my way) build a couple simple listeners that append the URL to the normalized data when Customer or CaseItem are being normalized in a certain context.

@ogizanagi

This comment has been minimized.

Show comment
Hide comment
@ogizanagi

ogizanagi Jun 1, 2016

Member

@simshaun: This is not over-complicating it: we're quite doing the same thing in a "pseudo-DDD" approach in our application for any data we pass to our templates. We build DTO objects called "Views" and assembled by an "Assembler".
But in many context of serialization, IMHO, the normalizer is almost the same thing as the assembler (except it will not return DTO objects, but simple arrays).

Creating a custom normalizer works in this case, but I'm not fully convinced it's always the best solution. My main concern is being constrained to a single normalizer per object + format. That 1 normalizer has to know about every serialization use-case I may have for a class.

Create your normalizer with a custom format. (Not just json).
Then only call normalize from the serializer and encode it to json only after (Or create a dedicated encoder registered in the serializer). Thus, you can have 1 format per use case.

You can also use the context and delegate to sub-normalizers once in the normalize method.

Member

ogizanagi commented Jun 1, 2016

@simshaun: This is not over-complicating it: we're quite doing the same thing in a "pseudo-DDD" approach in our application for any data we pass to our templates. We build DTO objects called "Views" and assembled by an "Assembler".
But in many context of serialization, IMHO, the normalizer is almost the same thing as the assembler (except it will not return DTO objects, but simple arrays).

Creating a custom normalizer works in this case, but I'm not fully convinced it's always the best solution. My main concern is being constrained to a single normalizer per object + format. That 1 normalizer has to know about every serialization use-case I may have for a class.

Create your normalizer with a custom format. (Not just json).
Then only call normalize from the serializer and encode it to json only after (Or create a dedicated encoder registered in the serializer). Thus, you can have 1 format per use case.

You can also use the context and delegate to sub-normalizers once in the normalize method.

@simshaun

This comment has been minimized.

Show comment
Hide comment
@simshaun

simshaun Aug 28, 2016

Contributor

Finally revisiting this...

Create your normalizer with a custom format. (Not just json). ... Thus, you can have 1 format per use case.

That seems hacky (not really an intended use of format.)

You can also use the context and delegate to sub-normalizers once in the normalize method.

The problem I have with this is that the context would need to know about every serialization use-case.

Contributor

simshaun commented Aug 28, 2016

Finally revisiting this...

Create your normalizer with a custom format. (Not just json). ... Thus, you can have 1 format per use case.

That seems hacky (not really an intended use of format.)

You can also use the context and delegate to sub-normalizers once in the normalize method.

The problem I have with this is that the context would need to know about every serialization use-case.

@theofidry

This comment has been minimized.

Show comment
Hide comment
@theofidry

theofidry Aug 28, 2016

Contributor

That seems hacky (not really an intended use of format.)

It is, format is just here for your encoder/decoders.

Your use case is relatively specific. If may require a bit of tinkering and customization on top of the Symfony Serializer, but I don't think it's something that should be done in the Serializer core.

Contributor

theofidry commented Aug 28, 2016

That seems hacky (not really an intended use of format.)

It is, format is just here for your encoder/decoders.

Your use case is relatively specific. If may require a bit of tinkering and customization on top of the Symfony Serializer, but I don't think it's something that should be done in the Serializer core.

@ogizanagi

This comment has been minimized.

Show comment
Hide comment
@ogizanagi

ogizanagi Aug 28, 2016

Member

That seems hacky (not really an intended use of format.)

It is, format is just here for your encoder/decoders.

You're right. But, the process of normalization is just about transforming complex data (objects) into simpler data (array & scalars). This transformation may lead to different representations according to your use case. Those representations can easily be identified through the format specified when normalizing your data. I do think it's the most straightforward usage in an application.

But fine, if you don't want to use the format, use the context. Simply register a normalizer supporting your object class and the expected format (json for instance). Then, inject a map of sub-normalizers (as value) to handle the normalization according to the use case (the key) specified into the context.
Same result, more work, but respectful of the serializer interfaces 😜

Otherwise, you can still assemble the expected data in a view DTO first, and serialize it directly (yes, again, it needs more work, but it also brings visibility).

Member

ogizanagi commented Aug 28, 2016

That seems hacky (not really an intended use of format.)

It is, format is just here for your encoder/decoders.

You're right. But, the process of normalization is just about transforming complex data (objects) into simpler data (array & scalars). This transformation may lead to different representations according to your use case. Those representations can easily be identified through the format specified when normalizing your data. I do think it's the most straightforward usage in an application.

But fine, if you don't want to use the format, use the context. Simply register a normalizer supporting your object class and the expected format (json for instance). Then, inject a map of sub-normalizers (as value) to handle the normalization according to the use case (the key) specified into the context.
Same result, more work, but respectful of the serializer interfaces 😜

Otherwise, you can still assemble the expected data in a view DTO first, and serialize it directly (yes, again, it needs more work, but it also brings visibility).

@ogizanagi

This comment has been minimized.

Show comment
Hide comment
@ogizanagi

ogizanagi Aug 28, 2016

Member

Also #19371 would simplify this by allowing you to create directly one normalizer per use case, by accessing the $context in the supportsNormalization method.
Not being able to access the $context in this method is the reason why the $format argument was so handy for such use cases.

Member

ogizanagi commented Aug 28, 2016

Also #19371 would simplify this by allowing you to create directly one normalizer per use case, by accessing the $context in the supportsNormalization method.
Not being able to access the $context in this method is the reason why the $format argument was so handy for such use cases.

@simshaun

This comment has been minimized.

Show comment
Hide comment
@simshaun

simshaun Aug 29, 2016

Contributor

Your use case is relatively specific.

I disagree. I think it's fairly common (at least in everything I've done in the past few years) to want to add additional data, like URLs, when an object is being serialized. View objects/DTOs solve the problem, but they're not very convenient to create when I've got a tree that's several levels deep and some of the entities have quite a few properties.


What I've done in the meantime is create a custom normalizer that calls a couple user-defined callbacks from the $context array.

  • Callback 1 lets me explicitly declare the attributes I want from each class being serialized. (or I can just use the groups annotation).
  • Callback 2 lets me add additional data to each class being serialized.

I don't know if that's good or bad, but it works.

Contributor

simshaun commented Aug 29, 2016

Your use case is relatively specific.

I disagree. I think it's fairly common (at least in everything I've done in the past few years) to want to add additional data, like URLs, when an object is being serialized. View objects/DTOs solve the problem, but they're not very convenient to create when I've got a tree that's several levels deep and some of the entities have quite a few properties.


What I've done in the meantime is create a custom normalizer that calls a couple user-defined callbacks from the $context array.

  • Callback 1 lets me explicitly declare the attributes I want from each class being serialized. (or I can just use the groups annotation).
  • Callback 2 lets me add additional data to each class being serialized.

I don't know if that's good or bad, but it works.

@psren

This comment has been minimized.

Show comment
Hide comment
@psren

psren Mar 16, 2018

Hey there.
I have a normalizer
serializer.normalizer
for
symfony/serializer
component.
In my output there should be some entity-related stuff, but not from the entity itself.
Injecting a Repository does not work with
Circular reference detected for service "doctrine.orm.default_entity_manager
.

Using
$context
on the
normalize
method doesn’t seem like a good idea.
(I always want to append the data, not in a specific context only).

I can't use the approach of @simshaun because the serialization in my case is done internally be the "algolia-bundle"

psren commented Mar 16, 2018

Hey there.
I have a normalizer
serializer.normalizer
for
symfony/serializer
component.
In my output there should be some entity-related stuff, but not from the entity itself.
Injecting a Repository does not work with
Circular reference detected for service "doctrine.orm.default_entity_manager
.

Using
$context
on the
normalize
method doesn’t seem like a good idea.
(I always want to append the data, not in a specific context only).

I can't use the approach of @simshaun because the serialization in my case is done internally be the "algolia-bundle"

@ogizanagi

This comment has been minimized.

Show comment
Hide comment
@ogizanagi

ogizanagi Mar 16, 2018

Member

Circular reference detected for service "doctrine.orm.default_entity_manager

Not sure why you get this circular ref here without looking at your code. But try injecting the doctrine manager registry instead Doctrine\Common\Persistence\ManagerRegistry $registry and get your repo using $registry->getRepository()

Member

ogizanagi commented Mar 16, 2018

Circular reference detected for service "doctrine.orm.default_entity_manager

Not sure why you get this circular ref here without looking at your code. But try injecting the doctrine manager registry instead Doctrine\Common\Persistence\ManagerRegistry $registry and get your repo using $registry->getRepository()

@psren

This comment has been minimized.

Show comment
Hide comment
@psren

psren Mar 16, 2018

Ah, thanks @ogizanagi that does the trick.

psren commented Mar 16, 2018

Ah, thanks @ogizanagi that does the trick.

@chiqui3d

This comment has been minimized.

Show comment
Hide comment
@chiqui3d

chiqui3d May 28, 2018

Still can't add value and properties extras?

chiqui3d commented May 28, 2018

Still can't add value and properties extras?

@theofidry

This comment has been minimized.

Show comment
Hide comment
@theofidry

theofidry May 28, 2018

Contributor

@chiqui3d anything concrete to add? Like a use case and what you would like to see?

Contributor

theofidry commented May 28, 2018

@chiqui3d anything concrete to add? Like a use case and what you would like to see?

@chiqui3d

This comment has been minimized.

Show comment
Hide comment
@chiqui3d

chiqui3d May 28, 2018

Yes @theofidry, for each object I want to create a link/path like property in the serialization based on the entity ID,

thanks

chiqui3d commented May 28, 2018

Yes @theofidry, for each object I want to create a link/path like property in the serialization based on the entity ID,

thanks

@theofidry

This comment has been minimized.

Show comment
Hide comment
@theofidry

theofidry May 28, 2018

Contributor

You can already do that easily with a custom format or normalizer as per the conversation above

Contributor

theofidry commented May 28, 2018

You can already do that easily with a custom format or normalizer as per the conversation above

@chiqui3d

This comment has been minimized.

Show comment
Hide comment
@chiqui3d

chiqui3d May 28, 2018

Sorry for delay @theofidry

Okay, I just realized that in the entity I can create a virtual getter for the URL and access the data. This is easier than creating a custom normalizer.

btw, I don't understand is when I create a calback for a particular field, I can't access the others values only own value :(

chiqui3d commented May 28, 2018

Sorry for delay @theofidry

Okay, I just realized that in the entity I can create a virtual getter for the URL and access the data. This is easier than creating a custom normalizer.

btw, I don't understand is when I create a calback for a particular field, I can't access the others values only own value :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment