Skip to content
This repository

Need a consensus on DB mapper / repository / hydrator / entity #25

Closed
EvanDotPro opened this Issue · 40 comments
Evan Coury
Owner

I'm starting this issue to give us a central location to make a documented decision on this stuff. I'll update this OP as we get everything defined.


  • Entity (aka Model or Domain Model) - There should be no required abstract entity. These should just be normal classes with properties, getters, and setters. (@prolic has pending PR's on ZfcUser and ZfcBase to accomplish the goal of no abstract entity.).
<?php

namespace FooModule\Entity;

use DateTime;

class BlogPost
{
    /**
     * Title of the blog post
     * 
     * @var string
     */
    protected $title;

    /**
     * Body of the blog post
     * 
     * @var string
     */
    protected $body;

    /**
     * Date/time that the post was published
     * 
     * @var DateTime
     */
    protected $date;

    /**
     * Get title of the blog post
     *
     * @return string
     */
    public function getTitle()
    {
        return $this->title;
    }

    /**
     * Set the title of the blog post
     *
     * @param sring $title
     */
    public function setTitle($title)
    {
        $this->title = $title;
        return $this;
    }

    /**
     * Get the body of the blog post
     *
     * @return string
     */
    public function getBody()
    {
        return $this->body;
    }

    /**
     * Set the body of the blog post
     *
     * @param string $body
     */
    public function setBody($body)
    {
        $this->body = $body;
        return $this;
    }

    /**
     * Get the date/time the blog post was published
     *
     * @return DateTime
     */
    public function getDate()
    {
        return $this->date;
    }

    /**
     * Set the date/time the blog post was published
     *
     * @param DateTime $body
     */
    public function setDate(DateTime $date)
    {
        $this->date = $date;
        return $this;
    }
}

  • Hydrator - The hydrator is responsible for the conversion between DB rows (arrays) and entity objects. This is already done for us in Zend\Stdlib\Hydrator, which provides us several different hydration strategies. Below is the standard hydrator interface from Zend\Stdlib\Hydrator\HydratorInterface.
    • Question: One important thing is decoupling mode/entity objects from database rows. Should the hydrator(s) allow for passing "mapping metadata" which contains propertyname => dbfieldname?
    • Question: Zend\Db returns rows as instances of Zend\Db\ResultSet\Row. There is a RowInterface, with the idea being that you can configure Zend\Db to use your own row object. Currently RowInterface simply defines public function populate(array $rowData); and extends \Countable and \ArrayAccess. Should Zend\Db be refactored to make use of Zend\Stdlib\Hydrator so that the result set itself can automatically return hydrated model/entity objects? (Done)
    • Question: Is this separate from the mapper, or could a hydrator, in a way, be considered a mapper? Do we need both?
<?php

namespace Zend\Stdlib\Hydrator;

interface HydratorInterface
{
    /**
     * Extract values from an object
     * 
     * @param  object $object 
     * @return array
     */
    public function extract($object);

    /**
     * Hydrate $object with the provided $data.
     * 
     * @param  array $data 
     * @param  object $object 
     * @return object
     */
    public function hydrate(array $data, $object);
}



Other Questions

  • Foreign keys and relationships? Should we do anything in ZfcBase to support them? If not, we should at least be able to provide an example of how something like this would work. An example provided by Bakura, was something like $user->getAddress()->getCountry()->getName(). Let me just say this: we will not be generating proxies and promoting bypassing the service layer like Doctrine. Period.

Concrete Decisions Made

  • We will use the term "entity" due to the ambiguity of the term "model".
  • We will not make use of magic methods.
Ben Youngblood

Since we can (and should!) use the hydrators in Zend\Stdlib, we obviously need to keep domain logic either in the model or the mapper. Say for example you have a DateTime field in your database and you need to transform it into a \Datetime or Zend\Date. If the model/entity is only responsible for storing and retrieving data, then this must fall to the mapper. The mapper "maps" (aha!) database fields (retrieved from the repository) to the proper PHP objects/scalars. This would leave the hydrator to simply facilitate the transfer of data between the model and the mapper.

Rob Allen
Owner

As I said on IRC recently, I would prefer to stay firmly in the data mapping side of ORM. i.e. Repository is not for ZfcBase.

Evan Coury
Owner

@Freeaqingme raised a good point on IRC, that HydratorInterface::hydrate(array $data, $object) probably shouldn't type hint an array. There may be cases where you're hydrating from non-array objects such as an instance of Zend\Db\ResultSet\Row or a \DOMDocument.

Matthew Weier O'Phinney
Owner

I'm fine with not using the array type-hint -- but that means all implementations then need to check for either array or Traversable, which is kind of a pain.

I personally feel Zend\Db should use hydrators to hydrate row objects -- I actually thought that was the plan for beta4 and was unpleasantly surprised to discover they weren't as I was developing my dpc12 workshop.

Ben Youngblood

@weierophinney I've been thinking about Zend\Db doing hydration, and I've already run into a bit of an issue. Say you have a DateTime field in your database and in your model you want that transformed into a Zend\Date.

a. How will the hydrator know to transform that field into a Zend\Date (or maybe a \DateTime or whatever you have)?
b. How will the hydrator know how to turn the above back into something the database understands when extracting?

Matthew Weier O'Phinney
Owner

@bjyoungblood That's exactly the sort of thing hydrators are supposed to solve -- I'd argue that if you have specialized hydration/extraction needs, you write a custom hydrator. The ones we provide in the framework should only be very, very generic.

prolic
prolic commented

One goal of ZfcUser is that you can use it with Zend\Db and optionally with Doctrine. Doctrine has the concept of repositories. If we don't want to have repositories in ZfcUser, the problem is how to offer a good architecture for doctrine integration.
My idea is, that a services uses mappers and repositories, both with distinct responsibilities and we offer a thin abstraction layer (see my open PR's at ZfcUser and ZfcUserDoctrineORM, additional work is still in progress), so you can easily replace these with the Doctrine ObjectManager (at least we should support ORM and ODM) and Doctrine Repositories.
On the other hand, if we only use mappers without repositories in default ZfcUser, we need to reimplement every single service, because Doctrine just behaves different. This will lead to lots of code duplication and result in bugfixing the same issue in ZfcUser, ZfcUserDoctrineORM, ZfcUserDoctrineMongoODM, and so on.
@akrabat noticed me, that the implementation I offer follows more the finder pattern, than the repository pattern. Therefore implementing a finder interface in the mapper class would be sufficient. I argue, that the interface I offered (again, take a look at my PR's, it's very easy and lightweight), can work with the original Doctrine repositories, and therefore I name them a repository. In "standard" ZfcUser, we use the interfaces of the repository, and simply implement them as easy as possible. Simple queries for a single table like "findAll()", "findBy(array $criteria, array $orderBy = null, $limit = null, $offset = null)" and "findOneBy(array $criteria)", could be easily implemented in an zfc base repository, leaving all other implementation details up to the user. Consider also: http://blog.fedecarg.com/2010/03/22/implementing-dynamic-finders-and-parsing-method-expressions/
This can be easily implemented on top of Zend\Db.

Please don't understand me wrong, my goal is not to reinvent another full featured data mapper like Doctrine but with Zend or ZfcBase namespace, but to offer an abstraction, so you really can replace things with Doctrine stuff. Just 2 simple interfaces:
1) https://github.com/ZF-Commons/ZfcBase/blob/master/src/ZfcBase/Repository/RepositoryInterface.php
2) https://github.com/ZF-Commons/ZfcBase/blob/master/src/ZfcBase/Mapper/DataMapperInterface.php
Don't consider them as final, I got still work in progress on that stuff ;-)

Evan Coury
Owner

@prolic, I'm curious -- why the redundant find($id) in both the repository and the mapper?

Ryan Hutchison

@EvanDotPro looks like prolics next PR will clean this up. I am curious how others feel about the magic method implementation @prolic brought up?

Deleted user
ghost commented

The content you are editing has changed. Reload the page and try again.

I'm pretty heavily against magic methods. The cons (no IDE completion, debugging pains, and worse performance) outweigh the pro (singular, convenience during initial development).

Sending Request…

Attach images by dragging & dropping or selecting them. Octocat-spinner-32 Uploading your images… Unfortunately, we don't support that file type. Try again with a PNG, GIF, or JPG. Yowza, that's a big file. Try again with an image file smaller than 10MB. This browser doesn't support image attachments. We recommend updating to the latest Internet Explorer, Google Chrome, or Firefox. Something went really wrong, and we can't process that image. Try again.

Mike Willbanks

I also want to note that we really need to also look at how we handle data overall; when you think about an entity, what has changed during the duration of that object. It might fall outside and that is alright, but realistically the entity is the container that should know everything about itself. Secondly hydration is great but should always work such that you can give it what you want and it deals with it (or produce multiple hydrators) for injecting standard objects, arrays and just about anything that could be transversable. I'm a huge proponent of mapper patterns when done correctly, it's just about the hardest thing to beat in terms of separation of duties and concerns and realistically barely adds anytime to the development cycle.

prolic
prolic commented

@EvanDotPro -- your question was: why the redundant find($id) in both the repository and the mapper?
Answer: the mapper is responsible for persisting and removing objects. Therefore "persist()" and "remove()" method. As a mapper can store an object, the most easiest task is, to get it back from database, given it's primary key. That's why i want that method in the mapper. Additional the repositories has all other queries ("findAll()", "findOnyBy()", and so on), and just to be complete, it has the "find($id)" method, too, and just proxies the call back to the mapper.

prolic
prolic commented

About magic methods:
You can use them to get dynamic finders. Something like "findOneByUsername($username)" without the need, that this function exists. It can be dynamically resolved. On the other hand, you are not forced to use them. Just write your own repository class (or extends to original one), add your method, and you'll be fine. Full autocomplete is available back again.

Rob Allen
Owner

re prolic's comment about ZfcUser: I don't mind if ZfcUser's Zend\Db interaction has to be "doctrine-aware", but I'm not sure that ZfcUser's specific needs have to be in ZfcBase.

I see most people who start with ZfcBase's db stuff using it because they don't want to use Doctrine and want something simple to understand and build on. I don't mind if we have separate mapper and finder objects, though I'm still unsure what benefit the separation provides.

prolic
prolic commented

@akrabat I am fine with providing a ZfcDbAbstractionModule or whatever we name it. Just putted it in ZfcBase, just because it was already there.

Rob Allen
Owner

I'm generally against magic methods too. To take the example of findOneByUsername(), I feel that it should fail if it's not written. Otherwise, I'd start expecting findOneByNameAndEmail() and findOneByUsernameOrEmail() to also work and then where do you stop?

prolic
prolic commented

I used the magic methods approach, because this way, i can provide a generic DoctrineRepositoryProxy, that just calls every method on the original Doctrine repository. So you can implement the methods in your doctrine repository and use it via the DoctrineRepositoryProxy, which in it's generic implementation, can't know about the other methods.
Just extend this one and add the methods, proxying to the underlying DoctrineRepository will get the magic method stuff out of your way. But that should be up to the user.

prolic
prolic commented

About the naming of "model" and "entity":
Generally, I would prefer the term "entity". On the other hand, in mongodb terms, they are "documents" and not "entities".
We can use "entity" for Zend\Db and Doctrine\ORM, but use "document" for MongoDB documents. The question then will be, how to name the UserModelInterface. UserEntityInterface? Will we have "class ZfcUserDoctrineMongoODM\Document\User implements ZfcUser\Entity\UserInterface" ?
I have no idea, perhabs we leave the "Model" namespace just for the interfaces. Thoughts?

Evan Coury
Owner

@prolic, re: naming of model/entity... I've generally always preferred the term "model", or specifically "domain model" (still Model for the namespace). However, at least in the PHP / MVC world, the term model has become quite ambiguous, while there is much less confusion surrounding the term "entity". For this reason, I'd be okay with normalizing on "entity", despite it not being my own personal preference.

For the Doctrine (ORM and Mongo) adapter modules like ZfcUserDoctrineMongoODM, I actually think our current approach in this area is pretty good. The abstract ZfcUserDoctrineMongoODM\Document\UserMappedSuperClass extends ZfcUser\Entity\User and then the concrete ZfcUserDoctrineMongoODM\Document\User extends ZfcUserDoctrineMongoODM\Document\UserMappedSuperClass. This allows things to work out-of-the-box functionality, while still giving the user a mapped superclass to extend if they'd prefer to use their own custom entity (add fields, etc).

Also, put me down as a general -1 for any kind of magic method approach. @SpiffyJr nailed the reasons in his comment.

prolic
prolic commented

@EvanDotPro agree with naming of model/entity.
@EvanDotPro again: i am fine with removing the magic methods approach. i just implemented it because it's used by DoctrineRepository, and i wanted to have a generic DoctrineRepositoryProxy, without the need to subclass this one.

Evan Coury
Owner

One of the purposes of a "mapper" is to decouple the entity (domain model) class from the db schema. Somewhere, there must be a map between the database field names and class properties. Should we put this field->property map in the mapper or the hydrator? Are those actually two different things? @weierophinney's PhlyPeep example has this mapping in his entity class, which I'm -1 on since the entity is then coupled to the db schema (though, I assume this was just to get a simple example working quickly).

Another thing to consider is that, instead of wrapping the TableGateway with the mapper (or repository?) we could simply extend the AbstractTableGatway, similar to how @weierophinney has done in his PhlyPeep module. However, I'll point out, at least by my current interpretation of Martin Fowler's PoEAA book, you should pick either Table Gateway OR Data Mapper, but not both. Data Mapper being for more complex use cases, whereas Table Gateway is appropriate for more simplistic use-cases, providing improved decoupling over Active Record. If my interpretation here is correct, my previous approach where I had a mapper object containing a table gateway actually doesn't make a ton of sense.

I also just want to remind everyone that @ralphschindler added a feature/event layer to the TableGateway that could prove useful... somehow. I'm not sure how, I just wanted to remind folks about the feature in case someone has a clever idea. See PR 1312 and the subsequent mailing list thread.

Jurian Sluiman

I would not use magic methods either. In the case of Doctrine, I would suggest you write your own repositories if you want to expose finder methods. Another approach would be to use a Criterion object. Then you can ZfcBase\Entity\Repository::findByCriterion(Criterion $criterion) where your $criterion is an object putting constraints for your finder.

Either way, I would expose explicit finder methods and/or use a generic findByCriterion() and not use magic methods. If you write a module Foo with domain entities and you have a FooZendDb module and FooDoctrineORM module to support the domain models, you might write the methods explicitly in the ZendDb module and use the magic methods of the DoctrineORM module.

A second comment would be the dependency on Zend\Db. I would prefer to keep ZfcBase out of any relation with Zend\Db. I prefer a simple module with contracts for domain models and contracts for mappers/repositories and two other modules providing implementations based on either Zend\Db or DoctrineORM.

prolic
prolic commented

Mapping information "field->propertiy mapping" must be in the mapper class. I don't know Zend\Db 2.0 that much, because I am a big fan of Doctrine, but I think we have Metadata?! no? So this metadata could be used to determine the primary key, f.e., or know which fields are DateTime, and which are simple scalars.
A concrete hydrator implementation (f.e. UserHydrator), can aggregate the mapper in order to do hydration. (The same approach works with my HumusDoctrineHydrator, which will soon be part of the DoctrineModule with a different name, probably RecursiveHydrator).
A hydrator is distinct from a mapper. Hydrator maps arrays -> objects and objects -> arrays, where a data mapper maps objects -> database and database -> objects. So they are kind of similar as they both do mapping.
About Zend\Db usage: As long as I have the same interface, I don't care about the implementation details. Could be a table gateway or simple Zend\Db\Sql queries.
I would argue, we can use Table Gateway for the more simple tasks and switch back to Zend\Db\Sql usage, when there is a more complex use-case.

Next question will be: do we need form element name -> entity field mapping + additional entity field -> database field mapping?

Evan Coury
Owner

@juriansluiman,

A second comment would be the dependency on Zend\Db. I would prefer to keep ZfcBase out of any relation with Zend\Db. I prefer a simple module with contracts for domain models and contracts for mappers/repositories and two other modules providing imnplementations [sic] based on either Zend\Db or DoctrineORM.

I can appreciate this approach. In general, I'm in favor of keeping ZfcBase fairly lightweight, and just about supporting common contracts and patterns. Keeping a hard Zend\Db dependency out of our generic implementation and contracts helps ensure we keep the abstraction in ZfcBase conducive to developing modules which can cleanly support both Zend\Db and Doctrine (or any other data access layer for that matter). After all, the entire purpose of this should be to abstract our domain layer from the underlying data storage / access method. Keep in mind that it's not uncommon for data to be coming from a source such as a web service instead of Zend\Db or Doctrine.

Edit: I did have ZfcUser split up with separate modules for Zend\Db and Doctrine support at one point -- people complained. I think it's okay to bundle a "default" data access layer with a module, as long as it can be swapped out by an additional module.

Evan Coury
Owner

@prolic,

Mapping information "field->propertiy mapping" must be in the mapper class. I don't know Zend\Db 2.0 that much, because I am a big fan of Doctrine, but I think we have Metadata?! no? So this metadata could be used to determine the primary key, f.e., or know which fields are DateTime, and which are simple scalars.

I'd argue we shouldn't need the assistance of metadata. Generally we'll be doing things explicitly ourselves. We know that the registered_date field is a DateTime when we're writing the entity / mapper. We shouldn't need the metadata to tell us this. Plus, there's the extra queries, caching, etc. Too much added complexity, IMO.

prolic
prolic commented

Completely new idea: We get rid of DataMapper and Repositories at all!

We have an AbstractUserService in ZfcUser, which holds all business logic, and as soon as it comes to persisting data, an abstract function "doPersist($object)" will get called.
A concrete service implementation will implement the doPersist method with whatever you want. doctrine, data mapper, sql, table gateway, we don't care.
That way we can have a very simple ZfcUser module, without the need to write custom mappers, repositories, and so on.

Thoughts?

Evan Coury
Owner

I just submitted PR 5102, which gives Zend\Db the ability to hydrate custom objects using a hydrator of your choosing (it just needs to implement Zend\Stdlib\Hydrator\HydratorInterface).

prolic
prolic commented

current state of my refactorings:
https://github.com/prolic/ZfcBase/tree/refactorings
https://github.com/prolic/ZfcUser/tree/refactorings

not all stuff running, but helps to show my ideas

prolic
prolic commented

i implemented a basic persistence layer abstraction, code still not working at all. further work until monday ;-)

Evan Coury
Owner

I'll just add that there is a relevant conversation brewing on ZF2 PR-1502.

prolic
prolic commented

just updated https://github.com/prolic/ZfcBase/tree/refactorings and https://github.com/prolic/ZfcUser/tree/refactorings

looking for your comments on persistence layer, mappinghydrator and usage of options in module

Michael Haessig

Hi everyone, I hope I do not disturb your conversation but I like the variant with persistence layer by prolic. The other variant with the repository and mapper is simply too complex and too much work for a single model

prolic
prolic commented

Another question that is running through my mind these days: Should we really copy & paste Doctrine\Common\Persistence* Interfaces under ZfcBase namespace in order to have the abstraction, that we are looking for? Doctrine Common has no dependency on Doctrine\ORM. So is it really bad to have a dependency on Doctrine\Common* without using Doctrine\ORM or Doctrine\ODM* at all?
Always the same argument: Zend\Db users/lovers feel that this is ugly, to have a dependency on something called "Doctrine\Common".
Will they be happier if we copy and paste them under ZfcBase or Zend namespace? really???

Matus Zeman

"If not, we should at least be able to provide an example of how something like this would work. An example provided by someone (I forget who), was something like $user->getAddress()->getCountry()->getName()."

I believe it was doctrine developer who gave this example. If it's one service call which loads all the entities I'm ok with it.
But the example suggests that an entity uses mapper/repository internally (or doctrine magic) to load its related entities. Personally I'm against this approach.
My plan with any service is that it should be easily expose-able via other programming interfaces such as REST, SOAP services, etc... and an entity carries "data" only and does not implement some logic or algorithm so it can also be easily transformed into any other format like JSON, XML .... using hydrator? Thus implementation of e.g. generic restful controller can be then quite easy: https://github.com/kapitchi/KapitchiBase/blob/master/src/KapitchiBase/Controller/ModelRestfulController.php.

prolic
prolic commented

Now Zend\Code\Annotation\Parser\DoctrineAnnotationParser has a dependency on Doctrine\Common\Annotations. So I like to definitly vote to use Doctrine\Common\Persistence interfaces and provide a simple implementation based on Zend\Db.

Evan Coury
Owner

@matuszemi I completely agree that entities should not call on the mapper/repository internally. The last thing I would want is something like Doctrine's proxy classes / lazy-loading which exposes developers directly to the data access layer in parts of the application that it shouldn't, completely bypassing the service layer. My own experience tells me this is a terrible design and should be avoided. +1 to simple / "dumb" entities. If you need some recursive hydration, that will usually be specific to a particular service method anyway, so do it there.

Evan Coury
Owner

@prolic I'm not sure I see the point in the dependency on the doctrine interfaces. Really, the only interface that matters is the mapper interface when implementing an alternative storage layer. Check out how ZfcUser and ZfcUserDoctrineORM are currently working together. Yes, it's sans-repository for the time being. I was unable to find any justification for breaking the mapper out into two different objects with the current (low) level of complexity.

prolic
prolic commented

Great

Evan Coury
Owner

Closing for now -- we seem to have a strategy that is working well for the time being.

Evan Coury EvanDotPro closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.