Convert from Laminas\Db to Doctrine #2233
Conversation
|
I spent some more time on proof-of-concept work today. I decided to try revising some of the code in the admin module's tag controller, since that does complex queries and seemed like a potentially interesting starting point. So far, results are promising -- not only is the DQL query language more readable than the Laminas\Db object-oriented query builder, but I also managed to find and fix a bug along the way. Some notes on today's progress: 1.) We need to have a place to put database-related logic. In the previous code, this was in the row and table classes. In the new code, I propose a set of VuFind\Db\Service classes designed to deal with various types of entities. I've started work on a TagService here, and I've set up the necessary plugin mechanism for interacting with services. This may need more granularity, but we can decide that as we gain more experience working with Doctrine. 2.) For extensibility purposes, we also need a way to abstract entity classes so that they can be overridden. I have created a VuFind\Db\Entity\PluginManager for this purpose, and I defined a marker interface for entities so that the plugin manager can validate what it is creating. Note that overriding entities appears to have one major limitation: annotations are not inherited from parent classes, so any local custom entity needs to fully duplicate its parent's properties and annotations. Perhaps there is a way around this, but given how infrequently I expect people to need to extend/override entities, it may also be a limitation that we can live with. There is still a massive amount of work that needs to be done here, but I'm encouraged by this start. I will continue to chip away as time permits, and I welcome input or offers of collaboration. Eventually, it may be best to "divide and conquer" on all the conversion work, since there is so much of it. |
|
Just a quick update on progress, because I believe this branch is currently in a somewhat broken state: In order to incorporate Doctrine queries with the Laminas Paginator, I had to add the The Anyway, I've created TODO checkboxes to capture all these balls that are in the air, and I apologize for leaving this in a somewhat broken state, but I didn't want to lose my work in progress, and I'm nearly out of time for today. I'll pick this up again as soon as practical. |
|
Hi @demiankatz, this change is very exciting because our Admin Interface has been using I wonder whether it would make sense to develop the new Doctrine based VuFind database layer as a separate module/package, in a similar way that has been discussed in #2272 and #2283 (which I hope to continue working on as soon as time allows)? In any case it would be great to have this developed in a way that would allow for code reuse, even if only by copying the classes over! |
|
Thanks, @aleksip, I'm glad that you're happy with this approach. I'm open to organizing the code any way that is helpful; I think it's just a question of what works best. Perhaps one approach could be to create a VuFindDb module to contain the Entity classes, ConnectionFactory, etc. I'm not sure if the actual Service classes would fit there as well, or if they'd be better off in the main VuFind module due to their potential entanglement with other dependencies. I think I'd have to get deeper into the project to have a stronger opinion about that, but I'm open to input! In any case, right now, my priority is finishing up all the outstanding work on VuFind 9, so this is more or less on hold, but I'm hoping this will be one of the priority changes for VuFind 10, and I expect I'll be spending a lot of time on it in 2023. I'm definitely happy to discuss/plan in the meantime so we have a clear path forward when it comes time to execute the work. I'm also very open to brainstorming strategies for collaboration if you or anyone on your team might be able to help with the updates, as this is a very big project. I'm hoping that, in addition to updating everything, we can make strides toward introducing better test coverage for core code, since if we do this right, it should make things easier to test than the old database library did. |
|
@demiankatz Sounds good! I am currently experimenting with potential |
|
Thanks, @aleksip, that's great! Certainly if you have classes from your admin code that it would make sense to move over here for use in the core, that might save us some time and trouble. I'm not sure if the existing auto-generated Entity classes are actually the best they could be, as most of them have not yet been tested. |
There was a problem hiding this comment.
@aleksip, thanks for the progress on configuration here -- just leaving a couple of comments/questions related to the latest changes.
| @@ -338,6 +338,7 @@ | |||
| 'service_manager' => [ | |||
| 'allow_override' => true, | |||
| 'factories' => [ | |||
| 'Doctrine\ORM\Mapping\Driver\AnnotationDriver' => 'VuFind\Db\AnnotationDriverFactory', | |||
There was a problem hiding this comment.
@aleksip, I see that you deleted this factory, but the configuration is still here. We should clean something up.
There was a problem hiding this comment.
Yes, forgot to remove that line. Should I push the change to your branch or what would be the best way to work on this together?
There was a problem hiding this comment.
Please feel free to push directly here, as long as the code still works. I'm not going to be working on this much until next year, so I welcome any progress you can make in the meantime, and I don't anticipate that we'll "step on each other," so to speak.
There was a problem hiding this comment.
Also, if at any point it would be helpful to get a second opinion or anything, or if you would like me to run any tests, just let me know and I'll be happy to help!
There was a problem hiding this comment.
@demiankatz I meant to push the changes to my repository and then point you there. Not sure how multi-user PRs should work... Should I push future suggested changes to my clone so you can then decide whether to merge to your branch from there?
There was a problem hiding this comment.
@aleksip, it is possible to make a pull request against a pull request -- you can just target a PR against this branch. That might be the easiest workflow going forward, especially when multi-user collaboration picks up... but since things are quiet right now, feel free to work directly in this branch, at least until you get things into a correct/stable state again. (I think it's easier to just move forward and fix things than to roll back changes and make a separate PR, at least under the present circumstances).
There was a problem hiding this comment.
@demiankatz Seems like I can create a PR for the branch in your repository! Should I do that? Edit: missed your above reply.
There was a problem hiding this comment.
If you want to try it as a proof-of-concept, feel free... but otherwise, as I said, you can just go ahead and push. :-)
|
@demiankatz I have accidentally pushed my changes to your branch, sorry about that! I have been refactoring our Admin Interface database implementation to be based on this PR, and those were the changes I needed to make. They were needed to support two entity managers with connections to different databases. I don't claim to fully understand how Doctrine's configuration works, but the I also wondered about the need for |
|
@aleksip, I am certainly not an expert on Doctrine's configuration either, and I've been learning as I go... so if the annotation driver can be replaced by pure configuration, I'm entirely in favor of that, and if a different naming convention would be more robust, I also have no objection to changes. |
demiankatz
added
the
architecture
This is work in progress on replacing VuFind's database abstraction layer.
TODO
doctrine/doctrine-orm-moduleandroave/psr-container-doctrinein composer.json, or if we can migrate fully from the psr-container-doctrine to the doctrine-orm-module.createNativeQueryworks correctly across platforms (e.g. make sure tag deduplication works in PostgreSQL)