[WIP] MongoDB Support #461

samvdb · 2016-03-16T11:35:22Z

Q	A
Bug fix?	no
New feature?	yes
BC breaks?	no
Deprecations?	no
Tests pass?	no
Fixed tickets	#439, #361
License	MIT

This is a work in progress. I will not be working on it but it might help others to get started.
Stuff to fix:

Add filters and enable them in the service xml
Add tests
Abstract ODM and ORM data providers since logic is duplicated
Abstract Paginator since it's duplicated
Move Query extension interfaces to data layer agnostic namespace
Filter/AbstractFilter should not contain ORM logic (see addJoinsForNestedProperty)
Add ext-mongo to travis-ci

dunglas · 2016-03-16T12:15:15Z

About embedded documents, they are not resources (in sense of API Platform) but raw objects. IMO the best is do nothing: let the standard ObjectNormalizer of Symfony normalizing and denormalizing it (as we already do for ORM @Embeddable and \DateTime).

The only adaptation I can think about is documenting such embedded types in the Hydra ApiDocumentationGenerator and ContextBuilder but it can be done later in another PR.

samvdb · 2016-03-16T12:33:35Z

@dunglas, i agree with you that embedded documents should not be processed as resources. However there were some problems in the old serializer (before refactor to symf abstract serializer).
My resources are beeing serialized atm without errors, embedded docs are still empty but that is probably a serialization context problem. Will test it out further. I hope other people are interested in mongo support and are willing to help out.

Thanks

dunglas · 2016-03-16T12:35:22Z

Yes this problem has been figured out in the v2.

I'm personally very interested by Mongo support.

dunglas · 2016-03-16T12:35:45Z

src/Bridge/Doctrine/MongoDB/CollectionDataProvider.php

+use ApiPlatform\Core\Bridge\Doctrine\MongoDB\Extension\QueryCollectionExtensionInterface;
+
+/**
+ * Collection data provider for the Doctrine ORM.


Doctrine MongoDB ODM.

dunglas · 2016-03-16T12:37:16Z

src/Bridge/Doctrine/MongoDB/CollectionDataProvider.php

+/**
+ * Collection data provider for the Doctrine ORM.
+ */
+class CollectionDataProvider implements CollectionDataProviderInterface


Is this exactly the same class that the class for the ORM?

If it is the case, this class can be moved in the Doctrine and used by both instead of being duplicated. If changes are small, an abstract class can be introduced.

Yeah, the interfaces for query extensions should be pulled out of ORM also i think. The paginator could be abstracted since the only read difference if the getResult and applyToCollection.

If you want me to proceed with this give me the go ahead :)

Yes 👍 for that.

dunglas · 2016-03-16T12:40:41Z

src/Bridge/Symfony/Bundle/DependencyInjection/Configuration.php

@@ -61,6 +63,15 @@ public function getConfigTreeBuilder()
                ->scalarNode('name_converter')->defaultNull()->info('Specify a name converter to use.')->end()
                ->booleanNode('enable_fos_user')->defaultValue(false)->info('Enable the FOSUserBundle integration.')->end()
                ->booleanNode('enable_nelmio_api_doc')->defaultTrue()->info('Enable the Nelmio Api doc integration.')->end()
+                ->scalarNode('db_driver')


It would be nice to be able MongoDB and a relational DBMS at the same time.

mmm, maybe work towards a service definition like - managers: - orm: default_em, - mongodb: default_dm ?

Or just register the service if the MongoODMBundle is registered. If a user want to disable it, it can still use a compiler pass.

Or better, flags like for FOSUser and NelmioApiDoc: enable_doctrine_orm, enable_doctrine_mongodb_odm.

dunglas · 2016-03-16T12:41:07Z

src/Bridge/Symfony/Bundle/DependencyInjection/Configuration.php

+                ->scalarNode('db_driver')
+                    ->validate()
+                        ->ifNotInArray($supportedDrivers)
+                        ->thenInvalid('The driver %s is not supported. Please choose one of '.json_encode($supportedDrivers))


samvdb · 2016-03-16T12:41:46Z

src/Bridge/Doctrine/MongoDB/ItemDataProvider.php

+
+        if (!$fetchData || $manager instanceof DocumentManager) {
+            // @todo: Check potential problems with using the $id instead of $identifiers
+            return $manager->getReference($resourceClass, $id);


@dunglas, any thoughts on this?

IIRC MongoDB doesn't support composed primary key? If so, it can be ignored and just use $identifierValues[0].

There are severals options for compound keys, i would however not include support for this in this PR.

samvdb · 2016-03-16T12:58:36Z

@dunglas, what about the abstraction of the data providers or pagination extension?
I updated the rest of the comments

samvdb · 2016-03-16T13:10:13Z

I can confirm that the serializer is working properly for embedded documents!

dunglas · 2016-03-16T13:13:28Z

@dunglas, what about the abstraction of the data providers or pagination extension?

I've not got it.

samvdb · 2016-03-16T14:41:18Z

@dunglas any idea how to fix the nelmio apidoc parser to support embedded documents?
Example: @Property(iri="https://schema.org/Offer", readableLink=false)
src/Metadata/Property/Factory/SerializerPropertyMetadataFactory.php:66 throws an exception because the document is not a resource and thus marking the property as a readable link.

When parsing the api it is being rendered as an IRI which is not correct.
see src/Bridge/NelmioApiDoc/Parser/ApiPlatformParser.php:194
Do you propose adding an extra property in the Annotation or create a workaround somewhere?

dunglas · 2016-03-16T16:31:45Z

Maybe can we use the Symfony PropertyInfo component to list properties of this embedded object. Same apply for Hydra doc. But IMO it should be done in a separate PR. Let's focus on MongoDB support here.

jonny827 · 2016-03-17T08:54:17Z

I am interested in support for DynamoDB and other non-MongoDB projects as well. Would it potentially be helpful to use an existing project to be able to push API-platform to several other platforms without "re-inventing the wheel"? You could benefit from their aligning several systems with a unified API. Use their API to communicate with API-platform. When they add support, we get all the benefits and you would only have to maintain the interface to their their API.

[https://github.com/doctrine/KeyValueStore]

sroze · 2016-03-17T09:08:33Z

src/Bridge/Doctrine/MongoDB/CollectionDataProvider.php

+    /**
+     * @param ManagerRegistry                      $managerRegistry
+     * @param QueryCollectionExtensionInterface[]  $collectionExtensions
+     * @param CollectionDataProviderInterface|null $decorated


My personal preference is to inject the decorated object as the first argument, so we can easily see that it's a decorator.

I believe it is best practice for arguments that are null by default to be listed last.

Oh I didn't see that the decorated argument could be null. So... that means that it's not the "decorated" one, you should find another name.

sroze · 2016-03-17T09:09:11Z

src/Bridge/Doctrine/MongoDB/CollectionDataProvider.php

+        }
+
+        $manager = $this->managerRegistry->getManagerForClass($resourceClass);
+        if (null === $manager) {


Maybe you can merge these lines?

if (null === ($manager = $this->managerRegistry->getManagerForClass($resourceClass))) {

sroze · 2016-03-17T09:12:29Z

src/Bridge/Doctrine/MongoDB/Extension/PaginationExtension.php

+        $this->enabled = $enabled;
+        $this->clientEnabled = $clientEnabled;
+        $this->clientItemsPerPage = $clientItemsPerPage;
+        $this->itemsPerPage = $itemsPerPage;


To me, we have too many parameters here. That would be nice to create a normalised PaginationConfiguration object, that we could use both here and in the Doctrine implementation.

Totally agree with you!

sroze · 2016-03-17T09:13:25Z

src/Bridge/Doctrine/MongoDB/Extension/PaginationExtension.php

+        return new Paginator($queryBuilder->getQuery()->execute());
+    }
+
+    private function isPaginationEnabled(Request $request, ResourceMetadata $resourceMetadata, string $operationName = null) : bool


That looks like to be copy/pasted, or at least, that's not specific to MongoDb. Is there a way you can decouple that?

It is copy pasted from the ORM version, don't have much time to work on it to cleanup everything. Like i said it's a PR kickstart mongo support :)

Yup, but you know... even if we tried to decouple from the ORM, it's always the first one that actually introduce the second real adapter that suffers 😛

sroze · 2016-03-17T09:15:38Z

src/Bridge/Doctrine/MongoDB/ItemDataProvider.php

+            }
+
+            $identifiers[$propertyName] = $identifierValues[$i];
+            ++$i;


Instead of having this $i variable, you can probably use the foreach ... as $i => $propertyName.

This logic should also be abstracted since it's duplicated for ORM and ODM

sroze · 2016-03-17T09:20:01Z

Thank you @samvdb. Obviously we need to go through the review process before. :)

Can you add tests? Also, I can see that many of your classes are actually not used (because commented in the service.xml file), so could you add in your PR description a checkbox list of the tasks you've done and the tasks you have to do before we review it again?

nettob · 2016-04-14T17:17:20Z

@eopaquette https://github.com/mongofill/mongofill is an pure PHP replacement of the MongoDB driver

dunglas · 2016-05-09T18:58:53Z

Hello @samvdb, any update regarding this PR? It would be so cool to have it done for the v2.

samvdb · 2016-05-09T19:04:26Z

Sorry kevin,

I'm using this branch as is, works fine but dont have the time to write
tests for it.

Gr

coudenysj · 2016-05-17T08:07:45Z

@samvdb Any help needed on this?

jonny827 · 2016-05-23T22:05:41Z

+1 for version 2. Need any help? I'm sure several of us could pitch in to help!

samvdb · 2016-05-23T22:38:04Z

if someone could write tests for this code i'm pretty sure it could be merged.

jonny827 · 2016-05-23T23:15:33Z

@samvdb --- Can you please rebase your branch so that tests pass and resolve any conflicts? This will allow me to pull from your repo and try and write some tests(given my below experience level in this particular area) What from your perspective may be needed to allow other NOSQL databases hook into Api-Platform? Thank you for pushing to get this feature supported in the platform!

@coudenysj -- Can you please help with writing some tests? Secondly, thank you interest in helping bring MongoDB to the platform.

I was hoping DynamoDB would lead from the push towards ODM. I have never used a NOSQL DB. I do know there is great use cases for them that would blend with my project. So in short, I have never written a test; especially for a DB I have never used. Anyone have a good tutorial on MongoDB tests? I am not afraid to learn. Also, I was going to help with DOCS but realize the 2.x branch implementation pretty much would only need a few lines changed potentially for the docs to explain the support of ODM in addition to ORM which is probably best done by someone with clearer understanding of how those work.

To recap, if anyone has some resources for test writing for MongoDB or just in general please let me know. Secondly, if anyone find a place I fit in to progress this PR/the platform as a whole then please let me know!

…tion

- add symfony mongo test - add mongo on travis & appveyor

Simperfit · 2016-11-08T10:42:01Z

This PR is updated with master.

@samvdb Can you please give me a test project / or a path in order to test this ?

ppounder · 2017-02-23T13:54:02Z

Hi all,

Could anyone advise how close we are for this PR to be merged? I can see a few issues above, and possibly more so if it gets rebased. Where I work we are trying to set up using ApiPlatform and MongoDB so would be happy to try and test it in our environment.

We've already tried by running this, but it's not picking up our Document's so if anyone can point us in the right direction with a little help/documentation we can try and assist in getting this over the line.

Regards
Paul

soyuka · 2017-02-23T15:27:40Z

@ppounder I think it's just missing a WriteListener https://github.com/api-platform/core/blob/master/src/Bridge/Doctrine/EventListener/WriteListener.php
https://github.com/api-platform/core/blob/master/src/Bridge/Symfony/Bundle/Resources/config/doctrine_orm.xml#L82

qneyrat · 2017-03-22T21:51:16Z

up, what's up?

dunglas · 2017-03-22T23:24:00Z

Still waiting for a stable version of Mongo ODM for PHP 7 using the new extension (I've not checked but I'm not aware of a new release).

qneyrat · 2017-03-22T23:48:03Z

I use mongo-php-adapter in prod. It's compulsory at the moment. I am afraid that doctrine upgrade in a long time. This repo is dev by one core developer and it's stable and it's recommended to use odm with php7.

ad3n · 2017-03-27T07:10:52Z

I use MongoDB in PHP7 and as far as i use, i don't have any major issue about it

ad3n · 2017-04-10T03:37:43Z

Just because change the root class (MongoDb Extension) that making stack? I don't know about it, but i think, if api platform can support mongodb, it is a good forward... We waiting for a year to support it... That is so long, right?

dunglas · 2017-04-10T07:25:31Z

Ok to use the bridge if it is stable enough. But someone needs to finish this PR first (make Travis green and update the documentation).

ad3n · 2017-04-11T03:21:33Z

@samvdb : can you help us to finish this PR? we need your help...

qneyrat · 2017-04-11T09:59:59Z

If necessary I am available

samvdb · 2017-04-11T10:36:51Z

I am no longer using this branch and have no time to spare atm. If anyone needs repo access i can grant it.

soyuka · 2017-04-11T12:18:14Z

Anyone can fork your branch and create a new PR based on it :). @qneyrat feel free to do so, we will be glad to help reviewing!

See: * api-platform/core#461 * doctrine/mongodb-odm#1553

ihr-it-projekt · 2017-07-13T21:00:54Z

i have fork this branch and merged it with the current master. If someone will help to make the lights green, i will be glad. https://github.com/ihr-it-projekt/core

PR: #1248

@samvdb Can you explain what i should do that api-plattform is right configured to use mongo db?

soyuka · 2017-08-28T09:15:46Z

Closing in favor of #1293 (continues this one).

[client] fixed the package name

dunglas added the enhancement label Mar 16, 2016

dunglas reviewed Mar 16, 2016
View changes

samvdb reviewed Mar 16, 2016
View changes

sroze reviewed Mar 17, 2016
View changes

theofidry mentioned this pull request May 1, 2016

can i use this api with mongodb ? api-platform/api-platform#65

Closed

dunglas mentioned this pull request May 1, 2016

No resource found for object of type one to many relation api-platform/api-platform#70

Closed

samvdb and others added 10 commits Jul 5, 2016

Change DataProviderInterface's namespace to use new DataProvider loca…

Unverified

The committer email address is not verified.

Learn about signing commits

35b81c0

…tion

Tag mongodb data providers

Unverified

The committer email address is not verified.

Learn about signing commits

1417ccb

Remove old dataprovider definitions from doctrine_mongodb.xml

Unverified

The committer email address is not verified.

Learn about signing commits

b882971

Use correct PaginatorInterface namespace

Unverified

The committer email address is not verified.

Learn about signing commits

86fe88a

hotfix: add mongodb to ci && fix unit test

Unverified

The committer email address is not verified.

Learn about signing commits

1ec400c

add composer.json

Unverified

The committer email address is not verified.

Learn about signing commits

388c35c

ignore platform reqs

Unverified

The committer email address is not verified.

Learn about signing commits

e05dcf6

ignore platform reqs

Unverified

The committer email address is not verified.

Learn about signing commits

c116adf

- add behat on mongo

Unverified

The committer email address is not verified.

Learn about signing commits

ce30ecb

- add symfony mongo test - add mongo on travis & appveyor

- add behat on mongo

Unverified

The committer email address is not verified.

Learn about signing commits

Loading status checks…

f56a53e

- add symfony mongo test - add mongo on travis & appveyor

soyuka mentioned this pull request Apr 6, 2017

[QUESTION] MongoDB routes generation api-platform/api-platform#273

Closed

rubenrua added a commit to rubenrua/Notes that referenced this pull request Jun 15, 2017

Add MongoDB sadness

44b8132

See: * api-platform/core#461 * doctrine/mongodb-odm#1553

Zayon mentioned this pull request Jul 28, 2017

[WIP] Mongodb support #1293

Closed

soyuka closed this Aug 28, 2017

api-platform / core

Sponsor api-platform/core

[WIP] MongoDB Support #461

[WIP] MongoDB Support #461

Conversation

samvdb commented Mar 16, 2016 • edited

dunglas commented Mar 16, 2016

samvdb commented Mar 16, 2016

dunglas commented Mar 16, 2016

This comment has been minimized.

dunglas Mar 16, 2016 Member

This comment has been minimized.

dunglas Mar 16, 2016 Member

This comment has been minimized.

samvdb Mar 16, 2016 Author Contributor

This comment has been minimized.

dunglas Mar 16, 2016 Member

This comment has been minimized.

dunglas Mar 16, 2016 Member

This comment has been minimized.

samvdb Mar 16, 2016 Author Contributor

This comment has been minimized.

dunglas Mar 16, 2016 Member

This comment has been minimized.

dunglas Mar 16, 2016 Member

This comment has been minimized.

dunglas Mar 16, 2016 Member

This comment has been minimized.

samvdb Mar 16, 2016 Author Contributor

This comment has been minimized.

dunglas Mar 16, 2016 Member

This comment has been minimized.

samvdb Mar 16, 2016 Author Contributor

samvdb commented Mar 16, 2016

samvdb commented Mar 16, 2016

dunglas commented Mar 16, 2016

samvdb commented Mar 16, 2016

dunglas commented Mar 16, 2016

jonny827 commented Mar 17, 2016

This comment has been minimized.

sroze Mar 17, 2016 Member

This comment has been minimized.

samvdb Mar 17, 2016 Author Contributor

This comment has been minimized.

sroze Mar 17, 2016 Member

This comment has been minimized.

sroze Mar 17, 2016 Member

This comment has been minimized.

sroze Mar 17, 2016 Member

This comment has been minimized.

samvdb Mar 17, 2016 Author Contributor

This comment has been minimized.

sroze Mar 17, 2016 Member

This comment has been minimized.

samvdb Mar 17, 2016 Author Contributor

This comment has been minimized.

sroze Mar 17, 2016 Member

This comment has been minimized.

theofidry Mar 17, 2016 Member

This comment has been minimized.

sroze Mar 17, 2016 Member

This comment has been minimized.

samvdb Mar 17, 2016 Author Contributor

sroze commented Mar 17, 2016

nettob commented Apr 14, 2016

dunglas commented May 9, 2016

samvdb commented May 9, 2016 • edited

coudenysj commented May 17, 2016

jonny827 commented May 23, 2016

samvdb commented May 23, 2016

jonny827 commented May 23, 2016 • edited

Simperfit commented Nov 8, 2016

ppounder commented Feb 23, 2017

soyuka commented Feb 23, 2017

qneyrat commented Mar 22, 2017

dunglas commented Mar 22, 2017

qneyrat commented Mar 22, 2017

ad3n commented Mar 27, 2017

ad3n commented Apr 10, 2017

dunglas commented Apr 10, 2017 • edited

samvdb commented Mar 16, 2016 •

edited

dunglas Mar 16, 2016
Member

dunglas Mar 16, 2016
Member

samvdb Mar 16, 2016
Author Contributor

dunglas Mar 16, 2016
Member

dunglas Mar 16, 2016
Member

samvdb Mar 16, 2016
Author Contributor

dunglas Mar 16, 2016
Member

dunglas Mar 16, 2016
Member

dunglas Mar 16, 2016
Member

samvdb Mar 16, 2016
Author Contributor

dunglas Mar 16, 2016
Member

samvdb Mar 16, 2016
Author Contributor

sroze Mar 17, 2016
Member

samvdb Mar 17, 2016
Author Contributor

sroze Mar 17, 2016
Member

sroze Mar 17, 2016
Member

sroze Mar 17, 2016
Member

samvdb Mar 17, 2016
Author Contributor

sroze Mar 17, 2016
Member

samvdb Mar 17, 2016
Author Contributor

sroze Mar 17, 2016
Member

theofidry Mar 17, 2016
Member

sroze Mar 17, 2016
Member

samvdb Mar 17, 2016
Author Contributor

samvdb commented May 9, 2016 •

edited

jonny827 commented May 23, 2016 •

edited

dunglas commented Apr 10, 2017 •

edited

ihr-it-projekt commented Jul 13, 2017 •

edited