Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Enhancement of ElasticSearch support, SearchManager, and Serialization #77

Merged
merged 14 commits into from

3 participants

@MrHash
Collaborator

Various updates to ElasticSearch integration. Breaking changes to Solr/Lucene clients but possibly not difficult to implement updated interface requirements.

In general the fundamental difficulty is in the serialization of the mappings and the entities. So far the annotation updates gives a richer support for the mapping property generation, and a serializer option has been added to provide an appropriate means of serializing entities into the required format, which would otherwise be difficult with mappings alone.

MrHash added some commits
@MrHash MrHash Support Elastica 0.20.x client
Breaks BC because of namespace change
f41f359
@MrHash MrHash Added field mappings into class metadata 8cb5b60
@MrHash MrHash Enrichment of SearchManager, ElasticSearch client, and serializer sup…
…port

Mostly updates to ElasticSearch integration with breaking changes for
other clients. Also added support for more complex annotations, Unit of
Work style persistence, and support for different entity serialization
methods for persistence. Updated for Elastica 0.20.x support.
6d1195f
@MrHash MrHash Skip object serialization during removal cf59371
@MrHash MrHash Refactoring client interface clean up concrete classes cf274a9
lib/Doctrine/Search/ElasticSearch/Client.php
((6 lines not shown))
use Doctrine\Search\SearchClientInterface;
+use Doctrine\Search\Mapping\ClassMetadata;
+use Elastica\Client as Elastica_Client;
@Baachi Collaborator
Baachi added a note

Please rename it to ElasticaClient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/Doctrine/Search/SearchManager.php
((21 lines not shown))
/**
* Constructor
*
* @param Configuration $config
* @param SearchClientInterface $sc
*/
- public function __construct(Configuration $config, SearchClientInterface $sc)
+ public function __construct(Configuration $config, SearchClientInterface $sc, SerializerInterface $se = null)
@Baachi Collaborator
Baachi added a note

Can you move the SerializerInterface to the Configuration class? I think its a better place for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@Baachi Baachi commented on the diff
lib/Doctrine/Search/SearchManager.php
@@ -89,6 +105,22 @@ public function getClassMetadata($className)
{
return $this->metadataFactory->getMetadataFor($className);
}
+
+ /**
+ * @return SearchClientInterface
+ */
+ public function getClient()
+ {
+ return $this->searchClient;
@Baachi Collaborator
Baachi added a note

Why you need this?

@MrHash Collaborator
MrHash added a note

We need this for creating indexes and some other tasks at the moment. If all the underlying client methods can be abstracted we can eventually do away with it but for now it is useful.

@Baachi Collaborator
Baachi added a note

Okay make sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/Doctrine/Search/SearchManager.php
@@ -121,8 +153,8 @@ public function persist($object)
if (!is_object($object)) {
throw new UnexpectedTypeException($object, 'object');
}
-
- //$this->searchClient->createIndex($index, $type, $query);
+
+ $this->persisted[] = $object;
@Baachi Collaborator
Baachi added a note

scheduledForPersist?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/Doctrine/Search/SearchManager.php
((4 lines not shown))
- }
-
- /**
- * Bulk action
- *
- * @param object $object
- *
- * @throws UnexpectedTypeException
- */
- public function bulk($object)
- {
- if (!is_object($object)) {
- throw new UnexpectedTypeException($object, 'object');
- }
+
+ $this->removed[] = $object;
@Baachi Collaborator
Baachi added a note

scheduledForDelete?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/Doctrine/Search/SearchManager.php
((31 lines not shown))
}
+
+ protected function commitPersisted()
+ {
+ $documents = $this->sortObjects($this->persisted);
@Baachi Collaborator
Baachi added a note

Please respect the CS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@Baachi
Collaborator

I'm not really familar with elasticsearch but i know that the entities must be serialized to json. I don't know, its the best way to go, because it make to use solr more difficult.

@MrHash
Collaborator

My understanding is that Solr also requires entities to be serialized or is this incorrect? In any case the serialization can be made configurable.

@Baachi
Collaborator

I think serialize is the wrong word. The library should extract the data from the entity/document and pass this to the search backend. This library should only deal with plain php arrays, the serialization process should be done in solarium or elastica.

@MrHash MrHash Support for Entity hydration, Pagination
Search manager now supports querying through a magic wrapper class.
Hydration is provided by default when an entity manager is provided
through configuration or injection, or directly through a custom
Doctrine Query if provided. Hydration can be bypassed to return search
results directly. Currently only functional for ElasticSearch.
aadbf8c
@MrHash
Collaborator

Search request example with pagination:

$sm = $context->getDatabaseConnection('elasticsearch');
$em = $context->getDatabaseConnection('doctrine');
$sm->setEntityManager($em);

$q = $sm->createQuery()
    ->from('Entities\Feature')
    ->searchWith(new ElasticaQuery())
    ->hydrateWith($em->createQuery('SELECT e FROM Entities\Feature e WHERE e.id IN (:ids)')
    ->addSort('_score') //Client specific methods are passed through to the searchWith query object
    ->addSort(['start' => ['order' => 'desc']])
    ->useResultCache(true);  //Hydration via Doctrine uses result cache

$adapter = new Doctrine\Search\ElasticSearch\PaginationAdapter($q);
$pager = new Zend\Paginator\Paginator($adapter);
$pager->setCurrentPageNumber($page);  //Paging is actually handled at the search engine level
$pager->setItemCountPerPage($limit);
$results = $pager->getCurrentItems();
@MrHash
Collaborator

Regarding serialization, yes i think you are right. In fact the JMS serialization adapter returns an array, but the library will natively support array serialization soon. This may have seemed misleading, as was the __toString default. So these minor changes can be made easily.

@Baachi
Collaborator

Wow nice :+1:
One problem, the library should also work with the MongoDB ODM.

@MrHash
Collaborator

Well Doctrine2 already supports MongoDB no? I don't use it so i'm not familiar with the API.

@Baachi
Collaborator

Quick introduction:

  • doctrine2 is the ORM which deals with PostgreSQL, MySQL and SQLite.
  • MongoDB ODM deals only with the mongodb driver

There are 2 different libraries but have nearly the same api.

@MrHash
Collaborator

Well I'm afraid i may not be the best person to handle such an integration. Perhaps you could tell me why these modifications are not suitable for the ODM? Surely it's just a case of a different hydration abstraction or is the search completely different also?

@MrHash
Collaborator

I don' really understand what the problem is that you are concerned about. There is a Query::hydrateWith method which can accept any type of hydration query. So all that would be required is a little abstraction to provide a different default hydration query or hydration call. It is possible to determine the type and method of hydration by checking the type of EntityManager or hydration Query that has been provided to the SearchManager.

If you are concerned about implementing a full-text search service on the MongoDB layer then that would require the implementation of a MongoDB search client which conforms to the SearchClientInterface specification. Therefore I do not see any problem with these developments. The fact that Mongo uses map/reduce and RDBMS is SQL based is irrelevant because the Doctrine libraries abstract away that difference.

lib/Doctrine/Search/Configuration.php
@@ -23,6 +23,9 @@
use Doctrine\Common\Cache\ArrayCache;
use Doctrine\Search\Mapping\ClassMetadataFactory;
use Doctrine\Common\Persistence\Mapping\Driver\MappingDriver;
+use Doctrine\Search\SerializerInterface;
+use Doctrine\Search\Serializer\CallbackSerializer;
+use Doctrine\ORM\EntityManager;
@beberlei Owner

hard dependency on the EntityManager, we should work with ObjetManager interface here

@MrHash Collaborator
MrHash added a note

Agreed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/Doctrine/Search/Query.php
((149 lines not shown))
+ {
+ $this->doctrineQuery = $doctrineQuery;
+ if($parameter) $this->hydrationParameter = $parameter;
+ return $this;
+ }
+
+ /**
+ * Return a provided Doctrine Query or a default.
+ *
+ * @return DoctrineQuery
+ */
+ protected function getHydrationQuery()
+ {
+ if($this->doctrineQuery) return $this->doctrineQuery;
+
+ $em = $this->getSearchManager()->getEntityManager();
@beberlei Owner

This needs to be EntityManager independent.

@MrHash Collaborator
MrHash added a note

The EntityManager dependency here is only in order to build a default hydration query. This dependency can be removed completely by requiring the user to provide a hydration query.

@beberlei Owner

This is not how dependencies work, This class has a dependency on the ORM, if its used or not in some cases. The Doctrine\Search namespace however should be persistence agnostic.

@MrHash Collaborator
MrHash added a note

So you are saying the property names should be changed? Or should i remove the default query completely and leave an exception saying that a hydration query is required. This removes pretty much any dependency on the ORM anyway since the provided query will have the required ObjectManager reference in it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/Doctrine/Search/Mapping/Driver/AnnotationDriver.php
@@ -163,10 +181,7 @@ private function addValuesToMetadata(array $reflectedClassProperties, ClassMetad
if (false === property_exists($metadata, $propertyName)) {
throw new DriverException\PropertyDoesNotExistsInMetadataException($reflectedProperty->getName());
} else {
- $metadata->$propertyName = $class->$propertyName;
- /*I am not sure if that is needed
- * $metadata->addField($reflectedProperty);
- $metadata->addFieldMapping($reflectedProperty);*/
+ if(!is_null($class->$propertyName)) $metadata->$propertyName = $class->$propertyName;
@beberlei Owner

please wrap the line in { } and put it on an extra line

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@beberlei beberlei commented on the diff
lib/Doctrine/Search/Mapping/Driver/AnnotationDriver.php
((17 lines not shown))
- $metadata = $this->addValuesToMetadata($reflFieldAnnotations->getProperties(),
- $metadata,
- $documentsFieldAnnotation);
-
+ return $metadata;
+ }
+
+ /**
+ * Extract the methods annotations.
+ *
+ * @param \ReflectionMethod[] $reflMethods
+ * @param ClassMetadata $metadata
+ *
+ * @return ClassMetadata
+ */
+ private function extractMethodsAnnotations(array $reflMethods, ClassMetadata $metadata)
@beberlei Owner

this method is very complex and nested very deep. Is this necessary?

@MrHash Collaborator
MrHash added a note

This was a quick duplication of the method used to extract property annotations. This allows us to support virtual properties since the annotations allows a name to override the actual property/method name in the mapping properties generation from the entity. Ultimately It could be refactored along with the property annotation extraction method and perhaps other parts of this class as the requirements become clearer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
MrHash added some commits
@MrHash MrHash Change depency to ObjectManager cef7b81
@MrHash MrHash CS fix 7b588c4
@MrHash MrHash Remove ORM dependency
Hydration query is now required to be specified if hydration mode is
not set to bypass.
ae1e70b
lib/Doctrine/Search/SearchManager.php
((24 lines not shown))
}
+
+ protected function commitPersisted()
@Baachi Collaborator
Baachi added a note

Should be private at the moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/Doctrine/Search/SearchManager.php
((24 lines not shown))
}
+
+ protected function commitPersisted()
+ {
+ $documents = $this->sortObjects($this->scheduledForPersist);
+
+ foreach($documents as $index => $documentTypes)
+ {
+ foreach($documentTypes as $type => $documents)
+ {
+ $this->searchClient->addDocuments($index, $type, $documents);
+ }
+ }
+ }
+
+ protected function commitRemoved()
@Baachi Collaborator
Baachi added a note

Should be private at the moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/Doctrine/Search/SearchManager.php
((37 lines not shown))
+ }
+
+ protected function commitRemoved()
+ {
+ $documents = $this->sortObjects($this->scheduledForDelete, false);
+
+ foreach($documents as $index => $documentTypes)
+ {
+ foreach($documentTypes as $type => $documents)
+ {
+ $this->searchClient->removeDocuments($index, $type, $documents);
+ }
+ }
+ }
+
+ protected function sortObjects(array $objects, $serialize = true)
@Baachi Collaborator
Baachi added a note

Should be private at the moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@Baachi
Collaborator

@MrHash Sorry for the very long delay. I would like to merge this PR, can you fix the last comments?
Thank you for your work!

@MrHash
Collaborator

@Baachi Updated as requested. I understand there is an outstanding issue with compatibility with the Doctrine ODM query api but i think that should be fairly trivial to implement.

@Baachi Baachi merged commit e17d196 into from
@Baachi
Collaborator

PR is merged, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Apr 24, 2013
  1. @MrHash

    Support Elastica 0.20.x client

    MrHash authored
    Breaks BC because of namespace change
  2. @MrHash
Commits on Apr 28, 2013
  1. @MrHash

    Enrichment of SearchManager, ElasticSearch client, and serializer sup…

    MrHash authored
    …port
    
    Mostly updates to ElasticSearch integration with breaking changes for
    other clients. Also added support for more complex annotations, Unit of
    Work style persistence, and support for different entity serialization
    methods for persistence. Updated for Elastica 0.20.x support.
  2. @MrHash
  3. @MrHash
Commits on Apr 29, 2013
  1. @MrHash

    Support for Entity hydration, Pagination

    MrHash authored
    Search manager now supports querying through a magic wrapper class.
    Hydration is provided by default when an entity manager is provided
    through configuration or injection, or directly through a custom
    Doctrine Query if provided. Hydration can be bypassed to return search
    results directly. Currently only functional for ElasticSearch.
  2. @MrHash
  3. @MrHash

    Code annotation cleanup

    MrHash authored
  4. @MrHash

    Remove pagination adapter

    MrHash authored
  5. @MrHash
Commits on Apr 30, 2013
  1. @MrHash

    Change depency to ObjectManager

    MrHash authored
  2. @MrHash

    CS fix

    MrHash authored
Commits on May 6, 2013
  1. @MrHash

    Remove ORM dependency

    MrHash authored
    Hydration query is now required to be specified if hydration mode is
    not set to bypass.
Commits on Jun 18, 2013
  1. @MrHash

    Update method visibility

    MrHash authored
This page is out of date. Refresh to see the latest.
View
41 lib/Doctrine/Search/Configuration.php
@@ -23,6 +23,9 @@
use Doctrine\Common\Cache\ArrayCache;
use Doctrine\Search\Mapping\ClassMetadataFactory;
use Doctrine\Common\Persistence\Mapping\Driver\MappingDriver;
+use Doctrine\Search\SerializerInterface;
+use Doctrine\Search\Serializer\CallbackSerializer;
+use Doctrine\Common\Persistence\ObjectManager;
/**
* Configuration SearchManager
@@ -134,4 +137,42 @@ public function getClassMetadataFactory()
return $this->attributes['classMetadataFactory'];
}
+ /**
+ * Sets an entity serializer
+ *
+ * @param SerializerInterface $serializer
+ */
+ public function setEntitySerializer(SerializerInterface $serializer)
+ {
+ $this->attributes['serializer'] = $serializer;
+ }
+
+ /**
+ * Gets the entity serializer or provides a default if not set
+ *
+ * @return SerializerInterface
+ */
+ public function getEntitySerializer()
+ {
+ if(isset($this->attributes['serializer'])) {
+ return $this->attributes['serializer'];
+ }
+ return new CallbackSerializer();
+ }
+
+ /**
+ * @param ObjectManager $entityManager
+ */
+ public function setEntityManager(ObjectManager $entityManager)
+ {
+ $this->attributes['entityManager'] = $serializer;
+ }
+
+ /**
+ * @return ObjectManager
+ */
+ public function getEntityManager()
+ {
+ if(isset($this->attributes['entityManager'])) return $this->attributes['entityManager'];
+ }
}
View
120 lib/Doctrine/Search/ElasticSearch/Client.php
@@ -19,7 +19,15 @@
namespace Doctrine\Search\ElasticSearch;
+
+
use Doctrine\Search\SearchClientInterface;
+use Doctrine\Search\Mapping\ClassMetadata;
+use Elastica\Client as ElasticaClient;
+use Elastica\Type\Mapping;
+use Elastica\Document;
+use Elastica\Index;
+use Elastica\Query\MatchAll;
/**
* SearchManager for ElasticSearch-Backend
@@ -31,14 +39,14 @@
class Client implements SearchClientInterface
{
/**
- * @var \Elastica_Client
+ * @var ElasticaClient
*/
private $client;
/**
- * @param \Elastica_Client $client
+ * @param ElasticaClient $client
*/
- public function __construct(\Elastica_Client $client)
+ public function __construct(ElasticaClient $client)
{
$this->client = $client;
}
@@ -46,36 +54,116 @@ public function __construct(\Elastica_Client $client)
/**
* {@inheritDoc}
*/
+ public function addDocuments($index, $type, array $documents)
+ {
+ $type = $this->getIndex($index)->getType($type);
+
+ $batch = array();
+ foreach($documents as $id => $document)
+ {
+ $batch[] = new Document($id, $document);
+ }
+
+ $type->addDocuments($batch);
+ }
+
+ /**
+ * {@inheritDoc}
+ */
+ public function removeDocuments($index, $type, array $documents)
+ {
+ $type = $this->getIndex($index)->getType($type);
+ $type->deleteIds(array_keys($documents));
+ }
+
+ /**
+ * {@inheritDoc}
+ */
+ public function removeAll($index, $type)
+ {
+ $type = $this->getIndex($index)->getType($type);
+ $type->deleteByQuery(new MatchAll());
+ }
+
+ /**
+ * {@inheritDoc}
+ */
public function find($index, $type, $query)
{
- $index = $this->client->getIndex($index);
- return iterator_to_array($index->search($query));
+ $type = $this->getIndex($index)->getType($type);
+ return $type->search($query);
}
/**
* {@inheritDoc}
*/
- public function createIndex($index, $type, array $data)
+ public function createIndex($name, array $config = array())
{
- $index = $this->client->getIndex($index);
- $index->create();
-
- $index->addDocuments($data);
+ $index = $this->getIndex($name);
+ $index->create($config, true);
+ return $index;
}
-
+
/**
* {@inheritDoc}
*/
- public function deleteIndex($index)
+ public function getIndex($name)
{
- $index = $this->client->getIndex($index);
- $index->delete();
+ return $this->client->getIndex($name);
}
-
+
+ /**
+ * {@inheritDoc}
+ */
+ public function deleteIndex($index)
+ {
+ $this->getIndex($index)->delete();
+ }
+
/**
* {@inheritDoc}
*/
- public function bulkSearch(array $data)
+ public function createType(ClassMetadata $metadata)
+ {
+ $type = $this->getIndex($metadata->index)->getType($metadata->type);
+ $properties = $this->getMapping($metadata->fieldMappings);
+
+ $mapping = new Mapping($type, $properties);
+ $mapping->disableSource($metadata->source);
+ $mapping->setParam('_boost', array('name' => '_boost', 'null_value' => $metadata->boost));
+ $mapping->send();
+
+ return $type;
+ }
+
+ /**
+ * Generates property mapping from entity annotations
+ *
+ * @param array $fieldMapping
+ */
+ protected function getMapping($fieldMapping)
{
+ $properties = array();
+
+ foreach($fieldMapping as $propertyName => $fieldMapping)
+ {
+ if(isset($fieldMapping->name)) $propertyName = $fieldMapping->name;
+ $properties[$propertyName]['type'] = $fieldMapping->type;
+ if(isset($fieldMapping->includeInAll)) $properties[$propertyName]['include_in_all'] = $fieldMapping->includeInAll;
+ if(isset($fieldMapping->index)) $properties[$propertyName]['index'] = $fieldMapping->index;
+ if(isset($fieldMapping->boost)) $properties[$propertyName]['boost'] = $fieldMapping->boost;
+
+ if($fieldMapping->type == 'multi_field' && isset($fieldMapping->fields))
+ {
+ $properties[$propertyName]['fields'] = $this->getMapping($fieldMapping->fields);
+ }
+
+ if(in_array($fieldMapping->type, array('nested', 'object')) && isset($fieldMapping->properties))
+ {
+ $properties[$propertyName]['properties'] = $this->getMapping($fieldMapping->properties);
+ }
+ }
+
+ return $properties;
}
}
View
75 lib/Doctrine/Search/Mapping/Annotations/DoctrineAnnotations.php
@@ -29,12 +29,13 @@
*/
class Searchable extends Annotation
{
- /**
- * @var string $index;
+ /**
+ * @var string $index
*/
public $index;
- /**
- * @var string $type;
+
+ /**
+ * @var string $type
*/
public $type;
}
@@ -45,27 +46,41 @@ class Searchable extends Annotation
*/
final class ElasticSearchable extends Searchable
{
- /**
- * @var int $numberOfShards;
+ /**
+ * @var int $numberOfShards
*/
public $numberOfShards;
- /**
+
+ /**
* @var int $numnberOfReplicas
*/
public $numberOfReplicas;
- /**
- * @var string $op_type;
+
+ /**
+ * @var string $op_type
*/
public $opType;
- /**
- * @var float $parent;
+
+ /**
+ * @var float $parent
*/
public $parent;
+
/**
* TTL in milliseconds
* @var int $timeToLive
*/
public $timeToLive;
+
+ /**
+ * @var float
+ */
+ public $boost;
+
+ /**
+ * @var boolean
+ */
+ public $source;
}
/**
@@ -74,10 +89,20 @@ class Searchable extends Annotation
*/
class Field extends Annotation
{
- /**
- * @var float
+ /**
+ * @var float
*/
public $boost;
+
+ /**
+ * @var string
+ */
+ public $type;
+
+ /**
+ * @var string
+ */
+ public $name;
}
/**
@@ -91,9 +116,27 @@ class Field extends Annotation
/**
* @Annotation
- * @Target("PROPERTY")
+ * @Target({"PROPERTY", "METHOD", "ANNOTATION"})
*/
final class ElasticField extends Field
{
- /* configuration */
-}
+ /**
+ * @var boolean
+ */
+ public $includeInAll;
+
+ /**
+ * @var string
+ */
+ public $index;
+
+ /**
+ * @var array
+ */
+ public $fields;
+
+ /**
+ * @var array
+ */
+ public $properties;
+}
View
21 lib/Doctrine/Search/Mapping/ClassMetadata.php
@@ -60,7 +60,7 @@ class ClassMetadata implements ClassMetadataInterface
/**
* @var int
*/
- public $numberOfReplicas = 1;
+ public $numberOfReplicas = 0;
/**
* @var int
@@ -81,12 +81,17 @@ class ClassMetadata implements ClassMetadataInterface
* @var int
*/
public $value = 1;
-
+
/**
- * @var float
+ * @var boolean
*/
- public $boost = 1.0;
+ public $source = true;
+ /**
+ * @var float
+ */
+ public $boost = 1.0;
+
/**
* @var string
*/
@@ -135,7 +140,7 @@ public function __sleep()
{
// This metadata is always serialized/cached.
return array(
- 'boost',
+ 'boost',
'className',
'fieldMappings',
'index',
@@ -232,11 +237,11 @@ public function hasField($fieldName)
* @param \ReflectionProperty $field
* @param array $mapping
*/
- /*public function addFieldMapping(\ReflectionProperty $field, $mapping = array())
+ public function addFieldMapping(\Reflector $field, $mapping = array())
{
$fieldName = $field->getName();
$this->fieldMappings[$fieldName] = $mapping;
- }*/
+ }
/**
* @param \ReflectionProperty $field
@@ -244,7 +249,7 @@ public function hasField($fieldName)
/*public function addField(\ReflectionProperty $field)
{
$fieldName = $field->getName();
- $this->reflFields[$fieldName] = $field;
+ $this->reflFields[] = $field;
}*/
View
45 lib/Doctrine/Search/Mapping/Driver/AnnotationDriver.php
@@ -69,10 +69,11 @@ public function loadMetadataForClass($className, ClassMetadata $metadata)
}
$reflProperties = $reflClass->getProperties();
+ $reflMethods = $reflClass->getMethods();
$this->extractClassAnnotations($reflClass, $metadata);
$this->extractPropertiesAnnotations($reflProperties, $metadata);
-
+ $this->extractMethodsAnnotations($reflMethods, $metadata);
}
@@ -126,26 +127,43 @@ private function extractPropertiesAnnotations(array $reflProperties, ClassMetada
$documentsFieldAnnotations = array();
foreach ($reflProperties as $reflProperty) {
foreach ($this->reader->getPropertyAnnotations($reflProperty) as $annotation) {
- foreach (self::$documentFieldAnnotationClasses as $i => $fieldAnnotationClass) {
+ foreach (self::$documentFieldAnnotationClasses as $fieldAnnotationClass) {
if ($annotation instanceof $fieldAnnotationClass) {
- $documentsFieldAnnotations[$i] = $annotation;
+ $metadata->addFieldMapping($reflProperty, $annotation);
continue 2;
}
}
}
}
- foreach ($documentsFieldAnnotations as $documentsFieldAnnotation) {
- $reflFieldAnnotations = new \ReflectionClass($documentsFieldAnnotation);
- $metadata = $this->addValuesToMetadata($reflFieldAnnotations->getProperties(),
- $metadata,
- $documentsFieldAnnotation);
-
+ return $metadata;
+ }
+
+ /**
+ * Extract the methods annotations.
+ *
+ * @param \ReflectionMethod[] $reflMethods
+ * @param ClassMetadata $metadata
+ *
+ * @return ClassMetadata
+ */
+ private function extractMethodsAnnotations(array $reflMethods, ClassMetadata $metadata)
@beberlei Owner

this method is very complex and nested very deep. Is this necessary?

@MrHash Collaborator
MrHash added a note

This was a quick duplication of the method used to extract property annotations. This allows us to support virtual properties since the annotations allows a name to override the actual property/method name in the mapping properties generation from the entity. Ultimately It could be refactored along with the property annotation extraction method and perhaps other parts of this class as the requirements become clearer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ {
+ $documentsFieldAnnotations = array();
+ foreach ($reflMethods as $reflMethod) {
+ foreach ($this->reader->getMethodAnnotations($reflMethod) as $annotation) {
+ foreach (self::$documentFieldAnnotationClasses as $fieldAnnotationClass) {
+ if ($annotation instanceof $fieldAnnotationClass) {
+ $metadata->addFieldMapping($reflMethod, $annotation);
+ continue 2;
+ }
+ }
+ }
}
return $metadata;
}
-
+
/**
* @param \ReflectionProperty[] $reflectedClassProperties
* @param ClassMetadata $metadata
@@ -163,10 +181,9 @@ private function addValuesToMetadata(array $reflectedClassProperties, ClassMetad
if (false === property_exists($metadata, $propertyName)) {
throw new DriverException\PropertyDoesNotExistsInMetadataException($reflectedProperty->getName());
} else {
- $metadata->$propertyName = $class->$propertyName;
- /*I am not sure if that is needed
- * $metadata->addField($reflectedProperty);
- $metadata->addFieldMapping($reflectedProperty);*/
+ if(!is_null($class->$propertyName)) {
+ $metadata->$propertyName = $class->$propertyName;
+ }
}
}
View
205 lib/Doctrine/Search/Query.php
@@ -0,0 +1,205 @@
+<?php
+
+namespace Doctrine\Search;
+
+use Doctrine\Search\Exception\DoctrineSearchException;
+
+class Query
+{
+ const HYDRATE_BYPASS = -1;
+
+ const HYDRATION_PARAMETER = 'ids';
+
+ /**
+ * @var SearchManager
+ */
+ protected $_sm;
+
+ /**
+ * @var object
+ */
+ protected $query;
+
+ /**
+ * @var object
+ */
+ protected $hydrationQuery;
+
+ /**
+ * @var string
+ */
+ protected $hydrationParameter = self::HYDRATION_PARAMETER;
+
+ /**
+ * @var string
+ */
+ protected $entityClass;
+
+ /**
+ * @var integer
+ */
+ protected $hydrationMode;
+
+ /**
+ * @var boolean
+ */
+ protected $useResultCache;
+
+ /**
+ * @var integer
+ */
+ protected $cacheLifetime;
+
+ /**
+ * @var integer
+ */
+ protected $count;
+
+ public function __construct(SearchManager $sm)
+ {
+ $this->_sm = $sm;
+ }
+
+ /**
+ * Magic method to pass query building to the underlying query
+ * object, saving the need to abstract.
+ *
+ * @param string $method
+ * @param array $arguments
+ */
+ public function __call($method, $arguments)
+ {
+ call_user_func_array(array($this->query, $method), $arguments);
+ return $this;
+ }
+
+ /**
+ * Specifies the searchable entity class to search against.
+ *
+ * @param string $entityClass
+ */
+ public function from($entityClass)
+ {
+ $this->entityClass = $entityClass;
+ return $this;
+ }
+
+ /**
+ * Set the query object to be executed on the search engine
+ *
+ * @param mixes $query
+ */
+ public function searchWith($query)
+ {
+ $this->query = $query;
+ return $this;
+ }
+
+ protected function getSearchManager()
+ {
+ return $this->_sm;
+ }
+
+ /**
+ * Set the hydration mode from the underlying query modes
+ * or bypass and return search result directly from the client
+ *
+ * @param integer $mode
+ */
+ public function setHydrationMode($mode)
+ {
+ $this->hydrationMode = $mode;
+ return $this;
+ }
+
+ /**
+ * If hydrating with Doctrine then you can use the result cache
+ * on the default or provided query
+ *
+ * @param boolean $useCache
+ * @param integer $cacheLifetime
+ */
+ public function useResultCache($useCache, $cacheLifetime = null)
+ {
+ $this->useCache = $useCache;
+ $this->cacheLifetime = $cacheLifetime;
+ return $this;
+ }
+
+ /**
+ * Return the total hit count for the given query as provided by
+ * the search engine.
+ */
+ public function count()
+ {
+ return $this->count;
+ }
+
+ /**
+ * Set a custom Doctrine Query to execute in order to hydrate the search
+ * engine results into required entities. The assumption is made the the
+ * search engine result id is correlated to the entity id. An optional
+ * query parameter override can be specified.
+ *
+ * @param object $hydrationQuery
+ * @param string $parameter
+ */
+ public function hydrateWith($hydrationQuery, $parameter = null)
+ {
+ $this->hydrationQuery = $hydrationQuery;
+ if($parameter) {
+ $this->hydrationParameter = $parameter;
+ }
+ return $this;
+ }
+
+ /**
+ * Return a provided hydration query
+ *
+ * @return object
+ */
+ protected function getHydrationQuery()
+ {
+ if(!$this->hydrationQuery) {
+ throw new DoctrineSearchException('A hydration query is required for hydrating results to entities.');
+ }
+
+ return $this->hydrationQuery;
+ }
+
+ /**
+ * Execute search and hydrate results if required.
+ *
+ * @param integer $hydrationMode
+ * @throws DoctrineSearchException
+ * @return mixed
+ */
+ public function getResult($hydrationMode = null)
+ {
+ if($hydrationMode) {
+ $this->hydrationMode = $hydrationMode;
+ }
+
+ $classMetadata = $this->getSearchManager()->getClassMetadata($this->entityClass);
+ $resultSet = $this->getSearchManager()->find($classMetadata->index, $classMetadata->type, $this->query);
+
+ switch(get_class($resultSet)) {
+ case 'Elastica\ResultSet':
+ $this->count = $resultSet->getTotalHits();
+ $results = $resultSet->getResults();
+ break;
+ default:
+ throw new DoctrineSearchException('Unknown result set class');
+ }
+
+ if($this->hydrationMode == self::HYDRATE_BYPASS) {
+ return $resultSet;
+ }
+
+ $ids = array_map(function($result) { return $result->getId(); }, $results);
+ return $this->getHydrationQuery()
+ ->setParameter($this->hydrationParameter, $ids ?: null)
+ ->useResultCache($this->useResultCache, $this->cacheLifetime)
+ ->getResult($this->hydrationMode);
+ }
+}
View
59 lib/Doctrine/Search/SearchClientInterface.php
@@ -20,6 +20,7 @@
namespace Doctrine\Search;
use Doctrine\Common\Persistence\ObjectManager;
+use Doctrine\Search\Mapping\ClassMetadata;
/**
* Interface for a Doctrine SearchManager class to implement.
@@ -40,23 +41,59 @@
function find($index, $type, $query);
/**
- * Allows to search by the search api of a backend like Solr directly
+ * Creates a document index
*
- * @param string $index The name of the index.
- * @param string $type The type of the index.
- * @param array $data The data to be indexed.
+ * @param string $name The name of the index.
+ * @param string $config The configuration of the index.
*/
- function createIndex($index, $type, array $data);
-
+ function createIndex($name, array $config = array());
+
/**
+ * Gets a document index reference
*
- * @param array $data
+ * @param string $name The name of the index.
*/
- function deleteIndex($index);
+ function getIndex($name);
/**
- * @param array $query
+ * Deletes an index and its types and documents
+ *
+ * @param string $index
*/
- function bulkSearch(array $query);
-
+ function deleteIndex($index);
+
+ /**
+ * Create a document type mapping as defined in the
+ * class annotations
+ *
+ * @param ClassMetadata $metadata
+ */
+ function createType(ClassMetadata $metadata);
+
+ /**
+ * Adds documents of a given type to the specified index
+ *
+ * @param string $index
+ * @param string $type
+ * @param array $documents Indexed by document id
+ */
+ function addDocuments($index, $type, array $documents);
+
+ /**
+ * Remove documents of a given type from the specified index
+ *
+ * @param string $index
+ * @param string $type
+ * @param array $documents Indexed by document id
+ */
+ function removeDocuments($index, $type, array $documents);
+
+ /**
+ * Remove all documents of a given type from the specified index
+ * without deleting the index itself
+ *
+ * @param string $index
+ * @param string $type
+ */
+ function removeAll($index, $type);
}
View
143 lib/Doctrine/Search/SearchManager.php
@@ -21,7 +21,6 @@
use Doctrine\Common\Persistence\ObjectManager;
use Doctrine\Search\SearchClientInterface;
-use Doctrine\Common\EventManager;
use Doctrine\Search\ElasticSearch\Client;
use Doctrine\Search\Configuration;
use Doctrine\Common\Annotations\AnnotationReader;
@@ -29,6 +28,8 @@
use Doctrine\Search\Exception\UnexpectedTypeException;
use Doctrine\Search\Mapping\ClassMetadata;
use Doctrine\Search\Mapping\ClassMetadataFactory;
+use Doctrine\Search\Serializer\CallbackSerializer;
+use Doctrine\ORM\EntityManager;
/**
* Interface for a Doctrine SearchManager class to implement.
@@ -38,21 +39,39 @@
*/
class SearchManager
{
- /**
- * @var SearchClientInterface
+ /**
+ * @var SearchClientInterface
*/
private $searchClient;
- /**
- * @var Configuration $configuration
+ /**
+ * @var Configuration $configuration
*/
private $configuration;
- /**
- * @var ClassMetadataFactory
+ /**
+ * @var ClassMetadataFactory
*/
private $metadataFactory;
-
+
+ /**
+ * @var SerializerInterface
+ */
+ private $serializer;
+
+ /**
+ * @var array
+ */
+ private $scheduledForPersist = array();
+
+ /**
+ * @var array
+ */
+ private $scheduledForDelete = array();
+
+ /** @var EntityManager */
+ private $entityManager;
+
/**
* Constructor
*
@@ -68,9 +87,30 @@ public function __construct(Configuration $config, SearchClientInterface $sc)
$this->metadataFactory->setSearchManager($this);
$this->metadataFactory->setConfiguration($this->configuration);
$this->metadataFactory->setCacheDriver($this->configuration->getMetadataCacheImpl());
+
+ $this->serializer = $this->configuration->getEntitySerializer();
+ $this->entityManager = $this->configuration->getEntityManager();
}
/**
+ * Inject a Doctrine 2 entity manager
+ *
+ * @param EntityManager $em
+ */
+ public function setEntityManager(EntityManager $em)
+ {
+ $this->entityManager = $em;
+ }
+
+ /**
+ * @return EntityManager
+ */
+ public function getEntityManager()
+ {
+ return $this->entityManager;
+ }
+
+ /**
* @return Configuration
*/
public function getConfiguration()
@@ -89,6 +129,22 @@ public function getClassMetadata($className)
{
return $this->metadataFactory->getMetadataFor($className);
}
+
+ /**
+ * @return SearchClientInterface
+ */
+ public function getClient()
+ {
+ return $this->searchClient;
@Baachi Collaborator
Baachi added a note

Why you need this?

@MrHash Collaborator
MrHash added a note

We need this for creating indexes and some other tasks at the moment. If all the underlying client methods can be abstracted we can eventually do away with it but for now it is useful.

@Baachi Collaborator
Baachi added a note

Okay make sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ }
+
+ /**
+ * @return mixed
+ */
+ public function getIndex($name)
+ {
+ return $this->getClient()->getIndex($name);
+ }
/**
* @return ClassMetadataFactory
@@ -104,9 +160,9 @@ public function getClassMetadataFactory()
* @param string $type
* @param string $query
*/
- public function find($index = null, $type = null, $query = null)
+ public function find($index, $type, $query)
{
- $this->searchClient->find($index, $type, $query);
+ return $this->searchClient->find($index, $type, $query);
}
/**
@@ -121,8 +177,8 @@ public function persist($object)
if (!is_object($object)) {
throw new UnexpectedTypeException($object, 'object');
}
-
- //$this->searchClient->createIndex($index, $type, $query);
+
+ $this->scheduledForPersist[] = $object;
}
/**
@@ -137,28 +193,65 @@ public function remove($object)
if (!is_object($object)) {
throw new UnexpectedTypeException($object, 'object');
}
+
+ $this->scheduledForDelete[] = $object;
}
/**
- * Bulk action
- *
- * @param object $object
- *
- * @throws UnexpectedTypeException
+ * Commit all changes
*/
- public function bulk($object)
+ public function commit()
{
- if (!is_object($object)) {
- throw new UnexpectedTypeException($object, 'object');
- }
+ $this->commitPersisted();
+ $this->commitRemoved();
}
+
+ private function commitPersisted()
+ {
+ $documents = $this->sortObjects($this->scheduledForPersist);
+
+ foreach($documents as $index => $documentTypes)
+ {
+ foreach($documentTypes as $type => $documents)
+ {
+ $this->searchClient->addDocuments($index, $type, $documents);
+ }
+ }
+ }
+
+ private function commitRemoved()
+ {
+ $documents = $this->sortObjects($this->scheduledForDelete, false);
+
+ foreach($documents as $index => $documentTypes)
+ {
+ foreach($documentTypes as $type => $documents)
+ {
+ $this->searchClient->removeDocuments($index, $type, $documents);
+ }
+ }
+ }
+
+ private function sortObjects(array $objects, $serialize = true)
+ {
+ $documents = array();
+ foreach($objects as $object)
+ {
+ $metadata = $this->getClassMetadata(get_class($object));
+ $document = $serialize ? $this->serializer->serialize($object) : $object;
+ $documents[$metadata->index][$metadata->type][$object->getId()] = $document;
+ }
+ return $documents;
+ }
/**
- * Commit all changes
- *
- * @return boolean
+ * Returns a search engine Query wrapper which can be executed
+ * to retrieve results;
+ *
+ * @return Query
*/
- public function commit()
+ public function createQuery()
{
+ return new Query($this);
}
}
View
10 lib/Doctrine/Search/Serializer/AnnotationSerializer.php
@@ -0,0 +1,10 @@
+<?php
+
+namespace Doctrine\Search\Serializer;
+
+use Doctrine\Search\SerializerInterface;
+
+class AnnotationSerializer implements SerializerInterface
+{
+
+}
View
20 lib/Doctrine/Search/Serializer/CallbackSerializer.php
@@ -0,0 +1,20 @@
+<?php
+
+namespace Doctrine\Search\Serializer;
+
+use Doctrine\Search\SerializerInterface;
+
+class CallbackSerializer implements SerializerInterface
+{
+ protected $callback;
+
+ public function __construct($callback = 'toArray')
+ {
+ $this->callback = $callback;
+ }
+
+ public function serialize($object)
+ {
+ return $object->{$this->callback}();
+ }
+}
View
25 lib/Doctrine/Search/Serializer/JMSSerializer.php
@@ -0,0 +1,25 @@
+<?php
+
+namespace Doctrine\Search\Serializer;
+
+use Doctrine\Search\SerializerInterface;
+use JMS\Serializer\SerializerBuilder;
+use JMS\Serializer\SerializationContext;
+
+class JMSSerializer implements SerializerInterface
+{
+ protected $serializer;
+ protected $context;
+
+ public function __construct(SerializationContext $context = null)
+ {
+ $this->context = $context;
+ $this->serializer = SerializerBuilder::create()->addDefaultHandlers()->build();
+ }
+
+ public function serialize($object)
+ {
+ $context = $this->context ? clone $this->context : null;
+ return json_decode($this->serializer->serialize($object, 'json', $context), true);
+ }
+}
View
8 lib/Doctrine/Search/SerializerInterface.php
@@ -0,0 +1,8 @@
+<?php
+
+namespace Doctrine\Search;
+
+interface SerializerInterface
+{
+ public function serialize($object);
+}
View
34 lib/Doctrine/Search/Solr/Client.php
@@ -20,6 +20,7 @@
namespace Doctrine\Search\Solr;
use Doctrine\Search\SearchClientInterface;
+use Doctrine\Search\Mapping\ClassMetadata;
use Doctrine\Common\Persistence\ObjectManager;
/**
@@ -54,27 +55,34 @@ public function createIndex($index, array $data)
{
}
-
- public function updateIndex(array $data)
+
+ public function createType(ClassMetadata $metadata)
+ {
+
+ }
+
+ public function getIndex($index)
{
}
- /**
- *
- * @param array $data
- */
public function deleteIndex($index)
{
- // TODO: Implement deleteIndex() method.
+
}
- /**
- * @param array $data
- */
- public function bulkAction(array $data)
+ public function addDocuments($index, $type, array $documents)
{
- // TODO: Implement bulkAction() method.
+
+ }
+
+ public function removeDocuments($index, $type, array $documents)
+ {
+
+ }
+
+ public function removeAll($index, $type)
+ {
+
}
-
}
View
37 lib/Doctrine/Search/ZendLucene/Client.php
@@ -19,7 +19,9 @@
namespace Doctrine\Search\ZendLucene;
+
use Doctrine\Search\SearchClientInterface;
+use Doctrine\Search\Mapping\ClassMetadata;
use Doctrine\Common\Persistence\ObjectManager;
/**
@@ -54,27 +56,34 @@ public function createIndex($index, array $data)
{
}
+
+ public function createType(ClassMetadata $metadata)
+ {
- public function updateIndex(array $data)
+ }
+
+ public function getIndex($index)
{
}
- /**
- *
- * @param array $data
- */
public function deleteIndex($index)
{
- // TODO: Implement deleteIndex() method.
+
}
-
- /**
- * @param array $data
- */
- public function bulkAction(array $data)
- {
- // TODO: Implement bulkAction() method.
+
+ public function addDocuments($index, $type, array $documents)
+ {
+
+ }
+
+ public function removeDocuments($index, $type, array $documents)
+ {
+
+ }
+
+ public function removeAll($index, $type)
+ {
+
}
-
}
Something went wrong with that request. Please try again.