forked from thibault/tjSolrDoctrineBehaviorPlugin
-
Notifications
You must be signed in to change notification settings - Fork 1
/
README
431 lines (311 loc) · 13.2 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
# tjSolrDoctrineBehaviorPlugin #
The `tjSolrDoctrineBehaviorPlugin` provides a Doctrine behavior to easily index and search model objects
in the Solr search engine.
## Installation ##
### Plugin installation ###
You can install this plugin the usual way (RTFM), or if you want to work with the trunk:
$ cd plugins
$ git clone git://github.com/thibault/tjSolrDoctrineBehaviorPlugin.git
Then activate the plugin in the `config/ProjectConfiguration.class.php` file.
### Setting up Solr ###
The recommended Solr version is the latest (1.4). It is not tested with any other Solr version.
A Solr installation is already embedded in the plugin, in the `lib/vendor/solr` directory.
It's a default installation, and the only modified file is the `schema.xml`.
If you want to use your own Solr installation, add those lines to the schema, in the "fields" section:
[xml]
<!-- unique document id -->
<field name="sf_unique_id" type="string" indexed="true" stored="true" required="true" />
<!-- indexed object class -->
<field name="sf_meta_class" type="string" indexed="true" stored="true" required="true" />
<!-- indexed object id -->
<field name="sf_meta_id" type="sint" indexed="true" stored="true" required="true" />
<!-- default search field -->
<field name="sf_text" type="text" indexed="true" stored="true" multiValued="true" />
And after the fields definition:
[xml]
<!-- Field to use to determine and enforce document uniqueness.
Unless this field is marked with required="false", it will be a required field
-->
<uniqueKey>sf_unique_id</uniqueKey>
<!-- field for the QueryParser to use when an explicit fieldname is absent -->
<defaultSearchField>sf_text</defaultSearchField>
<!-- copyField commands copy one field to another at the time a document
is added to the index. It's used either to index the same field differently,
or to add multiple fields to the same field for easier/faster searching.
-->
<copyField source="*_t" dest="sf_text" />
<copyField source="*_s" dest="sf_text" /
Once your configuration is correct, you can start Solr. Of course, you need a java installation.
$ cd plugins/tjSolrDoctrineBehaviorPlugin/lib/vendor/solr
$ java -jar start.jar
### Solr in a production environment ###
In a production environment, you should run Solr as a daemon. The right way to do depends on your server's
system, however, some startup scripts are included in the `lib/vendor/scripts` directory.
On a debian server :
$ cp plugins/tjSolrDoctrineBehaviorPlugin/lib/vendor/scripts/debian/solr /etc/init.d/solr
$ chmod 755 /etc/init.d/solr
$ update-rc.d solr defaults
Before running Solr, just set the PROJECT_NAME value to your symfony project directory.
PROJECT_NAME=mysfproject
The Solr index will be created in you `data` dir, and the logs will go in `log` dir.
You have to create one last directory before we're ready.
$ mkdir logs/solr
More logging configuration can be found in the
`plugins/tjSolrDoctrineBehaviorPlugin/lib/vendor/solr/logging.properties` file.
You can check out some more [documentation about logging here](http://wiki.apache.org/solr/LoggingInDefaultJettySetup).
Then you can start the daemon:
$ /etc/init.d/solr start
Check this URL to be sure everything went fine.
[http://localhost:8983/solr/admin/](http://localhost:8983/solr/admin/)
## How to use ? ##
### Enabling the behavior ###
To index some model objects into Solr, you have to modify your schema.yml file. Add the `Solr` behavior
to the object type you want to index, and define which fields needs to be indexed.
Here's an example schema file:
[yml]
Thread:
columns:
title:
type: string(255)
notnull: true
Post:
actAs:
Solr:
fields: [ title, body ]
columns:
thread_id:
type: integer
notnull: true
title:
type: string(255)
notnull: true
body:
type: clob
notnull: true
relations:
Thread:
onDelete: CASCADE
foreignAlias: Posts
Rebuild your model, load your data, et voilà!
Each time a Post object is created/updated/deleted, the Solr index will be automaticaly updated.
### Field mapping ###
In the previous example, the plugin will try to index the title and body fields into Solr.
You have to manualy define those names in the Solr schema, or indexing will fail :
[xml]
<field name="title" type="text" indexed="true" stored="true" multiValued="false" />
<field name="body" type="text" indexed="true" stored="true" multiValued="false" />
You also have to make sure that those fiels are copied in the "sf_text" fields :
[xml]
<copyField source="title" dest="sf_text" />
<copyField source="body" dest="sf_text" />
Configuring each field in the Solr schema can be a pain. That's why this plugin allows you to use
Solr's dynamic fields.
If your model field's name matches some specific pattern, the Solr field will be automaticaly created.
For example, each field suffixed with "_t" will be created with a "text" type, and copied into the "sf_text"
default search field.
You can configure this mapping in the Doctrine schema :
[yml]
Post:
actAs:
Solr:
fields: [ title, body ]
fieldmap: { title: title_t, body: body_t }
…
Look into the "dynamicField" entries in the Solr's schema to see available patterns.
### Indexing virtual fields ###
Thanks to the Doctrine magic, it is very easy to index virtual fields. All you have to do is
to add a custom getter in the model class. This way, you can even index some relations fields.
[yml]
Post:
actAs:
Solr:
fields: [ title, body, threadTitle ]
fieldmap: { title: title_t, body: body_t, threadTitle: thread_t }
Add this getter in the Post class :
[php]
public function getThreadTitle()
{
return $this->getThread()->getTitle();
}
### Connecting to Solr ###
This plugins uses the default Solr connexion parameters. You can override them in the doctrine schema :
[yml]
Post:
actAs:
Solr:
fields: [ title, body ]
fieldmap: { title: title_t, body: body_t }
host: localhost
post: 8389
path: '/solr'
…
If you have a Solr installation with a multicore index, and want to index differents object types,
just change the "path" parameter for each type.
### Searching ###
Indexing is good, but searching is better.
Once you have attached the behavior to your model, it will provide a "search" method.
[php]
// returns true or false wether solr is available or not
$solrAvailable = Doctrine_Core::getTable('Post')->isSearchAvailable();
if(!$solrAvailable)
throw new sfException('Search is unavailable right now. Please come back later');
$post = new Post();
$post->title = 'test title';
$post->body = 'this is my body';
$post->Thread = new Thread();
$post->Thread->title = 'test thread';
$post->save();
// returns every indexed elements
$results = Doctrine::getTable('Post')->search('*:*');
// search in every text fields
$results = Doctrine::getTable('Post')->search('test');
// search only in "title" field
$results = Doctrine::getTable('Post')->search('title_t:test');
// You can set the offset and limit params
$results = Doctrine::getTable('Post')->search('*:*', 0, 10);
// search can take extra params
$results = Doctrine::getTable('Post')->search('*:*', 0, 10, array('sort' => 'score desc', 'fl' => '*,score'));
The `search` methods retuns a php array corresponding to the Solr xml response.
[php]
$results = Doctrine::getTable('Post')->search('*:*');
var_dump($results);
array(2) {
["responseHeader"]=>
array(3) {
["status"]=>
int(0)
["QTime"]=>
int(0)
["params"]=>
array(7) {
["start"]=>
string(1) "0"
["q"]=>
string(3) "*:*"
["json.nl"]=>
string(3) "map"
["wt"]=>
string(4) "json"
["fq"]=>
string(18) "sf_meta_class:Post"
["version"]=>
string(3) "1.2"
["rows"]=>
string(2) "30"
}
}
["response"]=>
array(3) {
["numFound"]=>
int(1)
["start"]=>
int(0)
["docs"]=>
array(1) {
[0]=>
array(7) {
["sf_unique_id"]=>
string(6) "Post_9"
["sf_meta_class"]=>
string(4) "Post"
["sf_meta_id"]=>
int(9)
["timestamp"]=>
string(24) "2010-02-06T15:16:30.523Z"
["title_t"]=>
array(1) {
[0]=>
string(5) "title"
}
["sf_text"]=>
array(2) {
[0]=>
string(5) "title"
[1]=>
string(4) "body"
}
["body_t"]=>
array(1) {
[0]=>
string(4) "body"
}
}
}
}
}
Instead of a dummy php array, the plugin can also generate a Doctrine_Query object,
which you can manipulate as usual. For instance, you can modify sorting options, paginate the list,
add other criterias, etc.
[php]
$q = Doctrine::getTable('Post')->createSearchQuery('my query');
$maxResults = 50; // 256 is the function default, for a performance matter
$q2 = Doctrine::getTable('Post')->createSearchQuery('my other query', $maxResults);
There is a last available method to performs a search. You can use the `solr:search` task from the command line.
$ php symfony solr:search Post "my query"
$ php symfony help solr:search
### reset index ###
Sometimes, you may want to reset the index and clear all indexed objects. There is a function to do this.
[php]
// remove all posts from solr index
Doctrine_Core::getTable('Post')->deleteIndex();
You can also do this from the command line.
$ php symfony solr:reset-index Post
$ php symfony help solr:reset-index
This method will only remove objects from the current class (here: Post).
### Transactions ###
Solr supports transactions. By default, the plugins sends a `commit` message after every index operation.
However, you may want to perform many operations in one time, then it is more efficient to send only one
commit when the job's done.
[php]
$thread = new Thread();
$thread->title = 'test tread';
$tread->save();
Doctrine_Core::getTable('Post')->beginTransaction();
for($i = 0 ; $i < 20 ; $i++)
{
$post = new Post();
$post->title = "post $i";
$post->body = 'post body';
$post->Thread = $thread
$post->save();
}
// You can use the inTransaction function to know if a transaction exists
Doctrine_Core::getTable('Post')->inTransaction(); // returns true
// After the commit, data will be available for searching
Doctrine_Core::getTable('Post')->commit();
## Testing the plugin ##
This plugins comes with an auto-generated (with the `sfTaskExtraPlugin`) fixtures project, to run the tests
without touching your own app. Tests are not included in the package version, so you'll have to checkout
the trunk version.
Before you run the tests, make sure Solr is running is accessible.
Note that those tests are written to be run with lime 2 (alpha1).
WARNING : By default, the tests will be run over the default Solr index. Make sure you won't lose any data.
Here's how you can easily run the plugin test suite :
* export the SYMFONY shell var to the symfony lib path. Example :
$ export SYMFONY='/var/www/myproject/lib/vendor/symfony-1.4.1/lib/'
* Setup the database for testing
The plugin's tests are made to run against an independant db. You have to create it before anything else.
mysql> CREATE DATABASE solr_doctrine_behavior_test;
mysql> GRANT ALL ON solr_doctrine_behavior_test.* TO test;
If you want to use an existing database, you have to configure the database connexion by yourself
$ php plugins/tjSolrDoctrineBehaviorPlugin/test/fixtures/project/symfony configure:database --env=test
* build the bootstrap project. You don't have to load the fixtures, this will be done on time.
$ php plugins/tjSolrDoctrineBehaviorPlugin/test/fixtures/project/symfony doctrine:build --env=test --all
* run the tests
$ php symfony test:plugin tjSolrDoctrineBehaviorPlugin
or, if the sfTaskExtraPlugin is not installed :
$ php plugins/tjSolrDoctrineBehaviorPlugin/test/bin/prove.php
## Professional support ##
I'm a freelance developer, working from France. If you want to contact me for professional support,
use the email you will find in the "information" tab of this plugin. I speak french and english.
## Thanks ##
I'd wish to thank the following persons for contributing to this plugin :
* Guillaume Roques
## TODO ##
Here's some ideas for further developpements:
* Add an option to allow or not objects modifications when solr is unavailable
* Add a task to batch index objects
* Compatibility with the i18n behavior
* Add geolocation search
* add a moreLikeThis() function
* add an option to set the default search field