Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 649 lines (504 sloc) 22.078 kb
d8c46d2 Robert Newson Add travis build status image
authored
1 [![Build Status](https://secure.travis-ci.org/rnewson/couchdb-lucene.png)](http://travis-ci.org/rnewson/couchdb-lucene)
2
6d1c074 Robert Newson add notes about version compatibility.
authored
3 <h1>Version Compatibility</h1>
4
daf007b Robert Newson Simplify compatibily statement
authored
5 CouchDB-Lucene works with all version of CouchDB from 0.10 upwards.
6d1c074 Robert Newson add notes about version compatibility.
authored
6
9084540 Robert Newson add breaking changes section.
authored
7 <h1>Breaking Changes</h1>
8
9 <ul>
80b3ef9 Robert Newson update docs for impending 0.6.0 release
authored
10 <li>couchdb-lucene 0.5.x and higher runs as a standalone daemon (0.4 was run directly by couchdb).
9084540 Robert Newson add breaking changes section.
authored
11 <li>URL's now require the full design document id (where you would say "foo", you must now say "_design/foo").
12 </ul>
13
c6356fd Robert Newson update README.md and TODO to reflect progress.
authored
14 <h1>Issue Tracking</h1>
a785480 Robert Newson lighthouse sucks at formatting anything, abandon ship.
authored
15
7a0d1d3 Robert Newson lighthouse sucks at formatting anything, abandon ship.
authored
16 Issue tracking at <a href="http://github.com/rnewson/couchdb-lucene/issues">github</a>.
5d4e56a Robert Newson update readme.
authored
17
7dc5bac Robert Newson reorg begins.
authored
18 <h1>Minimum System Requirements</h1>
ef3f787 Robert Newson add sysreq for Sun JDK.
authored
19
0c6db37 Robert Newson make unzip requirement more obvious
authored
20 Java 1.5 (or above) is required; the <strike>Sun</strike> Oracle version is recommended as it's regularly tested against.
ef3f787 Robert Newson add sysreq for Sun JDK.
authored
21
0c6db37 Robert Newson make unzip requirement more obvious
authored
22 <h1>Build and run couchdb-lucene</h1>
b207965 Robert Newson improve README readability.
authored
23
0ac37b8 Robert Newson add note about brew recipe.
authored
24 If you are on OS X, you might find it easiest to;
25
26 <pre>
27 brew install couchdb-lucene
28 </pre>
29
b207965 Robert Newson improve README readability.
authored
30 <ol>
5a61a35 Matt Parker IF Maven 3 now works with CouchDB Lucene, update README to reflect that.
moonmaster9000 authored
31 <li>Install Maven (2 or 3).
b207965 Robert Newson improve README readability.
authored
32 <li>checkout repository
33 <li>type 'mvn'
0c6db37 Robert Newson make unzip requirement more obvious
authored
34 <li>cd target
2864de2 Robert Newson and escape html entities..
authored
35 <li>unzip couchdb-lucene-&lt;version&gt;.zip
36 <li>cd couchdb-lucene-&lt;version&gt;
0c6db37 Robert Newson make unzip requirement more obvious
authored
37 <li>./bin/run
b207965 Robert Newson improve README readability.
authored
38 </ol>
39
0c6db37 Robert Newson make unzip requirement more obvious
authored
40 The zip file contains all the couchdb-lucene code, dependencies, startup scripts and configuration files you need, so unzip it wherever you wish to install couchdb-lucene.
2d6180b Robert Newson include properties file in zip. still not available to run script, thoug...
authored
41
1c08058 Cliffano Subagio Added documentation about building a .war file.
cliffano authored
42 If you want to run couchdb-lucene on a servlet container like Tomcat, you can build the war file using Maven
43
44 <pre>
45 mvn war:war
46 </pre>
47
b207965 Robert Newson improve README readability.
authored
48 <h1>Configure CouchDB</h1>
49
0c6db37 Robert Newson make unzip requirement more obvious
authored
50 The following settings are needed in CouchDB's local.ini file in order for it to communicate with couchdb-lucene;
51
419ec28 Robert Newson enhance instructions for couchdb 1.1 proxy feature.
authored
52 <h2>For CouchDB versions prior to 1.1</h2>
b207965 Robert Newson improve README readability.
authored
53 <pre>
0563120 Robert Newson fixes.
authored
54 [couchdb]
55 os_process_timeout=60000 ; increase the timeout from 5 seconds.
56
b207965 Robert Newson improve README readability.
authored
57 [external]
5675f7a Robert Newson fix path
authored
58 fti=/path/to/python /path/to/couchdb-lucene/tools/couchdb-external-hook.py
b207965 Robert Newson improve README readability.
authored
59
60 [httpd_db_handlers]
95b38b9 Robert Newson more html escaping.
authored
61 _fti = {couch_httpd_external, handle_external_req, &lt;&lt;"fti"&gt;&gt;}
b207965 Robert Newson improve README readability.
authored
62 </pre>
63
419ec28 Robert Newson enhance instructions for couchdb 1.1 proxy feature.
authored
64 <h2>For CouchDB versions from 1.1 onward</h2>
65 <pre>
66 [httpd_global_handlers]
ab884f9 Robert Newson escape README.md
authored
67 _fti = {couch_httpd_proxy, handle_proxy_req, &lt;&lt;"http://127.0.0.1:5985"&gt;&gt;}
419ec28 Robert Newson enhance instructions for couchdb 1.1 proxy feature.
authored
68 </pre>
69
70 <b>Note:</b> The urls via the proxy have a different form:
71
72 <pre>http://127.0.0.1:5984/_fti/local/db1/_design/cl-test/idx?q=hello</pre>
73
74 The "local" matches the name of the key from <code>couchdb-lucene.ini</code>.
75
cbfe065 Robert Newson remove registry and indexkey, change output paths
authored
76 <h2>Hook options</h2>
77
78 <table>
79 <tr><th>Option</th><th>Meaning</th><th>Default Value</th></tr>
80 <tr><td>--remote-host</td><td>The hostname of the couchdb-lucene server</td><td>localhost</td></tr>
81 <tr><td>--remote-port</td><td>The port of the couchdb-lucene server</td><td>5985</td></tr>
c47c494 Robert Newson updated README to reflect latest Python scripts settings
authored
82 <tr><td>--local-key</td><td>The key for the local couchdb instance as known to the couchdb-lucene server</td><td>local</td></tr>
cbfe065 Robert Newson remove registry and indexkey, change output paths
authored
83 </table>
b047e4a Robert Newson update README.md to reflect (future) start/stop and config changes.
authored
84
cbfe065 Robert Newson remove registry and indexkey, change output paths
authored
85 <h1>Configure couchdb-lucene</h1>
b047e4a Robert Newson update README.md to reflect (future) start/stop and config changes.
authored
86
cbfe065 Robert Newson remove registry and indexkey, change output paths
authored
87 couchdb-lucene runs in a single, standalone JVM. As such, you can choose to locate your couchdb-lucene server on a different machine to couchdb if you wish, or keep it on the same machine, it's your call.
b047e4a Robert Newson update README.md to reflect (future) start/stop and config changes.
authored
88
89 <h1>Start couchdb-lucene</h1>
90
91 To start couchdb-lucene, run;
92 <pre>
2d6180b Robert Newson include properties file in zip. still not available to run script, thoug...
authored
93 bin/run
b047e4a Robert Newson update README.md to reflect (future) start/stop and config changes.
authored
94 </pre>
95
2d6180b Robert Newson include properties file in zip. still not available to run script, thoug...
authored
96 To stop couchdb-lucene, simply kill the Java process.
b047e4a Robert Newson update README.md to reflect (future) start/stop and config changes.
authored
97
b207965 Robert Newson improve README readability.
authored
98 <h1>Indexing Strategy</h1>
99
4a60080 Robert Newson use couchdb's content_type rather than auto-detect.
authored
100 <h2>Document Indexing</h2>
101
4c21806 Robert Newson mention that you can return Document[] from an index function.
authored
102 You must supply a index function in order to enable couchdb-lucene as, by default, nothing will be indexed. To suppress a document from the index, return null. It's more typical to return a single Document object which contains everything you'd like to query and retrieve. You may also return an array of Document objects if you wish.
a2e9024 Robert Newson wip
authored
103
437eae9 Robert Newson s/view/fulltext in README.md
authored
104 You may add any number of index views in any number of design documents. All searches will be constrained to documents emitted by the index functions.
c207a60 Robert Newson update README
authored
105
5077366 Robert Newson clarify design document and add matching query URL's.
authored
106 Here's an complete example of a design document with couchdb-lucene features:
a2e9024 Robert Newson wip
authored
107
697884b Robert Newson documentation of future features.
authored
108 <pre>
109 {
3d2fb72 Robert Newson fix example in README.
authored
110 "_id":"_design/foo",
5077366 Robert Newson clarify design document and add matching query URL's.
authored
111 "fulltext": {
112 "by_subject": {
113 "index":"function(doc) { var ret=new Document(); ret.add(doc.subject); return ret }"
114 },
115 "by_content": {
116 "index":"function(doc) { var ret=new Document(); ret.add(doc.content); return ret }"
117 }
697884b Robert Newson documentation of future features.
authored
118 }
119 }
120 </pre>
121
5077366 Robert Newson clarify design document and add matching query URL's.
authored
122 Here are some example URL's for the given design document;
123
124 <pre>
6b73dc6 Robert Newson commit to _design/foo in paths, use regexp to be sure.
authored
125 http://localhost:5984/database/_fti/_design/foo/by_subject?q=hello
126 http://localhost:5984/database/_fti/_design/foo/by_content?q=hello
5077366 Robert Newson clarify design document and add matching query URL's.
authored
127 </pre>
128
697884b Robert Newson documentation of future features.
authored
129 A fulltext object contains multiple index view declarations. An index view consists of;
130
131 <dl>
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
132 <dt>analyzer</dt><dd>(optional) The analyzer to use</dd>
133 <dt>defaults</dt><dd>(optional) The default for numerous indexing options can be overridden here. A full list of options follows.</dd>
697884b Robert Newson documentation of future features.
authored
134 <dt>index</dt><dd>The indexing function itself, documented below.</dd>
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
135 </dl>
697884b Robert Newson documentation of future features.
authored
136
137 <h3>The Defaults Object</h3>
138
139 The following indexing options can be defaulted;
140
141 <table>
142 <tr>
143 <th>name</th>
144 <th>description</th>
145 <th>available options</th>
146 <th>default</th>
147 </tr>
148 <tr>
a40523d Robert Newson documentation of future features.
authored
149 <th>field</th>
150 <td>the field name to index under</td>
151 <td>user-defined</td>
152 <td>default</td>
d2e1e9e Robert Newson add notes about typing.
authored
153 </tr>
154 <tr>
155 <th>type</th>
156 <td>the type of the field</td>
38ef320 Robert Newson fix documentation s/integer/int
authored
157 <td>date, double, float, int, long, string</td>
d2e1e9e Robert Newson add notes about typing.
authored
158 <td>string</td>
159 </tr>
a40523d Robert Newson documentation of future features.
authored
160 <tr>
697884b Robert Newson documentation of future features.
authored
161 <th>store</th>
f16fc9c Robert Newson docs
authored
162 <td>whether the data is stored. The value will be returned in the search result.</td>
697884b Robert Newson documentation of future features.
authored
163 <td>yes, no</td>
164 <td>no</td>
d2e1e9e Robert Newson add notes about typing.
authored
165 </tr>
697884b Robert Newson documentation of future features.
authored
166 <tr>
167 <th>index</th>
168 <td>whether (and how) the data is indexed</td>
8328332 Robert Newson typo
authored
169 <td>analyzed, analyzed_no_norms, no, not_analyzed, not_analyzed_no_norms</td>
697884b Robert Newson documentation of future features.
authored
170 <td>analyzed</td>
d2e1e9e Robert Newson add notes about typing.
authored
171 </tr>
f6bfce8 Robert Newson add EXPERIMENTAL index-time field boosting feature.
authored
172 <tr>
b00fb7f add termvector to the possible options for Field.
Santiago M. Mola authored
173 <th>termvector</th>
174 <td>whether and how a field should have term vectors</td>
175 <td>no, with_offsets, with_positions, with_positions_offsets, yes</td>
176 <td>no</td>
177 </tr>
178 <tr>
f6bfce8 Robert Newson add EXPERIMENTAL index-time field boosting feature.
authored
179 <th>boost</th>
180 <td>Sets the boost factor hits on this field. This value will be multiplied into the score of all hits on this this field of this document.</td>
181 <td>floating-point value</td>
182 <td>1.0</td>
183 </tr>
697884b Robert Newson documentation of future features.
authored
184 </table>
087dcec Robert Newson update documentation.
authored
185
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
186 <h3>The Analyzer Option</h3>
187
188 Lucene has numerous ways of converting free-form text into tokens, these classes are called Analyzer's. By default, the StandardAnalyzer is used which lower-cases all text, drops common English words ("the", "and", and so on), among other things. This processing might not always suit you, so you can choose from several others by setting the "analyzer" field to one of the following values;
189
190 <ul>
191 <li>brazilian</li>
192 <li>chinese</li>
193 <li>cjk</li>
194 <li>czech</li>
195 <li>dutch</li>
196 <li>english</li>
197 <li>french</li>
198 <li>german</li>
199 <li>keyword</li>
76d290d Robert Newson add perfield analyzer support
authored
200 <li>perfield</li>
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
201 <li>porter</li>
202 <li>russian</li>
203 <li>simple</li>
484bf40 Robert Newson add Snowball Analyzer.
authored
204 <li>snowball</li>
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
205 <li>standard</li>
206 <li>thai</li>
fe43cf0 Robert Newson add whitespace analyzer.
authored
207 <li>whitespace</li>
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
208 </ul>
209
484bf40 Robert Newson add Snowball Analyzer.
authored
210 <h4>The Snowball Analyzer</h4>
211
3124308 Robert Newson make link to Snowball Analyzer.
authored
212 This analyzer requires an extra argument to specify the language (see <a href="http://lucene.apache.org/java/3_0_3/api/contrib-snowball/org/apache/lucene/analysis/snowball/SnowballAnalyzer.html">here</a> for details);
484bf40 Robert Newson add Snowball Analyzer.
authored
213
214 <pre>
215 "analyzer":"snowball:English"
216 </pre>
217
218 Note: the argument is case-sensitive and is passed directly to the <code>SnowballAnalyzer</code>'s constructor.
219
220 <h4>The Per-field Analyzer"</h4>
221
76d290d Robert Newson add perfield analyzer support
authored
222 The "perfield" option lets you use a different analyzer for different fields and is configured as follows;
223
224 <pre>
dda9276 Robert Newson more README fixes.
authored
225 "analyzer":"perfield:{field_name:\"analyzer_name\"}"
76d290d Robert Newson add perfield analyzer support
authored
226 </pre>
227
86569b1 Robert Newson use 'default' consistently.
authored
228 Unless overridden, any field name not specified will be handled by the standard analyzer. To change the default, use the special default field name;
76d290d Robert Newson add perfield analyzer support
authored
229
230 <pre>
dda9276 Robert Newson more README fixes.
authored
231 "analyzer":"perfield:{default:\"keyword\"}"
76d290d Robert Newson add perfield analyzer support
authored
232 </pre>
233
087dcec Robert Newson update documentation.
authored
234 <h3>The Document class</h3>
235
236 You may construct a new Document instance with;
237
238 <pre>
239 var doc = new Document();
240 </pre>
241
a40523d Robert Newson documentation of future features.
authored
242 Data may be added to this document with the add method which takes an optional second object argument that can override any of the above default values.
087dcec Robert Newson update documentation.
authored
243
244 <pre>
a40523d Robert Newson documentation of future features.
authored
245 // Add with all the defaults.
246 doc.add("value");
247
d2e1e9e Robert Newson add notes about typing.
authored
248 // Add a numeric field.
38ef320 Robert Newson fix documentation s/integer/int
authored
249 doc.add(35, {"type":"int"});
d2e1e9e Robert Newson add notes about typing.
authored
250
4670582 Robert Newson use NumericField. currently broken.
authored
251 // Add a date field.
b4e054e Karel Minarik Added another example of valid format for date string in `date` field ty...
karmi authored
252 doc.add(new Date("1972/1/6 16:05:00"), {"type":"date"});
7f4e703 Robert Newson correct new Date(str) example and add matching test.
authored
253 doc.add(new Date("January 6, 1972 16:05:00"), {"type":"date"});
4670582 Robert Newson use NumericField. currently broken.
authored
254
d2e1e9e Robert Newson add notes about typing.
authored
255 // Add a date field (object must be a Date object
256
a40523d Robert Newson documentation of future features.
authored
257 // Add a subject field.
258 doc.add("this is the subject line.", {"field":"subject"});
9a71557 Robert Newson formatting
authored
259
a40523d Robert Newson documentation of future features.
authored
260 // Add but ensure it's stored.
261 doc.add("value", {"store":"yes"});
9a71557 Robert Newson formatting
authored
262
a40523d Robert Newson documentation of future features.
authored
263 // Add but don't analyze.
264 doc.add("don't analyze me", {"index":"not_analyzed"});
9a71557 Robert Newson formatting
authored
265
b0663ec Robert Newson correct misleading .attachment examples (but, hey, one of them was corre...
authored
266 // Extract text from the named attachment and index it to a named field
267 doc.attachment("attachment field", "attachment name");
9f99f19 Robert Newson add log.info example
authored
268
269 // log an event (trace, debug, info, warn and error are available)
270 if (doc.foo) {
271 log.info("doc has foo property!");
272 }
087dcec Robert Newson update documentation.
authored
273 </pre>
274
2d96db8 Robert Newson remove misleading 'transforms' term.
authored
275 <h3>Example Index Functions</h3>
ccb81a8 Robert Newson add example transforms section.
authored
276
390858a Robert Newson re-add Index Everything example.
authored
277 <h4>Index Everything</h4>
278
279 <pre>
280 function(doc) {
7bad7dc Robert Newson correct syntax error in JS fun.
authored
281 var ret = new Document();
282
283 function idx(obj) {
284 for (var key in obj) {
285 switch (typeof obj[key]) {
286 case 'object':
287 idx(obj[key]);
288 break;
289 case 'function':
290 break;
291 default:
292 ret.add(obj[key]);
293 break;
294 }
295 }
296 };
297
298 idx(doc);
299
300 if (doc._attachments) {
301 for (var i in doc._attachments) {
470171d Robert Newson change examples to index attachments into default field.
authored
302 ret.attachment("default", i);
7bad7dc Robert Newson correct syntax error in JS fun.
authored
303 }
390858a Robert Newson re-add Index Everything example.
authored
304 }
d2e1e9e Robert Newson add notes about typing.
authored
305
7bad7dc Robert Newson correct syntax error in JS fun.
authored
306 return ret;
390858a Robert Newson re-add Index Everything example.
authored
307 }
308 </pre>
309
ccb81a8 Robert Newson add example transforms section.
authored
310 <h4>Index Nothing</h4>
311
312 <pre>
313 function(doc) {
314 return null;
315 }
316 </pre>
317
c207a60 Robert Newson update README
authored
318 <h4>Index Select Fields</h4>
ccb81a8 Robert Newson add example transforms section.
authored
319
320 <pre>
321 function(doc) {
c207a60 Robert Newson update README
authored
322 var result = new Document();
c6356fd Robert Newson update README.md and TODO to reflect progress.
authored
323 result.add(doc.subject, {"field":"subject", "store":"yes"});
324 result.add(doc.content, {"field":"subject"});
5cfa20c Robert Newson fix error in example
authored
325 result.add(new Date(), {"field":"indexed_at"});
c207a60 Robert Newson update README
authored
326 return result;
ccb81a8 Robert Newson add example transforms section.
authored
327 }
328 </pre>
329
c207a60 Robert Newson update README
authored
330 <h4>Index Attachments</h4>
ccb81a8 Robert Newson add example transforms section.
authored
331
332 <pre>
333 function(doc) {
c207a60 Robert Newson update README
authored
334 var result = new Document();
335 for(var a in doc._attachments) {
470171d Robert Newson change examples to index attachments into default field.
authored
336 result.attachment("default", a);
ccb81a8 Robert Newson add example transforms section.
authored
337 }
c207a60 Robert Newson update README
authored
338 return result;
339 }
340 </pre>
341
342 <h4>A More Complex Example</h4>
343
344 <pre>
345 function(doc) {
346 var mk = function(name, value, group) {
c6356fd Robert Newson update README.md and TODO to reflect progress.
authored
347 var ret = new Document();
2946c9a Robert Newson fix example.
authored
348 ret.add(value, {"field": group, "store":"yes"});
c6356fd Robert Newson update README.md and TODO to reflect progress.
authored
349 ret.add(group, {"field":"group", "store":"yes"});
c207a60 Robert Newson update README
authored
350 return ret;
351 };
352 if(doc.type != "reference") return null;
a313b24 Robert Newson lots of stuff.
authored
353 var ret = new Array();
c207a60 Robert Newson update README
authored
354 for(var g in doc.groups) {
a313b24 Robert Newson lots of stuff.
authored
355 ret.push(mk("library", doc.groups[g].library, g));
356 ret.push(mk("method", doc.groups[g].method, g));
357 ret.push(mk("target", doc.groups[g].target, g));
c207a60 Robert Newson update README
authored
358 }
359 return ret;
360 }
361 </pre>
b207965 Robert Newson improve README readability.
authored
362
4a60080 Robert Newson use couchdb's content_type rather than auto-detect.
authored
363 <h2>Attachment Indexing</h2>
364
8059ce0 Robert Newson s/couchdb/couchdb-lucene
authored
365 Couchdb-lucene uses <a href="http://lucene.apache.org/tika/">Apache Tika</a> to index attachments of the following types, assuming the correct content_type is set in couchdb;
4a60080 Robert Newson use couchdb's content_type rather than auto-detect.
authored
366
ec94e21 Robert Newson updated README.md
authored
367 <h3>Supported Formats</h3>
368
4a60080 Robert Newson use couchdb's content_type rather than auto-detect.
authored
369 <ul>
370 <li>Excel spreadsheets (application/vnd.ms-excel)
371 <li>HTML (text/html)
372 <li>Images (image/*)
373 <li>Java class files
374 <li>Java jar archives
375 <li>MP3 (audio/mp3)
376 <li>OpenDocument (application/vnd.oasis.opendocument.*)
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
377 <li>Outlook (application/vnd.ms-outlook)
4a60080 Robert Newson use couchdb's content_type rather than auto-detect.
authored
378 <li>PDF (application/pdf)
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
379 <li>Plain text (text/plain)
380 <li>Powerpoint presentations (application/vnd.ms-powerpoint)
4a60080 Robert Newson use couchdb's content_type rather than auto-detect.
authored
381 <li>RTF (application/rtf)
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
382 <li>Visio (application/vnd.visio)
383 <li>Word documents (application/msword)
384 <li>XML (application/xml)
4a60080 Robert Newson use couchdb's content_type rather than auto-detect.
authored
385 </ul>
386
b207965 Robert Newson improve README readability.
authored
387 <h1>Searching with couchdb-lucene</h1>
388
c0c534a Robert Newson Prep notes for 0.8.0 release
authored
389 You can perform all types of queries using Lucene's default <a href="http://lucene.apache.org/java/3_5_0/queryparsersyntax.html">query syntax</a>.
2b14b76 Robert Newson describ syntax for all forms of numeric range query
authored
390
391 <h2>Numeric range queries</h2>
392
e0c2000 Robert Newson fix grammar, thanks tisba.
authored
393 In addition to normal text-based range searches (using the "field:[lower TO upper]" syntax), couchdb-lucene also supports numeric range searches for the following types: int, long, float, double and date. The type is specified after the field name, as follows;
2b14b76 Robert Newson describ syntax for all forms of numeric range query
authored
394
395 <table>
bf989bc Robert Newson make type of range query explicit.
authored
396 <tr><td>type</td><td>example</td></tr>
38ef320 Robert Newson fix documentation s/integer/int
authored
397 <tr><td>int</td><td>field&lt;int>:[0 TO 100]</td></tr>
20c9be3 Robert Newson update docs on numeric range queries
authored
398 <tr><td>long</td><td>field&lt;long>:[0 TO 100]</td></tr>
399 <tr><td>float</td><td>field&lt;float>:[0.0 TO 100.0]</td></tr>
400 <tr><td>double</td><td>field&lt;double>:[0.0 TO 100.0]</td></tr>
c79d771 Robert Newson more README fixes.
authored
401 <tr><td>date</td><td>field&lt;date>:[from TO to] where from and to match any of these patterns: <code>"yyyy-MM-dd'T'HH:mm:ssZ"</code>, <code>"yyyy-MM-dd'T'HH:mm:ss"<code>, <code>"yyyy-MM-ddZ"M/code>, <code>"yyyy-MM-dd"</code>, <code>"yyyy-MM-dd'T'HH:mm:ss.SSSZ"</code>, <code>"yyyy-MM-dd'T'HH:mm:ss.SSS"</code>. So, in order to search for articles published in July, you would issue a following query: <code>published_at&lt;date&gt;:["2010-07-01T00:00:00"+TO+"2010-07-31T23:59:59"]</code></td></tr>
2b14b76 Robert Newson describ syntax for all forms of numeric range query
authored
402 </table>
403
460f5c6 Robert Newson add example spatial/geographical query
authored
404 An example numeric range query for spatial searching.
405
406 <pre>
1217f9f Robert Newson update docs on numeric range queries
authored
407 ?q=pizza AND lat&lt;double>:[51.4707 TO 51.5224] AND long&lt;double>:[-0.6622 TO -0.5775]
460f5c6 Robert Newson add example spatial/geographical query
authored
408 </pre>
409
fad4eb1 Robert Newson document the syntax for numeric terms.
authored
410 <h2>Numeric term queries</h2>
411
15de3b7 Robert Newson escape some <>'s
authored
412 Fields indexed with numeric types can still be queried as normal terms, couchdb-lucene just needs to know the type. For example, ?q=age&lt;long&gt;:12 will find all documents where the field called 'age' has a value of 12 (when the field was indexed as "type":"int".
fad4eb1 Robert Newson document the syntax for numeric terms.
authored
413
414 <h2>Search parameters</h2>
415
2b14b76 Robert Newson describ syntax for all forms of numeric range query
authored
416 The following parameters can be passed for more sophisticated searches;
b207965 Robert Newson improve README readability.
authored
417
418 <dl>
70c8bf6 Robert Newson document new analyzer query parameter
authored
419 <dt>analyzer</dt><dd>Override the default analyzer used to parse the q parameter</dd>
420 <dt>callback</dt><dd>Specify a JSONP callback wrapper. The full JSON result will be prepended with this parameter and also placed with parentheses."</dd>
71cbc0b Robert Newson correct description of debug setting.
authored
421 <dt>debug</dt><dd>Setting this to true disables response caching (the query is executed every time) and indents the JSON response for readability.</dd>
bba040b Robert Newson allow default_operator=and. Closes #95
authored
422 <dt>default_operator</dt><dd>Change the default operator for boolean queries. Defaults to "OR", other permitted value is "AND".</dd>
ea19e55 Robert Newson add force_json=true to force 'application/json' response type.
authored
423 <dt>force_json<dt><dd>Usually couchdb-lucene determines the Content-Type of its response based on the presence of the Accept header. If Accept contains "application/json", you get "application/json" in the response, otherwise you get "text/plain;charset=utf8". Some tools, like JSONView for FireFox, do not send the Accept header but do render "application/json" responses if received. Setting force_json=true forces all response to "application/json" regardless of the Accept header.</dd>
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
424 <dt>include_docs</dt><dd>whether to include the source docs</dd>
c9d4f76 Robert Newson Add include_fields option to return a chosen subset of stored fields. (c...
authored
425 <dt>include_fields</dt><dd>By default, <i>all</i> stored fields are returned with results. Use a comma-separate list of field names with this parameter to refine the response</dd>
8194a9a nesteffe Update documentation to reflect new highlighter features and result set ...
nesteffe authored
426 <dt>include_termvectors</dt><dd>Return term vectors with the results. Default is <i>false</i>.</dd>
427 <dt>highlights</dt><dd>Number of highlights to include with results. Default is <i>0</i>. This uses the <i>fast-vector-highlighter</i> plugin.</dd>
428 <dt>highlight_length</dt><dd>Number of characters to include in a highlight row. Default and minimum is <i>18</i>.</dd>
f9c61e3 Robert Newson format README
authored
429 <dt>limit</dt><dd>the maximum number of results to return</dd>
caccea4 Robert Newson avoid JSON object encoding problem by changing multiple query feature sy...
authored
430 <dt>q</dt><dd>the query to run (e.g, subject:hello). If not specified, the default field is searched. Multiple queries can be supplied, separated by commas; the resulting JSON will be an array of responses.</dd>
f9c61e3 Robert Newson format README
authored
431 <dt>skip</dt><dd>the number of results to skip</dd>
81ea4c1 Robert Newson more escapin'
authored
432 <dt>sort</dt><dd>the comma-separated fields to sort on. Prefix with / for ascending order and \ for descending order (ascending is the default if not specified). Type-specific sorting is also available by appending the type between angle brackets (e.g, sort=amount&lt;float&gt;). Supported types are 'float', 'double', 'int', 'long' and 'date'.</dd>
1ddad7b Sebastian Cohnen Update the README to reflect current behavior of
tisba authored
433 <dt>stale=ok</dt><dd>If you set the <i>stale</i> option to <i>ok</i>, couchdb-lucene will not block if the index is not up to date and it will immediately return results. Therefore searches may be faster as Lucene caches important data (especially for sorting). A query without stale=ok will block and use the latest data committed to the index. Unlike CouchDBs stale=ok option for views, couchdb-lucene will trigger an index update unless one is already running.</dd>
ad9096f Robert Newson tweak README.md
authored
434 </dl>
b207965 Robert Newson improve README readability.
authored
435
436 <i>All parameters except 'q' are optional.</i>
437
ec94e21 Robert Newson updated README.md
authored
438 <h2>Special Fields</h2>
439
440 <dl>
087dcec Robert Newson update documentation.
authored
441 <dt>_id</dt><dd>The _id of the document.</dd>
46a3a37 Robert Newson include all DC attributes, if present.
authored
442 </dl>
443
444 <h2>Dublin Core</h2>
445
446 All Dublin Core attributes are indexed and stored if detected in the attachment. Descriptions of the fields come from the Tika javadocs.
447
448 <dl>
6e99faa Robert Newson dc. to _dc.
authored
449 <dt>_dc.contributor</dt><dd> An entity responsible for making contributions to the content of the resource.</dd>
450 <dt>_dc.coverage</dt><dd>The extent or scope of the content of the resource.</dd>
451 <dt>_dc.creator</dt><dd>An entity primarily responsible for making the content of the resource.</dd>
452 <dt>_dc.date</dt><dd>A date associated with an event in the life cycle of the resource.</dd>
453 <dt>_dc.description</dt><dd>An account of the content of the resource.</dd>
454 <dt>_dc.format</dt><dd>Typically, Format may include the media-type or dimensions of the resource.</dd>
455 <dt>_dc.identifier</dt><dd>Recommended best practice is to identify the resource by means of a string or number conforming to a formal identification system.</dd>
456 <dt>_dc.language</dt><dd>A language of the intellectual content of the resource.</dd>
457 <dt>_dc.modified</dt><dd>Date on which the resource was changed.</dd>
458 <dt>_dc.publisher</dt><dd>An entity responsible for making the resource available.</dd>
459 <dt>_dc.relation</dt><dd>A reference to a related resource.</dd>
460 <dt>_dc.rights</dt><dd>Information about rights held in and over the resource.</dd>
461 <dt>_dc.source</dt><dd>A reference to a resource from which the present resource is derived.</dd>
462 <dt>_dc.subject</dt><dd>The topic of the content of the resource.</dd>
463 <dt>_dc.title</dt><dd>A name given to the resource.</dd>
464 <dt>_dc.type</dt><dd>The nature or genre of the content of the resource.</dd>
ec94e21 Robert Newson updated README.md
authored
465 </dl>
466
b207965 Robert Newson improve README readability.
authored
467 <h2>Examples</h2>
468
469 <pre>
6b73dc6 Robert Newson commit to _design/foo in paths, use regexp to be sure.
authored
470 http://localhost:5984/dbname/_fti/_design/foo/view_name?q=field_name:value
471 http://localhost:5984/dbname/_fti/_design/foo/view_name?q=field_name:value&sort=other_field
4ae6c50 Stefan Kögl <int> type specifier belongs to the field, not the "sort" parameter
stefankoegl authored
472 http://localhost:5984/dbname/_fti/_design/foo/view_name?debug=true&sort=billing_size&lt;long&gt;&q=body:document AND customer:[A TO C]
b207965 Robert Newson improve README readability.
authored
473 </pre>
474
475 <h2>Search Results Format</h2>
476
0fcf578 Robert Newson update docs.
authored
477 The search result contains a number of fields at the top level, in addition to your search results.
478
479 <dl>
480 <dt>etag</dt><dd>An opaque token that reflects the current version of the index. This value is also returned in an ETag header to facilitate HTTP caching.</dd>
481 <dt>fetch_duration</dt><dd>The number of milliseconds spent retrieving the documents.</dd>
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
482 <dt>limit</dt><dd>The maximum number of results that can appear.</dd>
483 <dt>q</dt><dd>The query that was executed.</dd>
24591d9 Robert Newson docs
authored
484 <dt>rows</dt><dd>The search results array, described below.</dd>
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
485 <dt>search_duration</dt><dd>The number of milliseconds spent performing the search.</dd>
486 <dt>skip</dt><dd>The number of initial matches that was skipped.</dd>
487 <dt>total_rows</dt><dd>The total number of matches for this query.</dd>
0fcf578 Robert Newson update docs.
authored
488 </dl>
489
24591d9 Robert Newson docs
authored
490 <h2>The search results array</h2>
491
492 The search results arrays consists of zero, one or more objects with the following fields;
0fcf578 Robert Newson update docs.
authored
493
494 <dl>
f420bf7 Robert Newson support different Analyzer's at index and query time.
authored
495 <dt>doc</dt><dd>The original document from couch, if requested with include_docs=true</dd>
496 <dt>fields</dt><dd>All the fields that were stored with this match</dd>
0fcf578 Robert Newson update docs.
authored
497 <dt>id</dt><dd>The unique identifier for this match.</dd>
498 <dt>score</dt><dd>The normalized score (0.0-1.0, inclusive) for this match</dd>
499 </dl>
500
fd16315 Robert Newson update README.md
authored
501 Here's an example of a JSON response without sorting;
b207965 Robert Newson improve README readability.
authored
502
118d28e Robert Newson JSON example output.
authored
503 <pre>
504 {
c6356fd Robert Newson update README.md and TODO to reflect progress.
authored
505 "q": "+content:enron",
fd16315 Robert Newson update README.md
authored
506 "skip": 0,
507 "limit": 2,
508 "total_rows": 176852,
509 "search_duration": 518,
510 "fetch_duration": 4,
511 "rows": [
512 {
0fcf578 Robert Newson update docs.
authored
513 "id": "hain-m-all_documents-257.",
fd16315 Robert Newson update README.md
authored
514 "score": 1.601625680923462
515 },
516 {
0fcf578 Robert Newson update docs.
authored
517 "id": "hain-m-notes_inbox-257.",
fd16315 Robert Newson update README.md
authored
518 "score": 1.601625680923462
519 }
118d28e Robert Newson JSON example output.
authored
520 ]
521 }
522 </pre>
523
fd16315 Robert Newson update README.md
authored
524 And the same with sorting;
525
118d28e Robert Newson JSON example output.
authored
526 <pre>
527 {
0fcf578 Robert Newson update docs.
authored
528 "q": "+content:enron",
fd16315 Robert Newson update README.md
authored
529 "skip": 0,
530 "limit": 3,
531 "total_rows": 176852,
532 "search_duration": 660,
533 "fetch_duration": 4,
534 "sort_order": [
535 {
536 "field": "source",
537 "reverse": false,
538 "type": "string"
539 },
540 {
541 "reverse": false,
542 "type": "doc"
543 }
118d28e Robert Newson JSON example output.
authored
544 ],
fd16315 Robert Newson update README.md
authored
545 "rows": [
546 {
0fcf578 Robert Newson update docs.
authored
547 "id": "shankman-j-inbox-105.",
fd16315 Robert Newson update README.md
authored
548 "score": 0.6131107211112976,
549 "sort_order": [
550 "enron",
551 6
552 ]
553 },
554 {
0fcf578 Robert Newson update docs.
authored
555 "id": "shankman-j-inbox-8.",
fd16315 Robert Newson update README.md
authored
556 "score": 0.7492915391921997,
557 "sort_order": [
558 "enron",
559 7
560 ]
561 },
562 {
0fcf578 Robert Newson update docs.
authored
563 "id": "shankman-j-inbox-30.",
fd16315 Robert Newson update README.md
authored
564 "score": 0.507369875907898,
565 "sort_order": [
566 "enron",
567 8
568 ]
569 }
118d28e Robert Newson JSON example output.
authored
570 ]
571 }
572 </pre>
573
a4aa4e1 Robert Newson document Content-Type response negotiation
authored
574 <h3>Content-Type of response</h3>
575
d2e1e9e Robert Newson add notes about typing.
authored
576 The Content-Type of the response is negotiated via the Accept request header like CouchDB itself. If the Accept header includes "application/json" then that is also the Content-Type of the response. If not, "text/plain;charset=utf-8" is used.
a4aa4e1 Robert Newson document Content-Type response negotiation
authored
577
139a78c Robert Newson add info retrieval.
authored
578 <h1>Fetching information about the index</h1>
579
0cf4941 Robert Newson correct info urls.
authored
580 Calling couchdb-lucene without arguments returns a JSON object with information about the index.
139a78c Robert Newson add info retrieval.
authored
581
582 <pre>
0b94022 Robert Newson add missing gt;
authored
583 http://127.0.0.1:5984/&lt;db>/_fti/_design/foo/&lt;index&gt;
139a78c Robert Newson add info retrieval.
authored
584 </pre>
585
586 returns;
587
588 <pre>
f6ac048 Robert Newson line breaks
authored
589 {"current":true,"disk_size":110674,"doc_count":397,"doc_del_count":0,
590 "fields":["default","number"],"last_modified":"1263066382000",
591 "optimized":true,"ref_count":2}
139a78c Robert Newson add info retrieval.
authored
592 </pre>
8203af6 Robert Newson support _optimize and _expunge calls.
authored
593
594 <h1>Index Maintenance</h1>
595
596 For optimal query speed you can optimize your indexes. This causes the index to be rewritten into a single segment.
597
598 <pre>
6b73dc6 Robert Newson commit to _design/foo in paths, use regexp to be sure.
authored
599 curl -X POST http://localhost:5984/&lt;db>/_fti/_design/foo/&lt;index>/_optimize
8203af6 Robert Newson support _optimize and _expunge calls.
authored
600 </pre>
601
602 If you just want to expunge pending deletes, then call;
603
604 <pre>
6b73dc6 Robert Newson commit to _design/foo in paths, use regexp to be sure.
authored
605 curl -X POST http://localhost:5984/&lt;db>/_fti/_design/foo/&lt;index>/_expunge
8203af6 Robert Newson support _optimize and _expunge calls.
authored
606 </pre>
6e66766 Robert Newson most of index cleanup work
authored
607
608 If you recreate databases or frequently change your fulltext functions, you will probably have old indexes lying around on disk. To remove all of them, call;
609
610 <pre>
611 curl -X POST http://localhost:5984/&lt;db>/_fti/_cleanup
612 </pre>
378e322 Robert Newson add basic authentication for require_valid_user=true users.
authored
613
614 <h1>Authentication</h1>
615
616 By default couchdb-lucene does not attempt to authenticate to CouchDB. If you have set CouchDB's require_valid_user to true, you will need to modify couchdb-lucene.ini. Change the url setting to include a valid username and password. e.g, the default setting is;
617
618 <pre>
619 [local]
620 url=http://localhost:5984/
621 </pre>
622
623 Change it to;
624
625 <pre>
626 [local]
627 url=http://foo:bar@localhost:5984/
628 </pre>
629
630 and couchdb-lucene will authenticate to couchdb.
958d492 Robert Newson Document some new options
authored
631
632 <h1>Other Tricks</h1>
633
634 A couple of 'expert' options can be set in the couchdb-lucene.ini file;
635
636 Leading wildcards are prohibited by default as they perform very poorly most of the time. You can enable them as follows;
637
638 <pre>
639 [lucene]
640 allowLeadingWildcard=true
641 </pre>
642
643 Lucene automatically converts terms to lower case in wildcard situations. You can disable this with;
644
645 <pre>
646 [lucene]
647 lowercaseExpandedTerms=false
648 </pre>
Something went wrong with that request. Please try again.