Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 631 lines (493 sloc) 21.353 kb
6d1c074 @rnewson add notes about version compatibility.
authored
1 <h1>Version Compatibility</h1>
2
3 <table>
4 <tr><th>CouchDB</th><th>couchdb-lucene</th></tr>
5 <tr><td>0.9.1, 0.10</td><td>0.4</td></tr>
5cd5577 @rnewson 0.11 is now released.
authored
6 <tr><td>0.11</td><td>0.4-maint (0.4 with patch for trunk compatibility)</td></tr>
bc1bd8f @rnewson extend version compatibility matrix
authored
7 <tr><td>0.10.x, 0.11.x, 1.0.x, 1.1.x</td><td>0.5.x, 0.6.x, 0.7.x</td></tr>
6d1c074 @rnewson add notes about version compatibility.
authored
8 </table>
9
9084540 @rnewson add breaking changes section.
authored
10 <h1>Breaking Changes</h1>
11
12 <ul>
80b3ef9 @rnewson update docs for impending 0.6.0 release
authored
13 <li>couchdb-lucene 0.5.x and higher runs as a standalone daemon (0.4 was run directly by couchdb).
9084540 @rnewson add breaking changes section.
authored
14 <li>URL's now require the full design document id (where you would say "foo", you must now say "_design/foo").
15 </ul>
16
c6356fd @rnewson update README.md and TODO to reflect progress.
authored
17 <h1>Issue Tracking</h1>
a785480 @rnewson lighthouse sucks at formatting anything, abandon ship.
authored
18
7a0d1d3 @rnewson lighthouse sucks at formatting anything, abandon ship.
authored
19 Issue tracking at <a href="http://github.com/rnewson/couchdb-lucene/issues">github</a>.
5d4e56a @rnewson update readme.
authored
20
7dc5bac @rnewson reorg begins.
authored
21 <h1>Minimum System Requirements</h1>
ef3f787 @rnewson add sysreq for Sun JDK.
authored
22
0c6db37 @rnewson make unzip requirement more obvious
authored
23 Java 1.5 (or above) is required; the <strike>Sun</strike> Oracle version is recommended as it's regularly tested against.
ef3f787 @rnewson add sysreq for Sun JDK.
authored
24
0c6db37 @rnewson make unzip requirement more obvious
authored
25 <h1>Build and run couchdb-lucene</h1>
b207965 @rnewson improve README readability.
authored
26
0ac37b8 @rnewson add note about brew recipe.
authored
27 If you are on OS X, you might find it easiest to;
28
29 <pre>
30 brew install couchdb-lucene
31 </pre>
32
b207965 @rnewson improve README readability.
authored
33 <ol>
34 <li>Install Maven 2.
35 <li>checkout repository
36 <li>type 'mvn'
0c6db37 @rnewson make unzip requirement more obvious
authored
37 <li>cd target
2864de2 @rnewson and escape html entities..
authored
38 <li>unzip couchdb-lucene-&lt;version&gt;.zip
39 <li>cd couchdb-lucene-&lt;version&gt;
0c6db37 @rnewson make unzip requirement more obvious
authored
40 <li>./bin/run
b207965 @rnewson improve README readability.
authored
41 </ol>
42
0c6db37 @rnewson make unzip requirement more obvious
authored
43 The zip file contains all the couchdb-lucene code, dependencies, startup scripts and configuration files you need, so unzip it wherever you wish to install couchdb-lucene.
2d6180b @rnewson include properties file in zip. still not available to run script, th…
authored
44
1c08058 @cliffano Added documentation about building a .war file.
cliffano authored
45 If you want to run couchdb-lucene on a servlet container like Tomcat, you can build the war file using Maven
46
47 <pre>
48 mvn war:war
49 </pre>
50
b207965 @rnewson improve README readability.
authored
51 <h1>Configure CouchDB</h1>
52
0c6db37 @rnewson make unzip requirement more obvious
authored
53 The following settings are needed in CouchDB's local.ini file in order for it to communicate with couchdb-lucene;
54
419ec28 @rnewson enhance instructions for couchdb 1.1 proxy feature.
authored
55 <h2>For CouchDB versions prior to 1.1</h2>
b207965 @rnewson improve README readability.
authored
56 <pre>
0563120 @rnewson fixes.
authored
57 [couchdb]
58 os_process_timeout=60000 ; increase the timeout from 5 seconds.
59
b207965 @rnewson improve README readability.
authored
60 [external]
5675f7a @rnewson fix path
authored
61 fti=/path/to/python /path/to/couchdb-lucene/tools/couchdb-external-hook.py
b207965 @rnewson improve README readability.
authored
62
63 [httpd_db_handlers]
95b38b9 @rnewson more html escaping.
authored
64 _fti = {couch_httpd_external, handle_external_req, &lt;&lt;"fti"&gt;&gt;}
b207965 @rnewson improve README readability.
authored
65 </pre>
66
419ec28 @rnewson enhance instructions for couchdb 1.1 proxy feature.
authored
67 <h2>For CouchDB versions from 1.1 onward</h2>
68 <pre>
69 [httpd_global_handlers]
ab884f9 @rnewson escape README.md
authored
70 _fti = {couch_httpd_proxy, handle_proxy_req, &lt;&lt;"http://127.0.0.1:5985"&gt;&gt;}
419ec28 @rnewson enhance instructions for couchdb 1.1 proxy feature.
authored
71 </pre>
72
73 <b>Note:</b> The urls via the proxy have a different form:
74
75 <pre>http://127.0.0.1:5984/_fti/local/db1/_design/cl-test/idx?q=hello</pre>
76
77 The "local" matches the name of the key from <code>couchdb-lucene.ini</code>.
78
cbfe065 @rnewson remove registry and indexkey, change output paths
authored
79 <h2>Hook options</h2>
80
81 <table>
82 <tr><th>Option</th><th>Meaning</th><th>Default Value</th></tr>
83 <tr><td>--remote-host</td><td>The hostname of the couchdb-lucene server</td><td>localhost</td></tr>
84 <tr><td>--remote-port</td><td>The port of the couchdb-lucene server</td><td>5985</td></tr>
c47c494 @rnewson updated README to reflect latest Python scripts settings
authored
85 <tr><td>--local-key</td><td>The key for the local couchdb instance as known to the couchdb-lucene server</td><td>local</td></tr>
cbfe065 @rnewson remove registry and indexkey, change output paths
authored
86 </table>
b047e4a @rnewson update README.md to reflect (future) start/stop and config changes.
authored
87
cbfe065 @rnewson remove registry and indexkey, change output paths
authored
88 <h1>Configure couchdb-lucene</h1>
b047e4a @rnewson update README.md to reflect (future) start/stop and config changes.
authored
89
cbfe065 @rnewson remove registry and indexkey, change output paths
authored
90 couchdb-lucene runs in a single, standalone JVM. As such, you can choose to locate your couchdb-lucene server on a different machine to couchdb if you wish, or keep it on the same machine, it's your call.
b047e4a @rnewson update README.md to reflect (future) start/stop and config changes.
authored
91
92 <h1>Start couchdb-lucene</h1>
93
94 To start couchdb-lucene, run;
95 <pre>
2d6180b @rnewson include properties file in zip. still not available to run script, th…
authored
96 bin/run
b047e4a @rnewson update README.md to reflect (future) start/stop and config changes.
authored
97 </pre>
98
2d6180b @rnewson include properties file in zip. still not available to run script, th…
authored
99 To stop couchdb-lucene, simply kill the Java process.
b047e4a @rnewson update README.md to reflect (future) start/stop and config changes.
authored
100
b207965 @rnewson improve README readability.
authored
101 <h1>Indexing Strategy</h1>
102
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
103 <h2>Document Indexing</h2>
104
4c21806 @rnewson mention that you can return Document[] from an index function.
authored
105 You must supply a index function in order to enable couchdb-lucene as, by default, nothing will be indexed. To suppress a document from the index, return null. It's more typical to return a single Document object which contains everything you'd like to query and retrieve. You may also return an array of Document objects if you wish.
a2e9024 @rnewson wip
authored
106
437eae9 @rnewson s/view/fulltext in README.md
authored
107 You may add any number of index views in any number of design documents. All searches will be constrained to documents emitted by the index functions.
c207a60 @rnewson update README
authored
108
5077366 @rnewson clarify design document and add matching query URL's.
authored
109 Here's an complete example of a design document with couchdb-lucene features:
a2e9024 @rnewson wip
authored
110
697884b @rnewson documentation of future features.
authored
111 <pre>
112 {
3d2fb72 @rnewson fix example in README.
authored
113 "_id":"_design/foo",
5077366 @rnewson clarify design document and add matching query URL's.
authored
114 "fulltext": {
115 "by_subject": {
116 "index":"function(doc) { var ret=new Document(); ret.add(doc.subject); return ret }"
117 },
118 "by_content": {
119 "index":"function(doc) { var ret=new Document(); ret.add(doc.content); return ret }"
120 }
697884b @rnewson documentation of future features.
authored
121 }
122 }
123 </pre>
124
5077366 @rnewson clarify design document and add matching query URL's.
authored
125 Here are some example URL's for the given design document;
126
127 <pre>
6b73dc6 @rnewson commit to _design/foo in paths, use regexp to be sure.
authored
128 http://localhost:5984/database/_fti/_design/foo/by_subject?q=hello
129 http://localhost:5984/database/_fti/_design/foo/by_content?q=hello
5077366 @rnewson clarify design document and add matching query URL's.
authored
130 </pre>
131
697884b @rnewson documentation of future features.
authored
132 A fulltext object contains multiple index view declarations. An index view consists of;
133
134 <dl>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
135 <dt>analyzer</dt><dd>(optional) The analyzer to use</dd>
136 <dt>defaults</dt><dd>(optional) The default for numerous indexing options can be overridden here. A full list of options follows.</dd>
697884b @rnewson documentation of future features.
authored
137 <dt>index</dt><dd>The indexing function itself, documented below.</dd>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
138 </dl>
697884b @rnewson documentation of future features.
authored
139
140 <h3>The Defaults Object</h3>
141
142 The following indexing options can be defaulted;
143
144 <table>
145 <tr>
146 <th>name</th>
147 <th>description</th>
148 <th>available options</th>
149 <th>default</th>
150 </tr>
151 <tr>
a40523d @rnewson documentation of future features.
authored
152 <th>field</th>
153 <td>the field name to index under</td>
154 <td>user-defined</td>
155 <td>default</td>
d2e1e9e @rnewson add notes about typing.
authored
156 </tr>
157 <tr>
158 <th>type</th>
159 <td>the type of the field</td>
38ef320 @rnewson fix documentation s/integer/int
authored
160 <td>date, double, float, int, long, string</td>
d2e1e9e @rnewson add notes about typing.
authored
161 <td>string</td>
162 </tr>
a40523d @rnewson documentation of future features.
authored
163 <tr>
697884b @rnewson documentation of future features.
authored
164 <th>store</th>
f16fc9c @rnewson docs
authored
165 <td>whether the data is stored. The value will be returned in the search result.</td>
697884b @rnewson documentation of future features.
authored
166 <td>yes, no</td>
167 <td>no</td>
d2e1e9e @rnewson add notes about typing.
authored
168 </tr>
697884b @rnewson documentation of future features.
authored
169 <tr>
170 <th>index</th>
171 <td>whether (and how) the data is indexed</td>
8328332 @rnewson typo
authored
172 <td>analyzed, analyzed_no_norms, no, not_analyzed, not_analyzed_no_norms</td>
697884b @rnewson documentation of future features.
authored
173 <td>analyzed</td>
d2e1e9e @rnewson add notes about typing.
authored
174 </tr>
f6bfce8 @rnewson add EXPERIMENTAL index-time field boosting feature.
authored
175 <tr>
b00fb7f add termvector to the possible options for Field.
Santiago M. Mola authored
176 <th>termvector</th>
177 <td>whether and how a field should have term vectors</td>
178 <td>no, with_offsets, with_positions, with_positions_offsets, yes</td>
179 <td>no</td>
180 </tr>
181 <tr>
f6bfce8 @rnewson add EXPERIMENTAL index-time field boosting feature.
authored
182 <th>boost</th>
183 <td>Sets the boost factor hits on this field. This value will be multiplied into the score of all hits on this this field of this document.</td>
184 <td>floating-point value</td>
185 <td>1.0</td>
186 </tr>
697884b @rnewson documentation of future features.
authored
187 </table>
087dcec @rnewson update documentation.
authored
188
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
189 <h3>The Analyzer Option</h3>
190
191 Lucene has numerous ways of converting free-form text into tokens, these classes are called Analyzer's. By default, the StandardAnalyzer is used which lower-cases all text, drops common English words ("the", "and", and so on), among other things. This processing might not always suit you, so you can choose from several others by setting the "analyzer" field to one of the following values;
192
193 <ul>
194 <li>brazilian</li>
195 <li>chinese</li>
196 <li>cjk</li>
197 <li>czech</li>
198 <li>dutch</li>
199 <li>english</li>
200 <li>french</li>
201 <li>german</li>
202 <li>keyword</li>
76d290d @rnewson add perfield analyzer support
authored
203 <li>perfield</li>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
204 <li>porter</li>
205 <li>russian</li>
206 <li>simple</li>
484bf40 @rnewson add Snowball Analyzer.
authored
207 <li>snowball</li>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
208 <li>standard</li>
209 <li>thai</li>
fe43cf0 @rnewson add whitespace analyzer.
authored
210 <li>whitespace</li>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
211 </ul>
212
484bf40 @rnewson add Snowball Analyzer.
authored
213 <h4>The Snowball Analyzer</h4>
214
3124308 @rnewson make link to Snowball Analyzer.
authored
215 This analyzer requires an extra argument to specify the language (see <a href="http://lucene.apache.org/java/3_0_3/api/contrib-snowball/org/apache/lucene/analysis/snowball/SnowballAnalyzer.html">here</a> for details);
484bf40 @rnewson add Snowball Analyzer.
authored
216
217 <pre>
218 "analyzer":"snowball:English"
219 </pre>
220
221 Note: the argument is case-sensitive and is passed directly to the <code>SnowballAnalyzer</code>'s constructor.
222
223 <h4>The Per-field Analyzer"</h4>
224
76d290d @rnewson add perfield analyzer support
authored
225 The "perfield" option lets you use a different analyzer for different fields and is configured as follows;
226
227 <pre>
dda9276 @rnewson more README fixes.
authored
228 "analyzer":"perfield:{field_name:\"analyzer_name\"}"
76d290d @rnewson add perfield analyzer support
authored
229 </pre>
230
86569b1 @rnewson use 'default' consistently.
authored
231 Unless overridden, any field name not specified will be handled by the standard analyzer. To change the default, use the special default field name;
76d290d @rnewson add perfield analyzer support
authored
232
233 <pre>
dda9276 @rnewson more README fixes.
authored
234 "analyzer":"perfield:{default:\"keyword\"}"
76d290d @rnewson add perfield analyzer support
authored
235 </pre>
236
087dcec @rnewson update documentation.
authored
237 <h3>The Document class</h3>
238
239 You may construct a new Document instance with;
240
241 <pre>
242 var doc = new Document();
243 </pre>
244
a40523d @rnewson documentation of future features.
authored
245 Data may be added to this document with the add method which takes an optional second object argument that can override any of the above default values.
087dcec @rnewson update documentation.
authored
246
247 <pre>
a40523d @rnewson documentation of future features.
authored
248 // Add with all the defaults.
249 doc.add("value");
250
d2e1e9e @rnewson add notes about typing.
authored
251 // Add a numeric field.
38ef320 @rnewson fix documentation s/integer/int
authored
252 doc.add(35, {"type":"int"});
d2e1e9e @rnewson add notes about typing.
authored
253
4670582 @rnewson use NumericField. currently broken.
authored
254 // Add a date field.
b4e054e @karmi Added another example of valid format for date string in `date` field…
karmi authored
255 doc.add(new Date("1972/1/6 16:05:00"), {"type":"date"});
7f4e703 @rnewson correct new Date(str) example and add matching test.
authored
256 doc.add(new Date("January 6, 1972 16:05:00"), {"type":"date"});
4670582 @rnewson use NumericField. currently broken.
authored
257
d2e1e9e @rnewson add notes about typing.
authored
258 // Add a date field (object must be a Date object
259
a40523d @rnewson documentation of future features.
authored
260 // Add a subject field.
261 doc.add("this is the subject line.", {"field":"subject"});
9a71557 @rnewson formatting
authored
262
a40523d @rnewson documentation of future features.
authored
263 // Add but ensure it's stored.
264 doc.add("value", {"store":"yes"});
9a71557 @rnewson formatting
authored
265
a40523d @rnewson documentation of future features.
authored
266 // Add but don't analyze.
267 doc.add("don't analyze me", {"index":"not_analyzed"});
9a71557 @rnewson formatting
authored
268
b0663ec @rnewson correct misleading .attachment examples (but, hey, one of them was co…
authored
269 // Extract text from the named attachment and index it to a named field
270 doc.attachment("attachment field", "attachment name");
9f99f19 @rnewson add log.info example
authored
271
272 // log an event (trace, debug, info, warn and error are available)
273 if (doc.foo) {
274 log.info("doc has foo property!");
275 }
087dcec @rnewson update documentation.
authored
276 </pre>
277
2d96db8 @rnewson remove misleading 'transforms' term.
authored
278 <h3>Example Index Functions</h3>
ccb81a8 @rnewson add example transforms section.
authored
279
390858a @rnewson re-add Index Everything example.
authored
280 <h4>Index Everything</h4>
281
282 <pre>
283 function(doc) {
7bad7dc @rnewson correct syntax error in JS fun.
authored
284 var ret = new Document();
285
286 function idx(obj) {
287 for (var key in obj) {
288 switch (typeof obj[key]) {
289 case 'object':
290 idx(obj[key]);
291 break;
292 case 'function':
293 break;
294 default:
295 ret.add(obj[key]);
296 break;
297 }
298 }
299 };
300
301 idx(doc);
302
303 if (doc._attachments) {
304 for (var i in doc._attachments) {
470171d @rnewson change examples to index attachments into default field.
authored
305 ret.attachment("default", i);
7bad7dc @rnewson correct syntax error in JS fun.
authored
306 }
390858a @rnewson re-add Index Everything example.
authored
307 }
d2e1e9e @rnewson add notes about typing.
authored
308
7bad7dc @rnewson correct syntax error in JS fun.
authored
309 return ret;
390858a @rnewson re-add Index Everything example.
authored
310 }
311 </pre>
312
ccb81a8 @rnewson add example transforms section.
authored
313 <h4>Index Nothing</h4>
314
315 <pre>
316 function(doc) {
317 return null;
318 }
319 </pre>
320
c207a60 @rnewson update README
authored
321 <h4>Index Select Fields</h4>
ccb81a8 @rnewson add example transforms section.
authored
322
323 <pre>
324 function(doc) {
c207a60 @rnewson update README
authored
325 var result = new Document();
c6356fd @rnewson update README.md and TODO to reflect progress.
authored
326 result.add(doc.subject, {"field":"subject", "store":"yes"});
327 result.add(doc.content, {"field":"subject"});
5cfa20c @rnewson fix error in example
authored
328 result.add(new Date(), {"field":"indexed_at"});
c207a60 @rnewson update README
authored
329 return result;
ccb81a8 @rnewson add example transforms section.
authored
330 }
331 </pre>
332
c207a60 @rnewson update README
authored
333 <h4>Index Attachments</h4>
ccb81a8 @rnewson add example transforms section.
authored
334
335 <pre>
336 function(doc) {
c207a60 @rnewson update README
authored
337 var result = new Document();
338 for(var a in doc._attachments) {
470171d @rnewson change examples to index attachments into default field.
authored
339 result.attachment("default", a);
ccb81a8 @rnewson add example transforms section.
authored
340 }
c207a60 @rnewson update README
authored
341 return result;
342 }
343 </pre>
344
345 <h4>A More Complex Example</h4>
346
347 <pre>
348 function(doc) {
349 var mk = function(name, value, group) {
c6356fd @rnewson update README.md and TODO to reflect progress.
authored
350 var ret = new Document();
2946c9a @rnewson fix example.
authored
351 ret.add(value, {"field": group, "store":"yes"});
c6356fd @rnewson update README.md and TODO to reflect progress.
authored
352 ret.add(group, {"field":"group", "store":"yes"});
c207a60 @rnewson update README
authored
353 return ret;
354 };
355 if(doc.type != "reference") return null;
a313b24 @rnewson lots of stuff.
authored
356 var ret = new Array();
c207a60 @rnewson update README
authored
357 for(var g in doc.groups) {
a313b24 @rnewson lots of stuff.
authored
358 ret.push(mk("library", doc.groups[g].library, g));
359 ret.push(mk("method", doc.groups[g].method, g));
360 ret.push(mk("target", doc.groups[g].target, g));
c207a60 @rnewson update README
authored
361 }
362 return ret;
363 }
364 </pre>
b207965 @rnewson improve README readability.
authored
365
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
366 <h2>Attachment Indexing</h2>
367
8059ce0 @rnewson s/couchdb/couchdb-lucene
authored
368 Couchdb-lucene uses <a href="http://lucene.apache.org/tika/">Apache Tika</a> to index attachments of the following types, assuming the correct content_type is set in couchdb;
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
369
ec94e21 @rnewson updated README.md
authored
370 <h3>Supported Formats</h3>
371
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
372 <ul>
373 <li>Excel spreadsheets (application/vnd.ms-excel)
374 <li>HTML (text/html)
375 <li>Images (image/*)
376 <li>Java class files
377 <li>Java jar archives
378 <li>MP3 (audio/mp3)
379 <li>OpenDocument (application/vnd.oasis.opendocument.*)
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
380 <li>Outlook (application/vnd.ms-outlook)
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
381 <li>PDF (application/pdf)
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
382 <li>Plain text (text/plain)
383 <li>Powerpoint presentations (application/vnd.ms-powerpoint)
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
384 <li>RTF (application/rtf)
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
385 <li>Visio (application/vnd.visio)
386 <li>Word documents (application/msword)
387 <li>XML (application/xml)
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
388 </ul>
389
b207965 @rnewson improve README readability.
authored
390 <h1>Searching with couchdb-lucene</h1>
391
2b14b76 @rnewson describ syntax for all forms of numeric range query
authored
392 You can perform all types of queries using Lucene's default <a href="http://lucene.apache.org/java/2_4_0/queryparsersyntax.html">query syntax</a>.
393
394 <h2>Numeric range queries</h2>
395
e0c2000 @rnewson fix grammar, thanks tisba.
authored
396 In addition to normal text-based range searches (using the "field:[lower TO upper]" syntax), couchdb-lucene also supports numeric range searches for the following types: int, long, float, double and date. The type is specified after the field name, as follows;
2b14b76 @rnewson describ syntax for all forms of numeric range query
authored
397
398 <table>
bf989bc @rnewson make type of range query explicit.
authored
399 <tr><td>type</td><td>example</td></tr>
38ef320 @rnewson fix documentation s/integer/int
authored
400 <tr><td>int</td><td>field&lt;int>:[0 TO 100]</td></tr>
20c9be3 @rnewson update docs on numeric range queries
authored
401 <tr><td>long</td><td>field&lt;long>:[0 TO 100]</td></tr>
402 <tr><td>float</td><td>field&lt;float>:[0.0 TO 100.0]</td></tr>
403 <tr><td>double</td><td>field&lt;double>:[0.0 TO 100.0]</td></tr>
c79d771 @rnewson more README fixes.
authored
404 <tr><td>date</td><td>field&lt;date>:[from TO to] where from and to match any of these patterns: <code>"yyyy-MM-dd'T'HH:mm:ssZ"</code>, <code>"yyyy-MM-dd'T'HH:mm:ss"<code>, <code>"yyyy-MM-ddZ"M/code>, <code>"yyyy-MM-dd"</code>, <code>"yyyy-MM-dd'T'HH:mm:ss.SSSZ"</code>, <code>"yyyy-MM-dd'T'HH:mm:ss.SSS"</code>. So, in order to search for articles published in July, you would issue a following query: <code>published_at&lt;date&gt;:["2010-07-01T00:00:00"+TO+"2010-07-31T23:59:59"]</code></td></tr>
2b14b76 @rnewson describ syntax for all forms of numeric range query
authored
405 </table>
406
460f5c6 @rnewson add example spatial/geographical query
authored
407 An example numeric range query for spatial searching.
408
409 <pre>
1217f9f @rnewson update docs on numeric range queries
authored
410 ?q=pizza AND lat&lt;double>:[51.4707 TO 51.5224] AND long&lt;double>:[-0.6622 TO -0.5775]
460f5c6 @rnewson add example spatial/geographical query
authored
411 </pre>
412
fad4eb1 @rnewson document the syntax for numeric terms.
authored
413 <h2>Numeric term queries</h2>
414
15de3b7 @rnewson escape some <>'s
authored
415 Fields indexed with numeric types can still be queried as normal terms, couchdb-lucene just needs to know the type. For example, ?q=age&lt;long&gt;:12 will find all documents where the field called 'age' has a value of 12 (when the field was indexed as "type":"int".
fad4eb1 @rnewson document the syntax for numeric terms.
authored
416
417 <h2>Search parameters</h2>
418
2b14b76 @rnewson describ syntax for all forms of numeric range query
authored
419 The following parameters can be passed for more sophisticated searches;
b207965 @rnewson improve README readability.
authored
420
421 <dl>
70c8bf6 @rnewson document new analyzer query parameter
authored
422 <dt>analyzer</dt><dd>Override the default analyzer used to parse the q parameter</dd>
423 <dt>callback</dt><dd>Specify a JSONP callback wrapper. The full JSON result will be prepended with this parameter and also placed with parentheses."</dd>
71cbc0b @rnewson correct description of debug setting.
authored
424 <dt>debug</dt><dd>Setting this to true disables response caching (the query is executed every time) and indents the JSON response for readability.</dd>
bba040b @rnewson allow default_operator=and. Closes #95
authored
425 <dt>default_operator</dt><dd>Change the default operator for boolean queries. Defaults to "OR", other permitted value is "AND".</dd>
ea19e55 @rnewson add force_json=true to force 'application/json' response type.
authored
426 <dt>force_json<dt><dd>Usually couchdb-lucene determines the Content-Type of its response based on the presence of the Accept header. If Accept contains "application/json", you get "application/json" in the response, otherwise you get "text/plain;charset=utf8". Some tools, like JSONView for FireFox, do not send the Accept header but do render "application/json" responses if received. Setting force_json=true forces all response to "application/json" regardless of the Accept header.</dd>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
427 <dt>include_docs</dt><dd>whether to include the source docs</dd>
c9d4f76 @rnewson Add include_fields option to return a chosen subset of stored fields.…
authored
428 <dt>include_fields</dt><dd>By default, <i>all</i> stored fields are returned with results. Use a comma-separate list of field names with this parameter to refine the response</dd>
f9c61e3 @rnewson format README
authored
429 <dt>limit</dt><dd>the maximum number of results to return</dd>
caccea4 @rnewson avoid JSON object encoding problem by changing multiple query feature…
authored
430 <dt>q</dt><dd>the query to run (e.g, subject:hello). If not specified, the default field is searched. Multiple queries can be supplied, separated by commas; the resulting JSON will be an array of responses.</dd>
f9c61e3 @rnewson format README
authored
431 <dt>skip</dt><dd>the number of results to skip</dd>
81ea4c1 @rnewson more escapin'
authored
432 <dt>sort</dt><dd>the comma-separated fields to sort on. Prefix with / for ascending order and \ for descending order (ascending is the default if not specified). Type-specific sorting is also available by appending the type between angle brackets (e.g, sort=amount&lt;float&gt;). Supported types are 'float', 'double', 'int', 'long' and 'date'.</dd>
1ddad7b @tisba Update the README to reflect current behavior of
tisba authored
433 <dt>stale=ok</dt><dd>If you set the <i>stale</i> option to <i>ok</i>, couchdb-lucene will not block if the index is not up to date and it will immediately return results. Therefore searches may be faster as Lucene caches important data (especially for sorting). A query without stale=ok will block and use the latest data committed to the index. Unlike CouchDBs stale=ok option for views, couchdb-lucene will trigger an index update unless one is already running.</dd>
ad9096f @rnewson tweak README.md
authored
434 </dl>
b207965 @rnewson improve README readability.
authored
435
436 <i>All parameters except 'q' are optional.</i>
437
ec94e21 @rnewson updated README.md
authored
438 <h2>Special Fields</h2>
439
440 <dl>
087dcec @rnewson update documentation.
authored
441 <dt>_id</dt><dd>The _id of the document.</dd>
46a3a37 @rnewson include all DC attributes, if present.
authored
442 </dl>
443
444 <h2>Dublin Core</h2>
445
446 All Dublin Core attributes are indexed and stored if detected in the attachment. Descriptions of the fields come from the Tika javadocs.
447
448 <dl>
6e99faa @rnewson dc. to _dc.
authored
449 <dt>_dc.contributor</dt><dd> An entity responsible for making contributions to the content of the resource.</dd>
450 <dt>_dc.coverage</dt><dd>The extent or scope of the content of the resource.</dd>
451 <dt>_dc.creator</dt><dd>An entity primarily responsible for making the content of the resource.</dd>
452 <dt>_dc.date</dt><dd>A date associated with an event in the life cycle of the resource.</dd>
453 <dt>_dc.description</dt><dd>An account of the content of the resource.</dd>
454 <dt>_dc.format</dt><dd>Typically, Format may include the media-type or dimensions of the resource.</dd>
455 <dt>_dc.identifier</dt><dd>Recommended best practice is to identify the resource by means of a string or number conforming to a formal identification system.</dd>
456 <dt>_dc.language</dt><dd>A language of the intellectual content of the resource.</dd>
457 <dt>_dc.modified</dt><dd>Date on which the resource was changed.</dd>
458 <dt>_dc.publisher</dt><dd>An entity responsible for making the resource available.</dd>
459 <dt>_dc.relation</dt><dd>A reference to a related resource.</dd>
460 <dt>_dc.rights</dt><dd>Information about rights held in and over the resource.</dd>
461 <dt>_dc.source</dt><dd>A reference to a resource from which the present resource is derived.</dd>
462 <dt>_dc.subject</dt><dd>The topic of the content of the resource.</dd>
463 <dt>_dc.title</dt><dd>A name given to the resource.</dd>
464 <dt>_dc.type</dt><dd>The nature or genre of the content of the resource.</dd>
ec94e21 @rnewson updated README.md
authored
465 </dl>
466
b207965 @rnewson improve README readability.
authored
467 <h2>Examples</h2>
468
469 <pre>
6b73dc6 @rnewson commit to _design/foo in paths, use regexp to be sure.
authored
470 http://localhost:5984/dbname/_fti/_design/foo/view_name?q=field_name:value
471 http://localhost:5984/dbname/_fti/_design/foo/view_name?q=field_name:value&sort=other_field
4ae6c50 @stefankoegl <int> type specifier belongs to the field, not the "sort" parameter
stefankoegl authored
472 http://localhost:5984/dbname/_fti/_design/foo/view_name?debug=true&sort=billing_size&lt;long&gt;&q=body:document AND customer:[A TO C]
b207965 @rnewson improve README readability.
authored
473 </pre>
474
475 <h2>Search Results Format</h2>
476
0fcf578 @rnewson update docs.
authored
477 The search result contains a number of fields at the top level, in addition to your search results.
478
479 <dl>
480 <dt>etag</dt><dd>An opaque token that reflects the current version of the index. This value is also returned in an ETag header to facilitate HTTP caching.</dd>
481 <dt>fetch_duration</dt><dd>The number of milliseconds spent retrieving the documents.</dd>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
482 <dt>limit</dt><dd>The maximum number of results that can appear.</dd>
483 <dt>q</dt><dd>The query that was executed.</dd>
24591d9 @rnewson docs
authored
484 <dt>rows</dt><dd>The search results array, described below.</dd>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
485 <dt>search_duration</dt><dd>The number of milliseconds spent performing the search.</dd>
486 <dt>skip</dt><dd>The number of initial matches that was skipped.</dd>
487 <dt>total_rows</dt><dd>The total number of matches for this query.</dd>
0fcf578 @rnewson update docs.
authored
488 </dl>
489
24591d9 @rnewson docs
authored
490 <h2>The search results array</h2>
491
492 The search results arrays consists of zero, one or more objects with the following fields;
0fcf578 @rnewson update docs.
authored
493
494 <dl>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
495 <dt>doc</dt><dd>The original document from couch, if requested with include_docs=true</dd>
496 <dt>fields</dt><dd>All the fields that were stored with this match</dd>
0fcf578 @rnewson update docs.
authored
497 <dt>id</dt><dd>The unique identifier for this match.</dd>
498 <dt>score</dt><dd>The normalized score (0.0-1.0, inclusive) for this match</dd>
499 </dl>
500
fd16315 @rnewson update README.md
authored
501 Here's an example of a JSON response without sorting;
b207965 @rnewson improve README readability.
authored
502
118d28e @rnewson JSON example output.
authored
503 <pre>
504 {
c6356fd @rnewson update README.md and TODO to reflect progress.
authored
505 "q": "+content:enron",
fd16315 @rnewson update README.md
authored
506 "skip": 0,
507 "limit": 2,
508 "total_rows": 176852,
509 "search_duration": 518,
510 "fetch_duration": 4,
511 "rows": [
512 {
0fcf578 @rnewson update docs.
authored
513 "id": "hain-m-all_documents-257.",
fd16315 @rnewson update README.md
authored
514 "score": 1.601625680923462
515 },
516 {
0fcf578 @rnewson update docs.
authored
517 "id": "hain-m-notes_inbox-257.",
fd16315 @rnewson update README.md
authored
518 "score": 1.601625680923462
519 }
118d28e @rnewson JSON example output.
authored
520 ]
521 }
522 </pre>
523
fd16315 @rnewson update README.md
authored
524 And the same with sorting;
525
118d28e @rnewson JSON example output.
authored
526 <pre>
527 {
0fcf578 @rnewson update docs.
authored
528 "q": "+content:enron",
fd16315 @rnewson update README.md
authored
529 "skip": 0,
530 "limit": 3,
531 "total_rows": 176852,
532 "search_duration": 660,
533 "fetch_duration": 4,
534 "sort_order": [
535 {
536 "field": "source",
537 "reverse": false,
538 "type": "string"
539 },
540 {
541 "reverse": false,
542 "type": "doc"
543 }
118d28e @rnewson JSON example output.
authored
544 ],
fd16315 @rnewson update README.md
authored
545 "rows": [
546 {
0fcf578 @rnewson update docs.
authored
547 "id": "shankman-j-inbox-105.",
fd16315 @rnewson update README.md
authored
548 "score": 0.6131107211112976,
549 "sort_order": [
550 "enron",
551 6
552 ]
553 },
554 {
0fcf578 @rnewson update docs.
authored
555 "id": "shankman-j-inbox-8.",
fd16315 @rnewson update README.md
authored
556 "score": 0.7492915391921997,
557 "sort_order": [
558 "enron",
559 7
560 ]
561 },
562 {
0fcf578 @rnewson update docs.
authored
563 "id": "shankman-j-inbox-30.",
fd16315 @rnewson update README.md
authored
564 "score": 0.507369875907898,
565 "sort_order": [
566 "enron",
567 8
568 ]
569 }
118d28e @rnewson JSON example output.
authored
570 ]
571 }
572 </pre>
573
a4aa4e1 @rnewson document Content-Type response negotiation
authored
574 <h3>Content-Type of response</h3>
575
d2e1e9e @rnewson add notes about typing.
authored
576 The Content-Type of the response is negotiated via the Accept request header like CouchDB itself. If the Accept header includes "application/json" then that is also the Content-Type of the response. If not, "text/plain;charset=utf-8" is used.
a4aa4e1 @rnewson document Content-Type response negotiation
authored
577
139a78c @rnewson add info retrieval.
authored
578 <h1>Fetching information about the index</h1>
579
0cf4941 @rnewson correct info urls.
authored
580 Calling couchdb-lucene without arguments returns a JSON object with information about the index.
139a78c @rnewson add info retrieval.
authored
581
582 <pre>
6b73dc6 @rnewson commit to _design/foo in paths, use regexp to be sure.
authored
583 http://127.0.0.1:5984/&lt;db>/_fti/_design/foo/&lt;index
139a78c @rnewson add info retrieval.
authored
584 </pre>
585
586 returns;
587
588 <pre>
f6ac048 @rnewson line breaks
authored
589 {"current":true,"disk_size":110674,"doc_count":397,"doc_del_count":0,
590 "fields":["default","number"],"last_modified":"1263066382000",
591 "optimized":true,"ref_count":2}
139a78c @rnewson add info retrieval.
authored
592 </pre>
8203af6 @rnewson support _optimize and _expunge calls.
authored
593
594 <h1>Index Maintenance</h1>
595
596 For optimal query speed you can optimize your indexes. This causes the index to be rewritten into a single segment.
597
598 <pre>
6b73dc6 @rnewson commit to _design/foo in paths, use regexp to be sure.
authored
599 curl -X POST http://localhost:5984/&lt;db>/_fti/_design/foo/&lt;index>/_optimize
8203af6 @rnewson support _optimize and _expunge calls.
authored
600 </pre>
601
602 If you just want to expunge pending deletes, then call;
603
604 <pre>
6b73dc6 @rnewson commit to _design/foo in paths, use regexp to be sure.
authored
605 curl -X POST http://localhost:5984/&lt;db>/_fti/_design/foo/&lt;index>/_expunge
8203af6 @rnewson support _optimize and _expunge calls.
authored
606 </pre>
6e66766 @rnewson most of index cleanup work
authored
607
608 If you recreate databases or frequently change your fulltext functions, you will probably have old indexes lying around on disk. To remove all of them, call;
609
610 <pre>
611 curl -X POST http://localhost:5984/&lt;db>/_fti/_cleanup
612 </pre>
378e322 @rnewson add basic authentication for require_valid_user=true users.
authored
613
614 <h1>Authentication</h1>
615
616 By default couchdb-lucene does not attempt to authenticate to CouchDB. If you have set CouchDB's require_valid_user to true, you will need to modify couchdb-lucene.ini. Change the url setting to include a valid username and password. e.g, the default setting is;
617
618 <pre>
619 [local]
620 url=http://localhost:5984/
621 </pre>
622
623 Change it to;
624
625 <pre>
626 [local]
627 url=http://foo:bar@localhost:5984/
628 </pre>
629
630 and couchdb-lucene will authenticate to couchdb.
Something went wrong with that request. Please try again.