Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 585 lines (461 sloc) 19.573 kB
6d1c074 @rnewson add notes about version compatibility.
authored
1 <h1>Version Compatibility</h1>
2
3 <table>
4 <tr><th>CouchDB</th><th>couchdb-lucene</th></tr>
5 <tr><td>0.9.1, 0.10</td><td>0.4</td></tr>
5cd5577 @rnewson 0.11 is now released.
authored
6 <tr><td>0.11</td><td>0.4-maint (0.4 with patch for trunk compatibility)</td></tr>
cc88b5b @rnewson add note about compatibility with 1.0.x
authored
7 <tr><td>0.10.x, 0.11.x, 1.0.x</td><td>0.5.x</td></tr>
6d1c074 @rnewson add notes about version compatibility.
authored
8 </table>
9
9084540 @rnewson add breaking changes section.
authored
10 <h1>Breaking Changes</h1>
11
12 <ul>
79285e8 @rnewson remove warning about 0.5 now it's out.
authored
13 <li>couchdb-lucene 0.5.x runs as a standalone daemon (0.4 was run directly by couchdb).
9084540 @rnewson add breaking changes section.
authored
14 <li>URL's now require the full design document id (where you would say "foo", you must now say "_design/foo").
15 </ul>
16
c6356fd @rnewson update README.md and TODO to reflect progress.
authored
17 <h1>Issue Tracking</h1>
a785480 @rnewson lighthouse sucks at formatting anything, abandon ship.
authored
18
7a0d1d3 @rnewson lighthouse sucks at formatting anything, abandon ship.
authored
19 Issue tracking at <a href="http://github.com/rnewson/couchdb-lucene/issues">github</a>.
5d4e56a @rnewson update readme.
authored
20
7dc5bac @rnewson reorg begins.
authored
21 <h1>Minimum System Requirements</h1>
ef3f787 @rnewson add sysreq for Sun JDK.
authored
22
796a42b @rnewson jdk 5 compatibility for readme.
authored
23 Java 1.5 (or above) is required; the Sun version is recommended as it's regularly tested against.
ef3f787 @rnewson add sysreq for Sun JDK.
authored
24
5220b65 @rnewson tweak README.md
authored
25 <h1>Build couchdb-lucene</h1>
b207965 @rnewson improve README readability.
authored
26
27 <ol>
28 <li>Install Maven 2.
29 <li>checkout repository
30 <li>type 'mvn'
31 <li>configure couchdb (see below)
32 </ol>
33
2d6180b @rnewson include properties file in zip. still not available to run script, th…
authored
34 You will now have a zip file in the target/ directory. This contains all the couchdb-lucene code, dependencies, startup scripts and configuration files to run couchdb-lucene.
35
b207965 @rnewson improve README readability.
authored
36 <h1>Configure CouchDB</h1>
37
38 <pre>
0563120 @rnewson fixes.
authored
39 [couchdb]
40 os_process_timeout=60000 ; increase the timeout from 5 seconds.
41
b207965 @rnewson improve README readability.
authored
42 [external]
5675f7a @rnewson fix path
authored
43 fti=/path/to/python /path/to/couchdb-lucene/tools/couchdb-external-hook.py
b207965 @rnewson improve README readability.
authored
44
45 [httpd_db_handlers]
95b38b9 @rnewson more html escaping.
authored
46 _fti = {couch_httpd_external, handle_external_req, &lt;&lt;"fti"&gt;&gt;}
b207965 @rnewson improve README readability.
authored
47 </pre>
48
cbfe065 @rnewson remove registry and indexkey, change output paths
authored
49 <h2>Hook options</h2>
50
51 <table>
52 <tr><th>Option</th><th>Meaning</th><th>Default Value</th></tr>
53 <tr><td>--remote-host</td><td>The hostname of the couchdb-lucene server</td><td>localhost</td></tr>
54 <tr><td>--remote-port</td><td>The port of the couchdb-lucene server</td><td>5985</td></tr>
c47c494 @rnewson updated README to reflect latest Python scripts settings
authored
55 <tr><td>--local-key</td><td>The key for the local couchdb instance as known to the couchdb-lucene server</td><td>local</td></tr>
cbfe065 @rnewson remove registry and indexkey, change output paths
authored
56 </table>
b047e4a @rnewson update README.md to reflect (future) start/stop and config changes.
authored
57
cbfe065 @rnewson remove registry and indexkey, change output paths
authored
58 <h1>Configure couchdb-lucene</h1>
b047e4a @rnewson update README.md to reflect (future) start/stop and config changes.
authored
59
cbfe065 @rnewson remove registry and indexkey, change output paths
authored
60 couchdb-lucene runs in a single, standalone JVM. As such, you can choose to locate your couchdb-lucene server on a different machine to couchdb if you wish, or keep it on the same machine, it's your call.
b047e4a @rnewson update README.md to reflect (future) start/stop and config changes.
authored
61
62 <h1>Start couchdb-lucene</h1>
63
64 To start couchdb-lucene, run;
65 <pre>
2d6180b @rnewson include properties file in zip. still not available to run script, th…
authored
66 bin/run
b047e4a @rnewson update README.md to reflect (future) start/stop and config changes.
authored
67 </pre>
68
2d6180b @rnewson include properties file in zip. still not available to run script, th…
authored
69 To stop couchdb-lucene, simply kill the Java process.
b047e4a @rnewson update README.md to reflect (future) start/stop and config changes.
authored
70
b207965 @rnewson improve README readability.
authored
71 <h1>Indexing Strategy</h1>
72
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
73 <h2>Document Indexing</h2>
74
4c21806 @rnewson mention that you can return Document[] from an index function.
authored
75 You must supply a index function in order to enable couchdb-lucene as, by default, nothing will be indexed. To suppress a document from the index, return null. It's more typical to return a single Document object which contains everything you'd like to query and retrieve. You may also return an array of Document objects if you wish.
a2e9024 @rnewson wip
authored
76
437eae9 @rnewson s/view/fulltext in README.md
authored
77 You may add any number of index views in any number of design documents. All searches will be constrained to documents emitted by the index functions.
c207a60 @rnewson update README
authored
78
5077366 @rnewson clarify design document and add matching query URL's.
authored
79 Here's an complete example of a design document with couchdb-lucene features:
a2e9024 @rnewson wip
authored
80
697884b @rnewson documentation of future features.
authored
81 <pre>
82 {
3d2fb72 @rnewson fix example in README.
authored
83 "_id":"_design/foo",
5077366 @rnewson clarify design document and add matching query URL's.
authored
84 "fulltext": {
85 "by_subject": {
86 "index":"function(doc) { var ret=new Document(); ret.add(doc.subject); return ret }"
87 },
88 "by_content": {
89 "index":"function(doc) { var ret=new Document(); ret.add(doc.content); return ret }"
90 }
697884b @rnewson documentation of future features.
authored
91 }
92 }
93 </pre>
94
5077366 @rnewson clarify design document and add matching query URL's.
authored
95 Here are some example URL's for the given design document;
96
97 <pre>
6b73dc6 @rnewson commit to _design/foo in paths, use regexp to be sure.
authored
98 http://localhost:5984/database/_fti/_design/foo/by_subject?q=hello
99 http://localhost:5984/database/_fti/_design/foo/by_content?q=hello
5077366 @rnewson clarify design document and add matching query URL's.
authored
100 </pre>
101
697884b @rnewson documentation of future features.
authored
102 A fulltext object contains multiple index view declarations. An index view consists of;
103
104 <dl>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
105 <dt>analyzer</dt><dd>(optional) The analyzer to use</dd>
106 <dt>defaults</dt><dd>(optional) The default for numerous indexing options can be overridden here. A full list of options follows.</dd>
697884b @rnewson documentation of future features.
authored
107 <dt>index</dt><dd>The indexing function itself, documented below.</dd>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
108 </dl>
697884b @rnewson documentation of future features.
authored
109
110 <h3>The Defaults Object</h3>
111
112 The following indexing options can be defaulted;
113
114 <table>
115 <tr>
116 <th>name</th>
117 <th>description</th>
118 <th>available options</th>
119 <th>default</th>
120 </tr>
121 <tr>
a40523d @rnewson documentation of future features.
authored
122 <th>field</th>
123 <td>the field name to index under</td>
124 <td>user-defined</td>
125 <td>default</td>
d2e1e9e @rnewson add notes about typing.
authored
126 </tr>
127 <tr>
128 <th>type</th>
129 <td>the type of the field</td>
38ef320 @rnewson fix documentation s/integer/int
authored
130 <td>date, double, float, int, long, string</td>
d2e1e9e @rnewson add notes about typing.
authored
131 <td>string</td>
132 </tr>
a40523d @rnewson documentation of future features.
authored
133 <tr>
697884b @rnewson documentation of future features.
authored
134 <th>store</th>
f16fc9c @rnewson docs
authored
135 <td>whether the data is stored. The value will be returned in the search result.</td>
697884b @rnewson documentation of future features.
authored
136 <td>yes, no</td>
137 <td>no</td>
d2e1e9e @rnewson add notes about typing.
authored
138 </tr>
697884b @rnewson documentation of future features.
authored
139 <tr>
140 <th>index</th>
141 <td>whether (and how) the data is indexed</td>
8328332 @rnewson typo
authored
142 <td>analyzed, analyzed_no_norms, no, not_analyzed, not_analyzed_no_norms</td>
697884b @rnewson documentation of future features.
authored
143 <td>analyzed</td>
d2e1e9e @rnewson add notes about typing.
authored
144 </tr>
f6bfce8 @rnewson add EXPERIMENTAL index-time field boosting feature.
authored
145 <tr>
b00fb7f add termvector to the possible options for Field.
Santiago M. Mola authored
146 <th>termvector</th>
147 <td>whether and how a field should have term vectors</td>
148 <td>no, with_offsets, with_positions, with_positions_offsets, yes</td>
149 <td>no</td>
150 </tr>
151 <tr>
f6bfce8 @rnewson add EXPERIMENTAL index-time field boosting feature.
authored
152 <th>boost</th>
153 <td>Sets the boost factor hits on this field. This value will be multiplied into the score of all hits on this this field of this document.</td>
154 <td>floating-point value</td>
155 <td>1.0</td>
156 </tr>
697884b @rnewson documentation of future features.
authored
157 </table>
087dcec @rnewson update documentation.
authored
158
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
159 <h3>The Analyzer Option</h3>
160
161 Lucene has numerous ways of converting free-form text into tokens, these classes are called Analyzer's. By default, the StandardAnalyzer is used which lower-cases all text, drops common English words ("the", "and", and so on), among other things. This processing might not always suit you, so you can choose from several others by setting the "analyzer" field to one of the following values;
162
163 <ul>
164 <li>brazilian</li>
165 <li>chinese</li>
166 <li>cjk</li>
167 <li>czech</li>
168 <li>dutch</li>
169 <li>english</li>
170 <li>french</li>
171 <li>german</li>
172 <li>keyword</li>
76d290d @rnewson add perfield analyzer support
authored
173 <li>perfield</li>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
174 <li>porter</li>
175 <li>russian</li>
176 <li>simple</li>
177 <li>standard</li>
178 <li>thai</li>
fe43cf0 @rnewson add whitespace analyzer.
authored
179 <li>whitespace</li>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
180 </ul>
181
76d290d @rnewson add perfield analyzer support
authored
182 The "perfield" option lets you use a different analyzer for different fields and is configured as follows;
183
184 <pre>
dda9276 @rnewson more README fixes.
authored
185 "analyzer":"perfield:{field_name:\"analyzer_name\"}"
76d290d @rnewson add perfield analyzer support
authored
186 </pre>
187
86569b1 @rnewson use 'default' consistently.
authored
188 Unless overridden, any field name not specified will be handled by the standard analyzer. To change the default, use the special default field name;
76d290d @rnewson add perfield analyzer support
authored
189
190 <pre>
dda9276 @rnewson more README fixes.
authored
191 "analyzer":"perfield:{default:\"keyword\"}"
76d290d @rnewson add perfield analyzer support
authored
192 </pre>
193
087dcec @rnewson update documentation.
authored
194 <h3>The Document class</h3>
195
196 You may construct a new Document instance with;
197
198 <pre>
199 var doc = new Document();
200 </pre>
201
a40523d @rnewson documentation of future features.
authored
202 Data may be added to this document with the add method which takes an optional second object argument that can override any of the above default values.
087dcec @rnewson update documentation.
authored
203
204 <pre>
a40523d @rnewson documentation of future features.
authored
205 // Add with all the defaults.
206 doc.add("value");
207
d2e1e9e @rnewson add notes about typing.
authored
208 // Add a numeric field.
38ef320 @rnewson fix documentation s/integer/int
authored
209 doc.add(35, {"type":"int"});
d2e1e9e @rnewson add notes about typing.
authored
210
4670582 @rnewson use NumericField. currently broken.
authored
211 // Add a date field.
b4e054e @karmi Added another example of valid format for date string in `date` field…
karmi authored
212 doc.add(new Date("1972/1/6 16:05:00"), {"type":"date"});
7f4e703 @rnewson correct new Date(str) example and add matching test.
authored
213 doc.add(new Date("January 6, 1972 16:05:00"), {"type":"date"});
4670582 @rnewson use NumericField. currently broken.
authored
214
d2e1e9e @rnewson add notes about typing.
authored
215 // Add a date field (object must be a Date object
216
a40523d @rnewson documentation of future features.
authored
217 // Add a subject field.
218 doc.add("this is the subject line.", {"field":"subject"});
9a71557 @rnewson formatting
authored
219
a40523d @rnewson documentation of future features.
authored
220 // Add but ensure it's stored.
221 doc.add("value", {"store":"yes"});
9a71557 @rnewson formatting
authored
222
a40523d @rnewson documentation of future features.
authored
223 // Add but don't analyze.
224 doc.add("don't analyze me", {"index":"not_analyzed"});
9a71557 @rnewson formatting
authored
225
b0663ec @rnewson correct misleading .attachment examples (but, hey, one of them was co…
authored
226 // Extract text from the named attachment and index it to a named field
227 doc.attachment("attachment field", "attachment name");
9f99f19 @rnewson add log.info example
authored
228
229 // log an event (trace, debug, info, warn and error are available)
230 if (doc.foo) {
231 log.info("doc has foo property!");
232 }
087dcec @rnewson update documentation.
authored
233 </pre>
234
ccb81a8 @rnewson add example transforms section.
authored
235 <h3>Example Transforms</h3>
236
390858a @rnewson re-add Index Everything example.
authored
237 <h4>Index Everything</h4>
238
239 <pre>
240 function(doc) {
7bad7dc @rnewson correct syntax error in JS fun.
authored
241 var ret = new Document();
242
243 function idx(obj) {
244 for (var key in obj) {
245 switch (typeof obj[key]) {
246 case 'object':
247 idx(obj[key]);
248 break;
249 case 'function':
250 break;
251 default:
252 ret.add(obj[key]);
253 break;
254 }
255 }
256 };
257
258 idx(doc);
259
260 if (doc._attachments) {
261 for (var i in doc._attachments) {
470171d @rnewson change examples to index attachments into default field.
authored
262 ret.attachment("default", i);
7bad7dc @rnewson correct syntax error in JS fun.
authored
263 }
390858a @rnewson re-add Index Everything example.
authored
264 }
d2e1e9e @rnewson add notes about typing.
authored
265
7bad7dc @rnewson correct syntax error in JS fun.
authored
266 return ret;
390858a @rnewson re-add Index Everything example.
authored
267 }
268 </pre>
269
ccb81a8 @rnewson add example transforms section.
authored
270 <h4>Index Nothing</h4>
271
272 <pre>
273 function(doc) {
274 return null;
275 }
276 </pre>
277
c207a60 @rnewson update README
authored
278 <h4>Index Select Fields</h4>
ccb81a8 @rnewson add example transforms section.
authored
279
280 <pre>
281 function(doc) {
c207a60 @rnewson update README
authored
282 var result = new Document();
c6356fd @rnewson update README.md and TODO to reflect progress.
authored
283 result.add(doc.subject, {"field":"subject", "store":"yes"});
284 result.add(doc.content, {"field":"subject"});
5cfa20c @rnewson fix error in example
authored
285 result.add(new Date(), {"field":"indexed_at"});
c207a60 @rnewson update README
authored
286 return result;
ccb81a8 @rnewson add example transforms section.
authored
287 }
288 </pre>
289
c207a60 @rnewson update README
authored
290 <h4>Index Attachments</h4>
ccb81a8 @rnewson add example transforms section.
authored
291
292 <pre>
293 function(doc) {
c207a60 @rnewson update README
authored
294 var result = new Document();
295 for(var a in doc._attachments) {
470171d @rnewson change examples to index attachments into default field.
authored
296 result.attachment("default", a);
ccb81a8 @rnewson add example transforms section.
authored
297 }
c207a60 @rnewson update README
authored
298 return result;
299 }
300 </pre>
301
302 <h4>A More Complex Example</h4>
303
304 <pre>
305 function(doc) {
306 var mk = function(name, value, group) {
c6356fd @rnewson update README.md and TODO to reflect progress.
authored
307 var ret = new Document();
2946c9a @rnewson fix example.
authored
308 ret.add(value, {"field": group, "store":"yes"});
c6356fd @rnewson update README.md and TODO to reflect progress.
authored
309 ret.add(group, {"field":"group", "store":"yes"});
c207a60 @rnewson update README
authored
310 return ret;
311 };
312 if(doc.type != "reference") return null;
a313b24 @rnewson lots of stuff.
authored
313 var ret = new Array();
c207a60 @rnewson update README
authored
314 for(var g in doc.groups) {
a313b24 @rnewson lots of stuff.
authored
315 ret.push(mk("library", doc.groups[g].library, g));
316 ret.push(mk("method", doc.groups[g].method, g));
317 ret.push(mk("target", doc.groups[g].target, g));
c207a60 @rnewson update README
authored
318 }
319 return ret;
320 }
321 </pre>
b207965 @rnewson improve README readability.
authored
322
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
323 <h2>Attachment Indexing</h2>
324
8059ce0 @rnewson s/couchdb/couchdb-lucene
authored
325 Couchdb-lucene uses <a href="http://lucene.apache.org/tika/">Apache Tika</a> to index attachments of the following types, assuming the correct content_type is set in couchdb;
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
326
ec94e21 @rnewson updated README.md
authored
327 <h3>Supported Formats</h3>
328
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
329 <ul>
330 <li>Excel spreadsheets (application/vnd.ms-excel)
331 <li>HTML (text/html)
332 <li>Images (image/*)
333 <li>Java class files
334 <li>Java jar archives
335 <li>MP3 (audio/mp3)
336 <li>OpenDocument (application/vnd.oasis.opendocument.*)
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
337 <li>Outlook (application/vnd.ms-outlook)
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
338 <li>PDF (application/pdf)
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
339 <li>Plain text (text/plain)
340 <li>Powerpoint presentations (application/vnd.ms-powerpoint)
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
341 <li>RTF (application/rtf)
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
342 <li>Visio (application/vnd.visio)
343 <li>Word documents (application/msword)
344 <li>XML (application/xml)
4a60080 @rnewson use couchdb's content_type rather than auto-detect.
authored
345 </ul>
346
b207965 @rnewson improve README readability.
authored
347 <h1>Searching with couchdb-lucene</h1>
348
2b14b76 @rnewson describ syntax for all forms of numeric range query
authored
349 You can perform all types of queries using Lucene's default <a href="http://lucene.apache.org/java/2_4_0/queryparsersyntax.html">query syntax</a>.
350
351 <h2>Numeric range queries</h2>
352
e0c2000 @rnewson fix grammar, thanks tisba.
authored
353 In addition to normal text-based range searches (using the "field:[lower TO upper]" syntax), couchdb-lucene also supports numeric range searches for the following types: int, long, float, double and date. The type is specified after the field name, as follows;
2b14b76 @rnewson describ syntax for all forms of numeric range query
authored
354
355 <table>
bf989bc @rnewson make type of range query explicit.
authored
356 <tr><td>type</td><td>example</td></tr>
38ef320 @rnewson fix documentation s/integer/int
authored
357 <tr><td>int</td><td>field&lt;int>:[0 TO 100]</td></tr>
20c9be3 @rnewson update docs on numeric range queries
authored
358 <tr><td>long</td><td>field&lt;long>:[0 TO 100]</td></tr>
359 <tr><td>float</td><td>field&lt;float>:[0.0 TO 100.0]</td></tr>
360 <tr><td>double</td><td>field&lt;double>:[0.0 TO 100.0]</td></tr>
c79d771 @rnewson more README fixes.
authored
361 <tr><td>date</td><td>field&lt;date>:[from TO to] where from and to match any of these patterns: <code>"yyyy-MM-dd'T'HH:mm:ssZ"</code>, <code>"yyyy-MM-dd'T'HH:mm:ss"<code>, <code>"yyyy-MM-ddZ"M/code>, <code>"yyyy-MM-dd"</code>, <code>"yyyy-MM-dd'T'HH:mm:ss.SSSZ"</code>, <code>"yyyy-MM-dd'T'HH:mm:ss.SSS"</code>. So, in order to search for articles published in July, you would issue a following query: <code>published_at&lt;date&gt;:["2010-07-01T00:00:00"+TO+"2010-07-31T23:59:59"]</code></td></tr>
2b14b76 @rnewson describ syntax for all forms of numeric range query
authored
362 </table>
363
460f5c6 @rnewson add example spatial/geographical query
authored
364 An example numeric range query for spatial searching.
365
366 <pre>
1217f9f @rnewson update docs on numeric range queries
authored
367 ?q=pizza AND lat&lt;double>:[51.4707 TO 51.5224] AND long&lt;double>:[-0.6622 TO -0.5775]
460f5c6 @rnewson add example spatial/geographical query
authored
368 </pre>
369
fad4eb1 @rnewson document the syntax for numeric terms.
authored
370 <h2>Numeric term queries</h2>
371
15de3b7 @rnewson escape some <>'s
authored
372 Fields indexed with numeric types can still be queried as normal terms, couchdb-lucene just needs to know the type. For example, ?q=age&lt;long&gt;:12 will find all documents where the field called 'age' has a value of 12 (when the field was indexed as "type":"int".
fad4eb1 @rnewson document the syntax for numeric terms.
authored
373
374 <h2>Search parameters</h2>
375
2b14b76 @rnewson describ syntax for all forms of numeric range query
authored
376 The following parameters can be passed for more sophisticated searches;
b207965 @rnewson improve README readability.
authored
377
378 <dl>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
379 <dt>callback</dt><dd>Specify a JSONP callback wrapper. The full JSON result will be prepended with this parameter and also placed with parentheses."
71cbc0b @rnewson correct description of debug setting.
authored
380 <dt>debug</dt><dd>Setting this to true disables response caching (the query is executed every time) and indents the JSON response for readability.</dd>
ea19e55 @rnewson add force_json=true to force 'application/json' response type.
authored
381 <dt>force_json<dt><dd>Usually couchdb-lucene determines the Content-Type of its response based on the presence of the Accept header. If Accept contains "application/json", you get "application/json" in the response, otherwise you get "text/plain;charset=utf8". Some tools, like JSONView for FireFox, do not send the Accept header but do render "application/json" responses if received. Setting force_json=true forces all response to "application/json" regardless of the Accept header.</dd>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
382 <dt>include_docs</dt><dd>whether to include the source docs</dd>
f9c61e3 @rnewson format README
authored
383 <dt>limit</dt><dd>the maximum number of results to return</dd>
caccea4 @rnewson avoid JSON object encoding problem by changing multiple query feature…
authored
384 <dt>q</dt><dd>the query to run (e.g, subject:hello). If not specified, the default field is searched. Multiple queries can be supplied, separated by commas; the resulting JSON will be an array of responses.</dd>
f9c61e3 @rnewson format README
authored
385 <dt>skip</dt><dd>the number of results to skip</dd>
81ea4c1 @rnewson more escapin'
authored
386 <dt>sort</dt><dd>the comma-separated fields to sort on. Prefix with / for ascending order and \ for descending order (ascending is the default if not specified). Type-specific sorting is also available by appending the type between angle brackets (e.g, sort=amount&lt;float&gt;). Supported types are 'float', 'double', 'int', 'long' and 'date'.</dd>
1ddad7b @tisba Update the README to reflect current behavior of
tisba authored
387 <dt>stale=ok</dt><dd>If you set the <i>stale</i> option to <i>ok</i>, couchdb-lucene will not block if the index is not up to date and it will immediately return results. Therefore searches may be faster as Lucene caches important data (especially for sorting). A query without stale=ok will block and use the latest data committed to the index. Unlike CouchDBs stale=ok option for views, couchdb-lucene will trigger an index update unless one is already running.</dd>
ad9096f @rnewson tweak README.md
authored
388 </dl>
b207965 @rnewson improve README readability.
authored
389
390 <i>All parameters except 'q' are optional.</i>
391
ec94e21 @rnewson updated README.md
authored
392 <h2>Special Fields</h2>
393
394 <dl>
087dcec @rnewson update documentation.
authored
395 <dt>_id</dt><dd>The _id of the document.</dd>
46a3a37 @rnewson include all DC attributes, if present.
authored
396 </dl>
397
398 <h2>Dublin Core</h2>
399
400 All Dublin Core attributes are indexed and stored if detected in the attachment. Descriptions of the fields come from the Tika javadocs.
401
402 <dl>
6e99faa @rnewson dc. to _dc.
authored
403 <dt>_dc.contributor</dt><dd> An entity responsible for making contributions to the content of the resource.</dd>
404 <dt>_dc.coverage</dt><dd>The extent or scope of the content of the resource.</dd>
405 <dt>_dc.creator</dt><dd>An entity primarily responsible for making the content of the resource.</dd>
406 <dt>_dc.date</dt><dd>A date associated with an event in the life cycle of the resource.</dd>
407 <dt>_dc.description</dt><dd>An account of the content of the resource.</dd>
408 <dt>_dc.format</dt><dd>Typically, Format may include the media-type or dimensions of the resource.</dd>
409 <dt>_dc.identifier</dt><dd>Recommended best practice is to identify the resource by means of a string or number conforming to a formal identification system.</dd>
410 <dt>_dc.language</dt><dd>A language of the intellectual content of the resource.</dd>
411 <dt>_dc.modified</dt><dd>Date on which the resource was changed.</dd>
412 <dt>_dc.publisher</dt><dd>An entity responsible for making the resource available.</dd>
413 <dt>_dc.relation</dt><dd>A reference to a related resource.</dd>
414 <dt>_dc.rights</dt><dd>Information about rights held in and over the resource.</dd>
415 <dt>_dc.source</dt><dd>A reference to a resource from which the present resource is derived.</dd>
416 <dt>_dc.subject</dt><dd>The topic of the content of the resource.</dd>
417 <dt>_dc.title</dt><dd>A name given to the resource.</dd>
418 <dt>_dc.type</dt><dd>The nature or genre of the content of the resource.</dd>
ec94e21 @rnewson updated README.md
authored
419 </dl>
420
b207965 @rnewson improve README readability.
authored
421 <h2>Examples</h2>
422
423 <pre>
6b73dc6 @rnewson commit to _design/foo in paths, use regexp to be sure.
authored
424 http://localhost:5984/dbname/_fti/_design/foo/view_name?q=field_name:value
425 http://localhost:5984/dbname/_fti/_design/foo/view_name?q=field_name:value&sort=other_field
15de3b7 @rnewson escape some <>'s
authored
426 http://localhost:5984/dbname/_fti/_design/foo/view_name?debug=true&sort&lt;long&gt;=billing_size&q=body:document AND customer:[A TO C]
b207965 @rnewson improve README readability.
authored
427 </pre>
428
429 <h2>Search Results Format</h2>
430
0fcf578 @rnewson update docs.
authored
431 The search result contains a number of fields at the top level, in addition to your search results.
432
433 <dl>
434 <dt>etag</dt><dd>An opaque token that reflects the current version of the index. This value is also returned in an ETag header to facilitate HTTP caching.</dd>
435 <dt>fetch_duration</dt><dd>The number of milliseconds spent retrieving the documents.</dd>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
436 <dt>limit</dt><dd>The maximum number of results that can appear.</dd>
437 <dt>q</dt><dd>The query that was executed.</dd>
24591d9 @rnewson docs
authored
438 <dt>rows</dt><dd>The search results array, described below.</dd>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
439 <dt>search_duration</dt><dd>The number of milliseconds spent performing the search.</dd>
440 <dt>skip</dt><dd>The number of initial matches that was skipped.</dd>
441 <dt>total_rows</dt><dd>The total number of matches for this query.</dd>
0fcf578 @rnewson update docs.
authored
442 </dl>
443
24591d9 @rnewson docs
authored
444 <h2>The search results array</h2>
445
446 The search results arrays consists of zero, one or more objects with the following fields;
0fcf578 @rnewson update docs.
authored
447
448 <dl>
f420bf7 @rnewson support different Analyzer's at index and query time.
authored
449 <dt>doc</dt><dd>The original document from couch, if requested with include_docs=true</dd>
450 <dt>fields</dt><dd>All the fields that were stored with this match</dd>
0fcf578 @rnewson update docs.
authored
451 <dt>id</dt><dd>The unique identifier for this match.</dd>
452 <dt>score</dt><dd>The normalized score (0.0-1.0, inclusive) for this match</dd>
453 </dl>
454
fd16315 @rnewson update README.md
authored
455 Here's an example of a JSON response without sorting;
b207965 @rnewson improve README readability.
authored
456
118d28e @rnewson JSON example output.
authored
457 <pre>
458 {
c6356fd @rnewson update README.md and TODO to reflect progress.
authored
459 "q": "+content:enron",
fd16315 @rnewson update README.md
authored
460 "skip": 0,
461 "limit": 2,
462 "total_rows": 176852,
463 "search_duration": 518,
464 "fetch_duration": 4,
465 "rows": [
466 {
0fcf578 @rnewson update docs.
authored
467 "id": "hain-m-all_documents-257.",
fd16315 @rnewson update README.md
authored
468 "score": 1.601625680923462
469 },
470 {
0fcf578 @rnewson update docs.
authored
471 "id": "hain-m-notes_inbox-257.",
fd16315 @rnewson update README.md
authored
472 "score": 1.601625680923462
473 }
118d28e @rnewson JSON example output.
authored
474 ]
475 }
476 </pre>
477
fd16315 @rnewson update README.md
authored
478 And the same with sorting;
479
118d28e @rnewson JSON example output.
authored
480 <pre>
481 {
0fcf578 @rnewson update docs.
authored
482 "q": "+content:enron",
fd16315 @rnewson update README.md
authored
483 "skip": 0,
484 "limit": 3,
485 "total_rows": 176852,
486 "search_duration": 660,
487 "fetch_duration": 4,
488 "sort_order": [
489 {
490 "field": "source",
491 "reverse": false,
492 "type": "string"
493 },
494 {
495 "reverse": false,
496 "type": "doc"
497 }
118d28e @rnewson JSON example output.
authored
498 ],
fd16315 @rnewson update README.md
authored
499 "rows": [
500 {
0fcf578 @rnewson update docs.
authored
501 "id": "shankman-j-inbox-105.",
fd16315 @rnewson update README.md
authored
502 "score": 0.6131107211112976,
503 "sort_order": [
504 "enron",
505 6
506 ]
507 },
508 {
0fcf578 @rnewson update docs.
authored
509 "id": "shankman-j-inbox-8.",
fd16315 @rnewson update README.md
authored
510 "score": 0.7492915391921997,
511 "sort_order": [
512 "enron",
513 7
514 ]
515 },
516 {
0fcf578 @rnewson update docs.
authored
517 "id": "shankman-j-inbox-30.",
fd16315 @rnewson update README.md
authored
518 "score": 0.507369875907898,
519 "sort_order": [
520 "enron",
521 8
522 ]
523 }
118d28e @rnewson JSON example output.
authored
524 ]
525 }
526 </pre>
527
a4aa4e1 @rnewson document Content-Type response negotiation
authored
528 <h3>Content-Type of response</h3>
529
d2e1e9e @rnewson add notes about typing.
authored
530 The Content-Type of the response is negotiated via the Accept request header like CouchDB itself. If the Accept header includes "application/json" then that is also the Content-Type of the response. If not, "text/plain;charset=utf-8" is used.
a4aa4e1 @rnewson document Content-Type response negotiation
authored
531
139a78c @rnewson add info retrieval.
authored
532 <h1>Fetching information about the index</h1>
533
0cf4941 @rnewson correct info urls.
authored
534 Calling couchdb-lucene without arguments returns a JSON object with information about the index.
139a78c @rnewson add info retrieval.
authored
535
536 <pre>
6b73dc6 @rnewson commit to _design/foo in paths, use regexp to be sure.
authored
537 http://127.0.0.1:5984/&lt;db>/_fti/_design/foo/&lt;index
139a78c @rnewson add info retrieval.
authored
538 </pre>
539
540 returns;
541
542 <pre>
f6ac048 @rnewson line breaks
authored
543 {"current":true,"disk_size":110674,"doc_count":397,"doc_del_count":0,
544 "fields":["default","number"],"last_modified":"1263066382000",
545 "optimized":true,"ref_count":2}
139a78c @rnewson add info retrieval.
authored
546 </pre>
8203af6 @rnewson support _optimize and _expunge calls.
authored
547
548 <h1>Index Maintenance</h1>
549
550 For optimal query speed you can optimize your indexes. This causes the index to be rewritten into a single segment.
551
552 <pre>
6b73dc6 @rnewson commit to _design/foo in paths, use regexp to be sure.
authored
553 curl -X POST http://localhost:5984/&lt;db>/_fti/_design/foo/&lt;index>/_optimize
8203af6 @rnewson support _optimize and _expunge calls.
authored
554 </pre>
555
556 If you just want to expunge pending deletes, then call;
557
558 <pre>
6b73dc6 @rnewson commit to _design/foo in paths, use regexp to be sure.
authored
559 curl -X POST http://localhost:5984/&lt;db>/_fti/_design/foo/&lt;index>/_expunge
8203af6 @rnewson support _optimize and _expunge calls.
authored
560 </pre>
6e66766 @rnewson most of index cleanup work
authored
561
562 If you recreate databases or frequently change your fulltext functions, you will probably have old indexes lying around on disk. To remove all of them, call;
563
564 <pre>
565 curl -X POST http://localhost:5984/&lt;db>/_fti/_cleanup
566 </pre>
378e322 @rnewson add basic authentication for require_valid_user=true users.
authored
567
568 <h1>Authentication</h1>
569
570 By default couchdb-lucene does not attempt to authenticate to CouchDB. If you have set CouchDB's require_valid_user to true, you will need to modify couchdb-lucene.ini. Change the url setting to include a valid username and password. e.g, the default setting is;
571
572 <pre>
573 [local]
574 url=http://localhost:5984/
575 </pre>
576
577 Change it to;
578
579 <pre>
580 [local]
581 url=http://foo:bar@localhost:5984/
582 </pre>
583
584 and couchdb-lucene will authenticate to couchdb.
Something went wrong with that request. Please try again.