Item9808: DBIStoreContrib now works with MySQL, Postgresql and SQLite…

…. Still more to do, to improve the query mapping, so this is a checkpoint only.

git-svn-id: http://svn.foswiki.org/trunk/DBIStoreContrib@17192 0b4bb1d4-4e5a-0410-9cc4-b2b747904278

data/System/DBIStoreContrib.txt

@@ -6,30 +6,44 @@
 
 %TOC%
 
-This extension (currently) implements search operations (query and text search) using the Perl DBI interface to SQL databases. It has been tested with [[http://www.sqlite.org/][SQLite]], the popular lightweight implementation of an SQL relational database, and to a lesser extent with [[http://www.mysql.com/][MySQL]].
+This extension (currently) implements search operations (query and text search) using the Perl DBI interface to SQL databases. It has been tested with [[http://www.sqlite.org/][SQLite]], the popular lightweight implementation of an SQL relational database, [[http://www.mysql.com/][MySQL]], and [[http://www.postgresql.org/][Postgresql]].
 
 We get over the fact that SQL does not implement all the query search features of Foswiki by _hoisting_ SQL expressions out of the Foswiki search statements, leaving behind only those parts of the expression that SQL can't handle.
 
-The extension is currently classed as experimental because the SQL schema (and the query generator) are (probably) sub-optimal, and require extensive tuning.
+The extension is currently classed as experimental because the SQL schema (and the query generator) are (probably) sub-optimal, and require tuning.
 
 The longer term goal is to implement a full back-end store using an SQL RDBMS, rather than just a simple cache as at present.
 
-Mapping to another DBMS *should* be as simple as setting up a different
-DSN (but life is rarely that simple. Not all DBs implement the REGEXP
-operator, for example). Please go ahead and try, though.
+Note that regular expression searches are mapped to whatever regular expression
+support exists in the database. Regular expressions that use more than the standard POSIX syntax are unlikely to work.
 
 ---++ Installation Instructions
 
 %$INSTALL_INSTRUCTIONS%
    * Go to =configure= and:
-      1 Set a DSN for the contrib in the 'Extensions' section (the default is for sqlite3),
+      1 Set a DSN etc. for the contrib in the 'Extensions' section (the default is for sqlite3),
       1 Select =Foswiki::Store::QueryAlgorithms::DBIStoreContrib= for the ={Store}{QueryAlgorithm}= *EXPERT* setting
       1 *For Foswiki > 1.1* only
          * add =Foswiki::Contrib::DBIStoreContrib::DBIStore= to the ={Store}{ImplementationClasses}= *EXPERT* setting in the 'Store' section
          * select =Foswiki::Store::SearchAlgorithms::DBIStoreContrib= for the ={Store}{SearchAlgorithm}= setting
       1 *For Foswiki 1.1 only*
+         * Select Foswiki::Store::QueryAlgorithms::DBIStoreContrib for the ={Store}{QueryAlgorithm}= configuration setting
          * enable the integrated =DBIStorePlugin=
-         * search -> query mapping is *not* supported
+
+---+++ !MySQL Notes
+The !MySQL database user needs at least the following privileges:
+SELECT, INSERT, CREATE, and DROP.
+
+---+++ Postgresql Notes
+
+---+++ Microsoft SQL Server Notes
+If you are using Windows authentication for users on SQL Server, then the simplest
+thing to do is to use the ODBC driver with DBIStoreContrib and create a data
+source for SQL Server in the ODBC Administrator which uses Windows authentication.
+Then set an empty username and password for !DBIStoreContrib.
+
+---+++ SQLite Notes
+SQLite requires the =pcre= module to be installed to support regular expression searches. The path to this module is set up in =configure=.
 
 ---++ How it works
 ---+++ Searches and Queries

data/System/DBIStoreContribRegexTests.txt

@@ -0,0 +1,32 @@
+%META:TOPICINFO{author="SimianApe" comment="reprev" date="1387037563" format="1.1" reprev="20" version="20"}%
+<!--
+   * Set PS = type="query" topic="%TOPIC%" nonoise="on" format="$percentGREEN%ok$percentENDCOLOR%" separator="" zeroresults="failed"
+-->
+This table summarises the features required by Foswiki for regular expression
+searches, and shows whether they work with the current store. 
+| Simple text search | %RED% %SEARCH{"SimpleText =~ 'TestMe'" %PS% }% |
+| Backslash \ escapes one metacharacter | %RED% %SEARCH{"Backslash =~ 'x\\\\y'" %PS% }% |
+| \x00 through \xFF (ASCII character) | %RED% %SEARCH{"SimpleText =~ '\x41\x53\x43\x49\x49'" %PS% }% |
+| \n (LF), \r (CR) and \t (tab) | %RED% %SEARCH{"Whitespace =~ ' \t\r\n'" %PS%}% |
+| [abc] character class, [a-z] character class range | %RED% %SEARCH{"CharClass =~ '[A-Z][c-f][str]'" %PS% }% |
+| [^abc] negated class | %RED% %SEARCH{"fields[name='CharClass'].value =~ 'T[^E]st[^A-Z]e'" %PS% }% |
+| \d 9999 shorthand for digits | %RED% %SEARCH{"Digits =~ 'x\d\d\dy'" %PS%}% |
+| \w shorthand for word characters | %RED% %SEARCH{"CharClass =~ '\we\wt\we'" %PS%}% |
+| \s shorthand for whitespace | %RED% %SEARCH{"Whitespace =~ '\s\s\s\s'" %PS%}% |
+| \D, \W and \S shorthand negated character classes char$$$erless | %RED% %SEARCH{"CharClass =~ '\Dest\Se' and Whitespace =~ '\W\D\W\D'" %PS%}% |
+| \b (at the beginning or end of a word) | %RED% %SEARCH{"Words =~ '\bWhy\b'" %PS%}% |
+| ? (0 or 1) | %RED% %SEARCH{"Digits=~'^x?9' and Digits =~ 'y\d?'" %PS%}% |
+| * (0 or more) | %RED% %SEARCH{"Digits=~'x9*y'" %PS%}% |
+| + (1 or more) | %RED% %SEARCH{"Digits=~'x\d+y'" %PS%}% |
+| {n} | %RED% %SEARCH{"Digits =~ '9{3}' and not Digits=~ '9{4}'" %PS%}% |
+| {n,m} | %RED% %SEARCH{"Digits =~ '9{1,3}'" %PS%}% |
+| {n,} | %RED% %SEARCH{"Digits =~ '9{2,}'" %PS%}% |
+| LIKE |  %RED% %SEARCH{"SimpleText ~ 'Test*' AND SimpleText ~ '?estMe*'" %PS%}% |
+
+%META:FIELD{name="SimpleText" value="TestMe ASCII"}%
+%META:FIELD{name="CharClass" value="Testme"}%
+%META:FIELD{name="Digits" value="x999y"}%
+%META:FIELD{name="Word" value="%25word%25"}%
+%META:FIELD{name="Whitespace" value=" 	%0d%0a"}%
+%META:FIELD{name="Words" value="Why oh_why Nine9ty"}%
+%META:FIELD{name="Backslash" value="x\y"}%

lib/Foswiki/Contrib/DBIStoreContrib.pm

@@ -2,11 +2,12 @@
 package Foswiki::Contrib::DBIStoreContrib;
 
 use strict;
+use Foswiki ();
 
-our $VERSION = '$Rev$';    # version of *this file*.
+our $VERSION = '1.1';    # version of *this file*.
 
-our $RELEASE = '1.0';
+our $RELEASE = '12 Dec 2013';
 
 our $SHORTDESCRIPTION =
-  '(Experimental) use of DBI to implement an SQL query search';
+'Use DBI to implement searching using an SQL database. Supports SQL queries over Form data.';
 

lib/Foswiki/Contrib/DBIStoreContrib/Config.spec

@@ -15,3 +15,8 @@ $Foswiki::cfg{Plugins}{DBIStorePlugin}{Module} = 'Foswiki::Plugins::DBIStorePlug
 # **BOOLEAN**
 # Plugin enable switch (required on Foswiki 1.1 and earlier)
 $Foswiki::cfg{Plugins}{DBIStorePlugin}{Enabled} = 0;
+# **STRING 80**
+# Where to find the PCRE library for SQLite. Only used by SQLite. It is
+# installed on Debian Linux using apt-get install sqlite3-pcre
+# (or similar on other systems).
+$Foswiki::cfg{Extensions}{DBIStoreContrib}{SQLite}{PCRE} = '/usr/lib/sqlite3/pcre.so';