Permalink
Browse files

General syncback from HEAD branch (2.0)

  • Loading branch information...
1 parent 4e48f83 commit 386aed2335d6d8e5bd80cfbcedaa462195469580 diml committed Nov 8, 2007
View
89 search/LISEZMOI.txt
@@ -0,0 +1,89 @@
+Cette distribution partielle contient une refonte du moteur de
+recherche globalde Moodle.
+
+Le moteur de recherche est capable d'indexer et de rechercher
+des informations dans un grand nombre de contenus stock�s
+dans la plate-forme � travers la manipulation des activit�s et
+des blocs.
+
+Le moteur de recherche proc�de � une premi�re indexation des
+ressources disponibles par action de l'administrateur. Une fois
+cette indexation effectu�e, le moteur maintient r�guli�rement les
+indexes, en ajoutant les nouvelles entr�es et en nettoyant les
+entr�es obsol�tes.
+
+La recherche permet d'obtenir des r�f�rences d'acc�s au contexte
+qui diffuse cette information, au nom de l'utilisateur courant.
+Le filtrage des r�sultats enl�ve de la liste des r�ponses toute
+ressource que la situation de l'utilisateur emp�cherait de voir
+s'il y acc�dait dans son contexte habituel.
+
+Mise en oeuvre
+##############
+
+Pour d�ployer le moteur :
+
+
+* Copie de fichiers
+
+1. Ajouter les deux librairies fournies aux librairies de Moodle
+2. Ecraser le r�pertoire "search" par le r�pertoire fourni
+3. Ecraser le bloc "blocs/search" par le bloc fourni.
+
+* Installation logique
+
+4. Aller dans les notifications administratives et d�rouler la proc�dure d'installation/mise � jour du bloc. L'installation cr�e la table image
+des documents index�s et utilis�s dans le module search.
+
+5. Ins�rer un nouveau bloc de recherche globale dans la plate-forme
+
+6. Effectuer une recherche vide (en administrateur)
+
+7. Aller sur la page des statistiques
+
+8. Activer l'indexation (indexsplash.php). Attention, si la plate-form contient beaucoup de contenus cette indexation peut �tre TRES LONGUE.
+
+Pour effectuer des recherches, une fois la premi�re indexation termin�e, retourner au bloc de recherche et tenter une recherche.
+
+El�ments pris en charge
+#######################
+
+Dans l'�tat actuel, les �l�ments index�s par le moteur sont :
+
+- les entr�es de forum
+- les fiches de base de donn�es
+- les commentaires sur fiches de donn�es
+- les entr�es de glossaire
+- les commentaires sur entr�es de glossaire
+- les ressources natives Moodle
+- les ressources physiques de type MSWord
+- les ressources physiques de type PDF
+- les ressources physiques de type fichier texte (.txt)
+- les ressources physiques de type HTML (.htm et .html)
+- les ressources physiques de type XML (.xml)
+- les ressources physiques de type (Microsoft) Powerpoint (.ppt)
+- les pages de wiki
+- les entit�s de projet technique
+- les sessions de chat
+
+Extensions
+##########
+
+L'API du moteur de recherche permet d�sormais :
+
+- l'indexation de contenus de blocs.
+- l'indexation de modules contenant une information complexe ou de plusieurs types distincts
+- la s�curisation des informations index�es lors des extractions de r�sultats
+- l'indexation de tout module tiers par ajout d'un fichier php calibr�
+- l'indexation de toute nouvelle resource physique par ajout d'un fichier php calibr�
+
+Extensions futures
+##################
+
+- De nouvelles prises en charge de contenus tels que les attachements des forums, les attachement des glossaires, ainsi que d'autres modules non encore
+impl�ment�s.
+
+- l'extension mnet de la recherche dans un r�seau de moodle interconnect�s.
+
+
+
View
91 search/READMETOO.txt
@@ -0,0 +1,91 @@
+This partial distribution contains a complete review of the
+Global Search Engine of Moodle.
+
+The Global Search Engine stores indexes about a huge quantity
+of information from within modules, block or resources stored
+by Moodle either in the database or the file system.
+
+The administrator initialy indexes the existing content. Once this
+first initialization performed, the search engine maintains indexes
+regularily, adding new entries, deleting obsolete one or updating
+some that have changed.
+
+Search will produce links for acceding the information in a similar
+context as usually accessed, from the current user point of view.
+Results filtering removes from results any link to information the
+current user would not be allowed to acces on a straight situation.
+
+Deployement
+###########
+
+For setting the engine :
+
+
+* File copy
+
+1. Add to Moodle's library both additional libraries provided in the distribution
+2. Replace the "search" directory with the new one
+3. Replace the "blocks/search" with the new one.
+
+* Logical install
+
+4. Browse to the administrative notification screen and let the
+install/update process run. The install process creates the Moodle
+table needed for backing the indexed documents identities.
+
+5. Go to the block administration panel and setup once the Global Search
+block. This will initialize useful parameters for the global search engine.
+
+6. Insert a new Global Search block somewhere in a course or top-level screen.
+
+7. Launch an empty search (you must be administrator).
+
+8. Go to the statistics screen.
+
+9. Activate indexation (indexersplash.php). Beware, if your Moodle has
+a large amount of content, indexing process may be VERY LONG.
+
+To search, go back to the search block and try a query.
+
+Handled information for indexing
+################################
+
+In the actual state, the engine indexes the following information:
+
+- forum posts
+- database records (using textual fields only)
+- database comments
+- glossary entries
+- glossary comments on entries
+- Moodle native resources
+- physical MSWord files as resources (.doc)
+- physical Powerpoint files as resources (.ppt)
+- physical PDF files as resources
+- physical text files as resources (.txt)
+- physical html files as resources (.htm and .html)
+- physical xml files as resources (.xml)
+- wiki pages
+- techproject descriptions
+- char sessions
+- lesson pages
+
+Extensions
+##########
+
+The reviewed search engine API allows:
+
+- indexing of blocks contents
+- indexation of modules or blocks containing a complex information model
+- securing the access to the results
+- adding indexing handling adding a php calibrated script
+- adding physical filetype handling adding a php calibrated script
+
+Future extensions
+#################
+
+- Should be added more information to index such as forum and glossary attachements, so will other standard module contents.
+
+- extending the search capability to a mnet network information space.
+
+
+
View
235 search/add.php
@@ -1,103 +1,144 @@
<?php
-
- require_once('../config.php');
- require_once("$CFG->dirroot/search/lib.php");
-
- require_login();
-
- if (empty($CFG->enableglobalsearch)) {
- error('Global searching is not enabled.');
- }
-
- if (!isadmin()) {
- error("You need to be an admin user to use this page.", "$CFG->wwwroot/login/index.php");
- } //if
-
- //check for php5 (lib.php)
- if (!search_check_php5()) {
+/**
+* Global Search Engine for Moodle
+* Michael Champanis (mchampan) [cynnical@gmail.com]
+* review 1.8+ : Valery Fremaux [valery.fremaux@club-internet.fr]
+* 2007/08/02
+*
+* Asynchronous adder for new indexable contents
+*
+* Major chages in this review is passing the xxxx_db_names return to
+* multiple arity to handle multiple document types modules
+*/
+
+require_once('../config.php');
+require_once("$CFG->dirroot/search/lib.php");
+
+require_login();
+
+if (empty($CFG->enableglobalsearch)) {
+ error(get_string('globalsearchdisabled', 'search'));
+}
+
+if (!isadmin()) {
+ error(get_string('beadmin', 'search'), "$CFG->wwwroot/login/index.php");
+}
+
+//check for php5 (lib.php)
+if (!search_check_php5()) {
$phpversion = phpversion();
mtrace("Sorry, global search requires PHP 5.0.0 or later (currently using version $phpversion)");
exit(0);
- } //if
-
- require_once("$CFG->dirroot/search/indexlib.php");
-
- $index = new Zend_Search_Lucene(SEARCH_INDEX_PATH);
- $dbcontrol = new IndexDBControl();
- $addition_count = 0;
-
- $indexdate = $CFG->search_indexer_run_date;
-
- mtrace('<pre>Starting index update (additions)...');
- mtrace('Index size before: '.$CFG->search_index_size."\n");
-
- //get all modules
- if ($mods = get_records_select('modules')) {
- //append virtual modules onto array
- $mods = array_merge($mods, search_get_additional_modules());
-
- foreach ($mods as $mod) {
- //build include file and function names
- $class_file = $CFG->dirroot.'/search/documents/'.$mod->name.'_document.php';
- $db_names_function = $mod->name.'_db_names';
- $get_document_function = $mod->name.'_single_document';
- $additions = array();
-
- if (file_exists($class_file)) {
- require_once($class_file);
-
- //if both required functions exist
- if (function_exists($db_names_function) and function_exists($get_document_function)) {
- mtrace("Checking $mod->name module for additions.");
- $values = $db_names_function();
- $where = (isset($values[4])) ? $values[4] : '';
-
- //select records in MODULE table, but not in SEARCH_DATABASE_TABLE
- $sql = "select id, ".$values[0]." as docid from ".$values[1].
- " where id not in".
- " (select docid from ".SEARCH_DATABASE_TABLE." where doctype like '$mod->name')".
- " and ".$values[2]." > $indexdate".
- " $where";
-
- $records = get_records_sql($sql);
-
- //foreach record, build a module specific search document using the get_document function
- if (is_array($records)) {
- foreach($records as $record) {
- $additions[] = $get_document_function($record->id);
- } //foreach
- } //if
-
- //foreach document, add it to the index and database table
- foreach ($additions as $add) {
- ++$addition_count;
-
- //object to insert into db
- $dbid = $dbcontrol->addDocument($add);
-
- //synchronise db with index
- $add->addField(Zend_Search_Lucene_Field::Keyword('dbid', $dbid));
-
- mtrace(" Add: $add->title (database id = $add->dbid, moodle instance id = $add->docid)");
-
- $index->addDocument($add);
- } //foreach
-
- mtrace("Finished $mod->name.\n");
- } //if
- } //if
- } //foreach
- } //if
-
- //commit changes
- $index->commit();
-
- //update index date and size
- set_config("search_indexer_run_date", time());
- set_config("search_index_size", (int)$CFG->search_index_size + (int)$addition_count);
-
- //print some additional info
- mtrace("Added $addition_count documents.");
- mtrace('Index size after: '.$index->count().'</pre>');
+}
+
+require_once("$CFG->dirroot/search/indexlib.php");
+
+$index = new Zend_Search_Lucene(SEARCH_INDEX_PATH);
+$dbcontrol = new IndexDBControl();
+$addition_count = 0;
+$startindextime = time();
+
+$indexdate = $CFG->search_indexer_run_date;
+
+mtrace('<pre>Starting index update (additions)...');
+mtrace('Index size before: '.$CFG->search_index_size."\n");
+
+//get all modules
+if ($mods = get_records_select('modules')) {
+
+//append virtual modules onto array
+$mods = array_merge($mods, search_get_additional_modules());
+ foreach ($mods as $mod) {
+ //build include file and function names
+ $class_file = $CFG->dirroot.'/search/documents/'.$mod->name.'_document.php';
+ $db_names_function = $mod->name.'_db_names';
+ $get_document_function = $mod->name.'_single_document';
+ $get_newrecords_function = $mod->name.'_new_records';
+ $additions = array();
+
+ if (file_exists($class_file)) {
+ require_once($class_file);
+
+ //if both required functions exist
+ if (function_exists($db_names_function) and function_exists($get_document_function)) {
+ mtrace("Checking $mod->name module for additions.");
+ $valuesArray = $db_names_function();
+ if ($valuesArray){
+ foreach($valuesArray as $values){
+ $where = (isset($values[5])) ? 'AND ('.$values[5].')' : '';
+ $itemtypes = ($values[4] != '*') ? " AND itemtype = '{$values[4]}' " : '' ;
+
+ //select records in MODULE table, but not in SEARCH_DATABASE_TABLE
+ $table = SEARCH_DATABASE_TABLE;
+ $query = "
+ SELECT
+ docid,
+ itemtype
+ FROM
+ {$CFG->prefix}{$table}
+ WHERE
+ doctype = '{$mod->name}'
+ $itemtypes
+ ";
+ $docIds = get_records_sql_menu($query);
+ $docIdList = ($docIds) ? implode("','", array_keys($docIds)) : '' ;
+
+ $query = "
+ SELECT id,
+ {$values[0]} as docid
+ FROM
+ {$CFG->prefix}{$values[1]}
+ WHERE
+ id NOT IN ('{$docIdList}') and
+ {$values[2]} > {$indexdate}
+ $where
+ ";
+ $records = get_records_sql($query);
+
+ // foreach record, build a module specific search document using the get_document function
+ if (is_array($records)) {
+ foreach($records as $record) {
+ $add = $get_document_function($record->docid, $values[4]);
+ // some documents may not be indexable
+ if ($add)
+ $additions[] = $add;
+ }
+ }
+ }
+
+ // foreach document, add it to the index and database table
+ foreach ($additions as $add) {
+ ++$addition_count;
+
+ // object to insert into db
+ $dbid = $dbcontrol->addDocument($add);
+
+ // synchronise db with index
+ $add->addField(Zend_Search_Lucene_Field::Keyword('dbid', $dbid));
+
+ mtrace(" Add: $add->title (database id = $add->dbid, moodle instance id = $add->docid)");
+
+ $index->addDocument($add);
+ }
+ }
+ else{
+ mtrace("No types to add.\n");
+ }
+ mtrace("Finished $mod->name.\n");
+ }
+ }
+ }
+}
+
+// commit changes
+$index->commit();
+
+// update index date and size
+set_config("search_indexer_run_date", $startindextime);
+set_config("search_index_size", (int)$CFG->search_index_size + (int)$addition_count);
+
+// print some additional info
+mtrace("Added $addition_count documents.");
+mtrace('Index size after: '.$index->count().'</pre>');
?>
View
55 search/cron.php
@@ -1,35 +1,28 @@
<?php
- /* cron script to perform all the periodic search tasks
- *
- * delete.php
- * updates the index by pruning deleted documents
- *
- * update.php
- * updates document info in the index if the document has been modified since indexing
- *
- * add.php
- * adds documents created since the last index run
- */
+/* cron script to perform all the periodic search tasks
+*
+* delete.php
+* updates the index by pruning deleted documents
+*
+* update.php
+* updates document info in the index if the document has been modified since indexing
+*
+* add.php
+* adds documents created since the last index run
+*/
- require_once('../config.php');
- require_once("$CFG->dirroot/search/lib.php");
+ require_once('../config.php');
+ require_once("$CFG->dirroot/search/lib.php");
- if (empty($CFG->enableglobalsearch)) {
- mtrace('Global searching is not enabled.');
- return;
- }
-
- mtrace("<pre>Starting cron...\n");
-
- mtrace("--DELETE----");
- require_once("$CFG->dirroot/search/delete.php");
- mtrace("--UPDATE----");
- require_once("$CFG->dirroot/search/update.php");
- mtrace("--ADD-------");
- require_once("$CFG->dirroot/search/add.php");
- mtrace("------------");
-
- mtrace("cron finished.</pre>");
-
-?>
+ if (!search_check_php5()) {
+ $phpversion = phpversion();
+ mtrace("Sorry, cannot cron global search as it requires PHP 5.0.0 or later (currently using version $phpversion)");
+ }
+ else if (empty($CFG->enableglobalsearch)) {
+ mtrace('Global searching is not enabled. Nothing performed by search.');
+ }
+ else{
+ include("{$CFG->dirroot}/search/cron_php5.php");
+ }
+?>
View
32 search/cron_php5.php
@@ -0,0 +1,32 @@
+<?php
+/*
+* Moodle global search engine
+* This is a special externalized code for cron handling in PHP5.
+* Should never be called by a php 4.3.0 implementation.
+*/
+try{
+ // overrides php limits
+ $maxtimelimit = ini_get('max_execution_time');
+ ini_set('max_execution_time', 300);
+ $maxmemoryamount = ini_get('memory_limit');
+ ini_set('memory_limit', '48M');
+
+ mtrace("<pre>Starting cron...\n");
+ mtrace("--DELETE----");
+ require_once("$CFG->dirroot/search/delete.php");
+ mtrace("--UPDATE----");
+ require_once("$CFG->dirroot/search/update.php");
+ mtrace("--ADD-------");
+ require_once("$CFG->dirroot/search/add.php");
+ mtrace("------------");
+ mtrace("cron finished.</pre>");
+
+ // set back normal values for php limits
+ ini_set('max_execution_time', $maxtimelimit);
+ ini_set('memory_limit', $maxmemoryamount);
+}
+catch(Exception $ex){
+ mtrace('Fatal exception from Lucene subsystem. Search engine may not have been updated.');
+ mtrace($ex);
+}
+?>
View
12 search/db/mysql.sql
@@ -1,12 +0,0 @@
-CREATE TABLE IF NOT EXISTS `prefix_search_documents` (
- `id` int(11) NOT NULL auto_increment,
- `docid` int(11) NOT NULL,
- `doctype` varchar(12) NOT NULL default 'none',
- `title` varchar(100) NOT NULL default '',
- `url` varchar(100) NOT NULL default '',
- `docdate` timestamp NOT NULL default '0',
- `updated` timestamp NOT NULL default CURRENT_TIMESTAMP,
- `courseid` int(11) NOT NULL default '0',
- `groupid` int(11) NOT NULL default '0',
- PRIMARY KEY (`id`)
-) ENGINE=MyISAM AUTO_INCREMENT=1;
View
11 search/db/postgres7.sql
@@ -1,11 +0,0 @@
-CREATE TABLE prefix_search_documents (
- id SERIAL8 PRIMARY KEY,
- docid int4 NOT NULL,
- doctype varchar(12) NOT NULL DEFAULT 'none',
- title varchar(100) NOT NULL default '',
- url varchar(100) NOT NULL default '',
- docdate timestamp NOT NULL,
- updated timestamp NOT NULL DEFAULT NOW(),
- courseid int4,
- groupid int4
-);
View
214 search/delete.php
@@ -1,94 +1,132 @@
<?php
-
- require_once('../config.php');
- require_once("$CFG->dirroot/search/lib.php");
-
- require_login();
-
- if (empty($CFG->enableglobalsearch)) {
- error('Global searching is not enabled.');
- }
-
- if (!isadmin()) {
- error("You need to be an admin user to use this page.", "$CFG->wwwroot/login/index.php");
- } //if
-
- //check for php5 (lib.php)
- if (!search_check_php5()) {
+/**
+* Global Search Engine for Moodle
+* Michael Champanis (mchampan) [cynnical@gmail.com]
+* review 1.8+ : Valery Fremaux [valery.fremaux@club-internet.fr]
+* 2007/08/02
+*
+* Asynchronous index cleaner
+*
+* Major chages in this review is passing the xxxx_db_names return to
+* multiple arity to handle multiple document types modules
+*/
+
+require_once('../config.php');
+require_once("$CFG->dirroot/search/lib.php");
+
+require_login();
+
+if (empty($CFG->enableglobalsearch)) {
+ error(get_string('globalsearchdisabled', 'search'));
+}
+
+if (!isadmin()) {
+ error(get_string('beadmin', 'search'), "$CFG->wwwroot/login/index.php");
+} //if
+
+//check for php5 (lib.php)
+if (!search_check_php5()) {
$phpversion = phpversion();
mtrace("Sorry, global search requires PHP 5.0.0 or later (currently using version $phpversion)");
exit(0);
- } //if
-
- require_once("$CFG->dirroot/search/indexlib.php");
-
- $index = new Zend_Search_Lucene(SEARCH_INDEX_PATH);
- $dbcontrol = new IndexDBControl();
- $deletion_count = 0;
-
- mtrace('<pre>Starting clean-up of removed records...');
- mtrace('Index size before: '.$CFG->search_index_size."\n");
-
- if ($mods = get_records_select('modules')) {
- $mods = array_merge($mods, search_get_additional_modules());
-
- foreach ($mods as $mod) {
- //build function names
- $class_file = $CFG->dirroot.'/search/documents/'.$mod->name.'_document.php';
- $delete_function = $mod->name.'_delete';
- $db_names_function = $mod->name.'_db_names';
- $deletions = array();
-
- if (file_exists($class_file)) {
- require_once($class_file);
-
- if (function_exists($delete_function) and function_exists($db_names_function)) {
- mtrace("Checking $mod->name module for deletions.");
- $values = $db_names_function();
-
- $sql = "select id, docid from ".SEARCH_DATABASE_TABLE.
- " where doctype like '$mod->name'".
- " and docid not in".
- " (select ".$values[0]." from ".$values[1].")";
-
- $records = get_records_sql($sql);
-
- //build an array of all the deleted records
- if (is_array($records)) {
- foreach($records as $record) {
- $deletions[] = $delete_function($record->docid);
- } //foreach
- } //if
-
- foreach ($deletions as $delete) {
- //find the specific document in the index, using it's docid and doctype as keys
- $doc = $index->find("+docid:$delete +doctype:$mod->name");
-
- //get the record, should only be one
- foreach ($doc as $thisdoc) {
- ++$deletion_count;
- mtrace(" Delete: $thisdoc->title (database id = $thisdoc->dbid, index id = $thisdoc->id, moodle instance id = $thisdoc->docid)");
-
- //remove it from index and database table
- $dbcontrol->delDocument($thisdoc);
- $index->delete($thisdoc->id);
- } //foreach
- } //foreach
-
- mtrace("Finished $mod->name.\n");
- } //if
- } //if
- } //foreach
- } //if
-
- //commit changes
- $index->commit();
-
- //update index date and index size
- set_config("search_indexer_run_date", time());
- set_config("search_index_size", (int)$CFG->search_index_size - (int)$deletion_count);
-
- mtrace("Finished $deletion_count removals.");
- mtrace('Index size after: '.$index->count().'</pre>');
+}
+
+require_once("$CFG->dirroot/search/indexlib.php");
+
+$index = new Zend_Search_Lucene(SEARCH_INDEX_PATH);
+$dbcontrol = new IndexDBControl();
+$deletion_count = 0;
+$startcleantime = time();
+
+mtrace('<pre>Starting clean-up of removed records...');
+mtrace('Index size before: '.$CFG->search_index_size."\n");
+
+if ($mods = get_records_select('modules')) {
+ $mods = array_merge($mods, search_get_additional_modules());
+
+ foreach ($mods as $mod) {
+ //build function names
+ $class_file = $CFG->dirroot.'/search/documents/'.$mod->name.'_document.php';
+ $delete_function = $mod->name.'_delete';
+ $db_names_function = $mod->name.'_db_names';
+ $deletions = array();
+
+ if (file_exists($class_file)) {
+ require_once($class_file);
+
+ //if both required functions exist
+ if (function_exists($delete_function) and function_exists($db_names_function)) {
+ mtrace("Checking $mod->name module for deletions.");
+ $valuesArray = $db_names_function();
+ if ($valuesArray){
+ foreach($valuesArray as $values){
+ $where = (isset($values[5])) ? 'WHERE '.$values[5] : '';
+ $itemtypes = ($values[4] != '*') ? " itemtype = '{$values[4]}' AND " : '' ;
+ $query = "
+ SELECT
+ id,
+ {$values[0]}
+ FROM
+ {$CFG->prefix}{$values[1]}
+ $where
+ ";
+ $docIds = get_records_sql($query);
+ $docIdList = ($docIds) ? implode("','", array_keys($docIds)) : '' ;
+
+ $table = SEARCH_DATABASE_TABLE;
+ $query = "
+ SELECT
+ id,
+ docid
+ FROM
+ {$CFG->prefix}{$table}
+ WHERE
+ doctype = '{$mod->name}' AND
+ $itemtypes
+ docid not in ('{$docIdList}')
+ ";
+ $records = get_records_sql($query);
+
+ // build an array of all the deleted records
+ if (is_array($records)) {
+ foreach($records as $record) {
+ $deletions[] = $delete_function($record->docid, $values[4]);
+ }
+ }
+ }
+
+ foreach ($deletions as $delete) {
+ // find the specific document in the index, using it's docid and doctype as keys
+ $doc = $index->find("+docid:{$delete->id} +doctype:$mod->name +itemtype:{$delete->itemtype}");
+
+ // get the record, should only be one
+ foreach ($doc as $thisdoc) {
+ ++$deletion_count;
+ mtrace(" Delete: $thisdoc->title (database id = $thisdoc->dbid, index id = $thisdoc->id, moodle instance id = $thisdoc->docid)");
+
+ //remove it from index and database table
+ $dbcontrol->delDocument($thisdoc);
+ $index->delete($thisdoc->id);
+ }
+ }
+ }
+ else{
+ mtrace("No types to delete.\n");
+ }
+ mtrace("Finished $mod->name.\n");
+ }
+ }
+ }
+}
+
+//commit changes
+$index->commit();
+
+//update index date and index size
+set_config("search_indexer_cleanup_date", $startcleantime);
+set_config("search_index_size", (int)$CFG->search_index_size - (int)$deletion_count);
+
+mtrace("Finished $deletion_count removals.");
+mtrace('Index size after: '.$index->count().'</pre>');
?>
View
294 search/documents/chat_document.php
@@ -0,0 +1,294 @@
+<?php
+/**
+* Global Search Engine for Moodle
+* add-on 1.8+ : Valery Fremaux [valery.fremaux@club-internet.fr]
+* 2007/08/02
+*
+* document handling for chat activity module
+* This file contains the mapping between a chat history and it's indexable counterpart,
+*
+* Functions for iterating and retrieving the necessary records are now also included
+* in this file, rather than mod/chat/lib.php
+*
+* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
+* @package search
+* @version 2007110400
+**/
+
+require_once("$CFG->dirroot/search/documents/document.php");
+require_once("$CFG->dirroot/mod/chat/lib.php");
+
+/*
+* a class for representing searchable information
+*
+**/
+class ChatTrackSearchDocument extends SearchDocument {
+
+ /**
+ * constructor
+ *
+ */
+ public function __construct(&$chatsession, $chat_module_id, $course_id, $group_id, $context_id) {
+ // generic information; required
+ $doc->docid = $chat_module_id.'-'.$chatsession['sessionstart'].'-'.$chatsession['sessionend'];
+ $doc->documenttype = SEARCH_TYPE_CHAT;
+ $doc->itemtype = 'session';
+ $doc->contextid = $context_id;
+
+ $duration = $chatsession['sessionend'] - $chatsession['sessionstart'];
+ // we cannot call userdate with relevant locale at indexing time.
+ $doc->title = get_string('chatreport', 'chat').' '.get_string('openedon', 'search').' TT_'.$chatsession['sessionstart'].'_TT ('.get_string('duration', 'search').' : '.get_string('numseconds', '', $duration).')';
+ $doc->date = $chatsession['sessionend'];
+
+ //remove '(ip.ip.ip.ip)' from chat author list
+ $doc->author = preg_replace('/\(.*?\)/', '', $chatsession['authors']);
+ $doc->contents = $chatsession['content'];
+ $doc->url = chat_make_link($chat_module_id, $chatsession['sessionstart'], $chatsession['sessionend']);
+
+ // module specific information; optional
+ $data->chat = $chat_module_id;
+
+ // construct the parent class
+ parent::__construct($doc, $data, $course_id, $group_id, 0, PATH_FOR_SEARCH_TYPE_CHAT);
+ } //constructor
+} //ChatTrackSearchDocument
+
+
+/**
+* constructs a valid link to a chat content
+* @param cm_id the chat course module
+* @param start the start time of the session
+* @param end th end time of the session
+* @return a well formed link to session display
+*/
+function chat_make_link($cm_id, $start, $end) {
+ global $CFG;
+
+ return $CFG->wwwroot.'/mod/chat/report.php?id='.$cm_id.'&amp;start='.$start.'&amp;end='.$end;
+} //chat_make_link
+
+/**
+* fetches all the records for a given session and assemble them as a unique track
+* we revamped here the code of report.php for making sessions, but without any output.
+* note that we should collect sessions "by groups" if groupmode() is SEPARATEGROUPS.
+* @param chat_id the database
+* @return an array of objects representing the chat sessions.
+*/
+function chat_get_session_tracks($chat_id, $fromtime = 0, $totime = 0) {
+ global $CFG;
+
+ $chat = get_record('chat', 'id', $chat_id);
+ $course = get_record('course', 'id', $chat->course);
+ $coursemodule = get_field('modules', 'id', 'name', 'data');
+ $cm = get_record('course_modules', 'course', $course->id, 'module', $coursemodule, 'instance', $chat->id);
+ $groupmode = groupmode($course, $cm);
+
+ $fromtimeclause = ($fromtime) ? "AND timestamp >= {$fromtime}" : '';
+ $totimeclause = ($totime) ? "AND timestamp <= {$totime}" : '';
+ $tracks = array();
+ $messages = get_records_select('chat_messages', "chatid = '{$chat_id}' $fromtimeclause $totimeclause", "timestamp DESC");
+ if ($messages){
+ // splits discussions against groups
+ $groupedMessages = array();
+ if ($groupmode != SEPARATEGROUPS){
+ foreach($messages as $aMessage){
+ $groupedMessages[$aMessage->groupid][] = $aMessage;
+ }
+ }
+ else{
+ $groupedMessages[-1] = &$messages;
+ }
+ $sessiongap = 5 * 60; // 5 minutes silence means a new session
+ $sessionend = 0;
+ $sessionstart = 0;
+ $sessionusers = array();
+ $lasttime = time();
+
+ foreach ($groupedMessages as $groupId => $messages) { // We are walking BACKWARDS through the messages
+ $messagesleft = count($messages);
+ foreach ($messages as $message) { // We are walking BACKWARDS through the messages
+ $messagesleft --; // Countdown
+
+ if ($message->system) {
+ continue;
+ }
+ // we are within a session track
+ if ((($lasttime - $message->timestamp) < $sessiongap) and $messagesleft) { // Same session
+ if (count($tracks) > 0){
+ if ($message->userid) { // Remember user and count messages
+ $tracks[count($tracks) - 1]->sessionusers[$message->userid] = $message->userid;
+ // update last track (if exists) record appending content (remember : we go backwards)
+ }
+ $tracks[count($tracks) - 1]->content .= ' '.$message->message;
+ $tracks[count($tracks) - 1]->sessionstart = $message->timestamp;
+ }
+ }
+ // we initiate a new session track (backwards)
+ else {
+ $track = new Object();
+ $track->sessionend = $message->timestamp;
+ $track->sessionstart = $message->timestamp;
+ $track->content = $message->message;
+ // reset the accumulator of users
+ $track->sessionusers = array();
+ $track->sessionusers[$message->userid] = $message->userid;
+ $track->groupid = $groupId;
+ $tracks[] = $track;
+ }
+ $lasttime = $message->timestamp;
+ }
+ }
+ }
+ return $tracks;
+} //chat_get_session_tracks
+
+/**
+* part of search engine API
+*
+*/
+function chat_iterator() {
+ $chatrooms = get_records('chat');
+ return $chatrooms;
+} //chat_iterator
+
+/**
+* part of search engine API
+*
+*/
+function chat_get_content_for_index(&$chat) {
+ $documents = array();
+ $course = get_record('course', 'id', $chat->course);
+ $coursemodule = get_field('modules', 'id', 'name', 'chat');
+ $cm = get_record('course_modules', 'course', $chat->course, 'module', $coursemodule, 'instance', $chat->id);
+ if ($cm){
+ $context = get_context_instance(CONTEXT_MODULE, $cm->id);
+
+ // getting records for indexing
+ $sessionTracks = chat_get_session_tracks($chat->id);
+ if ($sessionTracks){
+ foreach($sessionTracks as $aTrackId => $aTrack) {
+ foreach($aTrack->sessionusers as $aUserId){
+ $user = get_record('user', 'id', $aUserId);
+ $aTrack->authors = ($user) ? $user->firstname.' '.$user->lastname : '' ;
+ $documents[] = new ChatTrackSearchDocument(get_object_vars($aTrack), $cm->id, $chat->course, $aTrack->groupid, $context->id);
+ }
+ }
+ }
+ return $documents;
+ }
+ return array();
+} //chat_get_content_for_index
+
+/**
+* returns a single data search document based on a chat_session id
+* chat session id is a text composite identifier made of :
+* - the chat id
+* - the timestamp when the session starts
+* - the timestamp when the session ends
+* @param id the multipart chat session id
+* @param itemtype the type of information (session is the only type)
+*/
+function chat_single_document($id, $itemtype) {
+ list($chat_id, $sessionstart, $sessionend) = split('-', $id);
+ $chat = get_record('chat', 'id', $chat_id);
+ $course = get_record('course', 'id', $chat->course);
+ $coursemodule = get_field('modules', 'id', 'name', 'chat');
+ $cm = get_record('course_modules', 'course', $course->id, 'module', $coursemodule, 'instance', $chat->id);
+ if ($cm){
+ $context = get_context_instance(CONTEXT_MODULE, $cm->id);
+
+ // should be only one
+ $tracks = chat_get_session_tracks($chat->id, $sessionstart, $sessionstart);
+ if ($tracks){
+ $aTrack = $tracks[0];
+ $document = new ChatTrackSearchDocument(get_object_vars($aTrack), $cm->id, $chat->course, $aTrack->groupid, $context->id);
+ }
+ return $document;
+ }
+ return null;
+} //chat_single_document
+
+/**
+* dummy delete function that packs id with itemtype.
+* this was here for a reason, but I can't remember it at the moment.
+*
+*/
+function chat_delete($info, $itemtype) {
+ $object->id = $info;
+ $object->itemtype = $itemtype;
+ return $object;
+} //chat_delete
+
+/**
+* returns the var names needed to build a sql query for addition/deletions
+* // TODO chat indexable records are virtual. Should proceed in a special way
+*/
+function chat_db_names() {
+ //[primary id], [table name], [time created field name], [time modified field name]
+ return null;
+} //chat_db_names
+
+/**
+* this function handles the access policy to contents indexed as searchable documents. If this
+* function does not exist, the search engine assumes access is allowed.
+* When this point is reached, we already know that :
+* - user is legitimate in the surrounding context
+* - user may be guest and guest access is allowed to the module
+* - the function may perform local checks within the module information logic
+* @param path the access path to the module script code
+* @param itemtype the information subclassing (usefull for complex modules, defaults to 'standard')
+* @param this_id the item id within the information class denoted by entry_type. In chats, this id
+* points out a session history which is a close sequence of messages.
+* @param user the user record denoting the user who searches
+* @param group_id the current group used by the user when searching
+* @return true if access is allowed, false elsewhere
+*/
+function chat_check_text_access($path, $itemtype, $this_id, $user, $group_id, $context_id){
+ global $CFG;
+
+ include_once("{$CFG->dirroot}/{$path}/lib.php");
+
+ list($chat_id, $sessionstart, $sessionend) = split('-', $id);
+
+ // get the chat session and all related stuff
+ $chat = get_record('chat', 'id', $chat_id);
+ $context = get_record('context', 'id', $context_id);
+ $cm = get_record('course_modules', 'id', $context->instanceid);
+ // $cm = get_coursemodule_from_instance('chat', $chat->id, $chat->course);
+ // $context = get_context_instance(CONTEXT_MODULE, $cm->id);
+
+ if (!$cm->visible and !has_capability('moodle/course:viewhiddenactivities', $context)){
+ if (!empty($CFG->search_access_debug)) echo "search reject : hidden chat ";
+ return false;
+ }
+
+ //group consistency check : checks the following situations about groups
+ // trap if user is not same group and groups are separated
+ $current_group = get_current_group($course->id);
+ $course = get_record('course', 'id', $chat->course);
+ if ((groupmode($course, $cm) == SEPARATEGROUPS) && !ismember($group_id) && !has_capability('moodle/site:accessallgroups', $context)){
+ if (!empty($CFG->search_access_debug)) echo "search reject : chat element is in separated group ";
+ return false;
+ }
+
+ //ownership check : checks the following situations about user
+ // trap if user is not owner and has cannot see other's entries
+ // TODO : typically may be stored into indexing cache
+ if (!has_capability('mod/chat:readlog', $context)){
+ if (!empty($CFG->search_access_debug)) echo "search reject : cannot read past sessions ";
+ return false;
+ }
+
+ return true;
+} //chat_check_text_access
+
+/**
+* this call back is called when displaying the link for some last post processing
+*
+*/
+function chat_link_post_processing($title){
+ setLocale(LC_TIME, substr(current_language(), 0, 2));
+ $title = preg_replace('/TT_(.*)_TT/e', "userdate(\\1)", $title);
+ return $title;
+} //chat_link_post_processing
+?>
View
398 search/documents/data_document.php
@@ -0,0 +1,398 @@
+<?php
+/**
+* Global Search Engine for Moodle
+* add-on 1.8+ : Valery Fremaux [valery.fremaux@club-internet.fr]
+* 2007/08/02
+*
+* document handling for data activity module
+* This file contains the mapping between a database object and it's indexable counterpart,
+*
+* Functions for iterating and retrieving the necessary records are now also included
+* in this file, rather than mod/data/lib.php
+*
+* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
+* @package search
+* @version 2007110400
+**/
+
+require_once("$CFG->dirroot/search/documents/document.php");
+require_once("$CFG->dirroot/mod/data/lib.php");
+
+/*
+* a class for representing searchable information (data records)
+*
+**/
+class DataSearchDocument extends SearchDocument {
+
+ /**
+ * constructor
+ *
+ */
+ public function __construct(&$record, $course_id, $context_id) {
+ // generic information; required
+ $doc->docid = $record['id'];
+ $doc->documenttype = SEARCH_TYPE_DATA;
+ $doc->itemtype = 'record';
+ $doc->contextid = $context_id;
+
+ $doc->title = $record['title'];
+ $doc->date = $record['timemodified'];
+ //remove '(ip.ip.ip.ip)' from data record author field
+ if ($record['userid']){
+ $user = get_record('user', 'id', $record['userid']);
+ }
+ $doc->author = (isset($user)) ? $user->firstname.' '.$user->lastname : '' ;
+ $doc->contents = $record['content'];
+ $doc->url = data_make_link($record['dataid'], $record['id']);
+
+ // module specific information; optional
+ // $data->params = serialize(@$record['params']); may be useful
+ $data->database = $record['dataid'];
+
+ // construct the parent class
+ parent::__construct($doc, $data, $course_id, $record['groupid'], $record['userid'], PATH_FOR_SEARCH_TYPE_DATA);
+ } //constructor
+} //ChatSearchDocument
+
+/*
+* a class for representing searchable information (comments on data records)
+*
+**/
+class DataCommentSearchDocument extends SearchDocument {
+
+ /**
+ * constructor
+ *
+ */
+ public function __construct(&$comment, $course_id, $context_id) {
+ // generic information; required
+ $doc->docid = $comment['id'];
+ $doc->documenttype = SEARCH_TYPE_DATA;
+ $doc->itemtype = 'comment';
+ $doc->contextid = $context_id;
+
+ $doc->title = get_string('commenton', 'search').' '.$comment['title'];
+ $doc->date = $comment['modified'];
+ //remove '(ip.ip.ip.ip)' from data record author field
+ $doc->author = preg_replace('/\(.*?\)/', '', $comment['author']);
+ $doc->contents = $comment['content'];
+ $doc->url = data_make_link($data_id, $comment['recordid']);
+
+ // module specific information; optional
+ $data->database = $comment['dataid'];
+
+ // construct the parent class
+ parent::__construct($doc, $data, $course_id, $comment['groupid'], $comment['userid'], PATH_FOR_SEARCH_TYPE_DATA);
+ } //constructor
+} //ChatCommentSearchDocument
+
+/**
+* constructs a valid link to a data record content
+* @param database_id the database reference
+* @param record_id the record reference
+* @return a valid url top access the information as a string
+*/
+function data_make_link($database_id, $record_id) {
+ global $CFG;
+
+ return $CFG->wwwroot.'/mod/data/view.php?d='.$database_id.'&amp;rid='.$record_id;
+} //data_make_link
+
+/**
+* fetches all the records for a given database
+* @param database_id the database
+* @param typematch a comma separated list of types that should be considered for searching or *
+* @return an array of objects representing the data records.
+*/
+function data_get_records($database_id, $typematch = '*') {
+ global $CFG;
+
+ $fieldset = get_records('data_fields', 'dataid', $database_id);
+ $query = "
+ SELECT
+ c.*
+ FROM
+ {$CFG->prefix}data_content as c,
+ {$CFG->prefix}data_records as r
+ WHERE
+ c.recordid = r.id AND
+ r.dataid = {$database_id}
+ ORDER BY
+ c.fieldid
+ ";
+ $data = get_records_sql($query);
+ $records = array();
+ if ($data){
+ foreach($data as $aDatum){
+ if($typematch == '*' || preg_match("/\\b{$fieldset[$aDatum->fieldid]->type}\\b/", $typematch)){
+ if (!isset($records[$aDatum->recordid])){
+ $records[$aDatum->recordid]['_first'] = $aDatum->content.' '.$aDatum->content1.' '.$aDatum->content2.' '.$aDatum->content3.' '.$aDatum->content4.' ';
+ }
+ else{
+ $records[$aDatum->recordid][$fieldset[$aDatum->fieldid]->name] = $aDatum->content.' '.$aDatum->content1.' '.$aDatum->content2.' '.$aDatum->content3.' '.$aDatum->content4.' ';
+ }
+ }
+ }
+ }
+ return $records;
+} //data_get_records
+
+/**
+* fetches all the comments for a given database
+* @param database_id the database
+* @return an array of objects representing the data record comments.
+*/
+function data_get_comments($database_id) {
+ global $CFG;
+
+ $query = "
+ SELECT
+ c.id,
+ r.groupid,
+ c.userid,
+ c.recordid,
+ c.content,
+ c.created,
+ c.modified,
+ r.dataid
+ FROM
+ {$CFG->prefix}data_comments as c,
+ {$CFG->prefix}data_records as r
+ WHERE
+ c.recordid = r.id
+ ";
+ $comments = get_records_sql($query);
+ return $comments;
+} //data_get_comments
+
+
+/**
+* part of search engine API
+*
+*/
+function data_iterator() {
+ $databases = get_records('data');
+ return $databases;
+} //data_iterator
+
+/**
+* part of search engine API
+* @param database the database instance
+* @return an array of searchable documents
+*/
+function data_get_content_for_index(&$database) {
+
+ $documents = array();
+ $recordTitles = array();
+ $coursemodule = get_field('modules', 'id', 'name', 'data');
+ $cm = get_record('course_modules', 'course', $database->course, 'module', $coursemodule, 'instance', $database->id);
+ $context = get_context_instance(CONTEXT_MODULE, $cm->id);
+
+ // getting records for indexing
+ $records_content = data_get_records($database->id, 'text');
+ if ($records_content){
+ foreach(array_keys($records_content) as $aRecordId) {
+
+ // extract title as first record in order
+ $first = $records_content[$aRecordId]['_first'];
+ unset($records_content[$aRecordId]['_first']);
+
+ // concatenates all other texts
+ foreach($records_content[$aRecordId] as $aField){
+ $content = @$content.' '.$aField;
+ }
+ if (strlen($content) > 0) {
+ unset($recordMetaData);
+ $recordMetaData = get_record('data_records', 'id', $aRecordId);
+ $recordMetaData->title = $first;
+ $recordTitles[$aRecordId] = $first;
+ $recordMetaData->content = $content;
+ $documents[] = new DataSearchDocument(get_object_vars($recordMetaData), $database->course, $context->id);
+ }
+ }
+ }
+
+ // getting comments for indexing
+ $records_comments = data_get_comments($database->id);
+ if ($records_comments){
+ foreach($records_comments as $aComment){
+ $aComment->title = $recordsTitle[$aComment->recordid];
+ $documents[] = new DataCommentSearchDocument(get_object_vars($aComment), $database->course, $context->id);
+ }
+ }
+ return $documents;
+} //data_get_content_for_index
+
+/**
+* returns a single data search document based on a data entry id
+* @param id the id of the record
+* @param the type of the information
+* @return a single searchable document
+*/
+function data_single_document($id, $itemtype) {
+
+ if ($itemtype == 'record'){
+ // get main record
+ $recordMetaData = get_record('data_records', 'id', $id);
+ // get context
+ $record_course = get_field('data', 'course', 'id', $recordMetaData->dataid);
+ $coursemodule = get_field('modules', 'id', 'name', 'data');
+ $cm = get_record('course_modules', 'course', $record_course, 'module', $coursemodule, 'instance', $recordMetaData->dataid);
+ $context = get_context_instance(CONTEXT_MODULE, $cm->id);
+ // compute text
+ $recordData = get_records_select('data_content', "recordid = $id AND type = 'text'", 'recordid');
+ $accumulator = '';
+ if ($recordData){
+ $first = $recordData[0];
+ if (count($recordData) > 1){
+ $others = array_splice($recordData, 0, 1);
+ foreach($others as $aDatum){
+ $accumulator .= $data->content.' '.$data->content1.' '.$data->content2.' '.$data->content3.' '.$data->content4.' ';
+ }
+ }
+ }
+ // add extra fields
+ $recordMetaData->title = $first;
+ $recordMetaData->content = $accumulator;
+ // make document
+ $documents[] = new DataSearchDocument(get_object_vars($recordMetaData), $record_course, $context->id);
+ }
+ elseif($itemtype == 'comment'){
+ // get main records
+ $comment = get_record('data_comments', 'id', $id);
+ $record = get_record('data_records', 'id', $comment->recordid);
+ // get context
+ $record_course = get_field('data', 'course', 'id', $record->dataid);
+ $coursemodule = get_field('modules', 'id', 'name', 'data');
+ $cm = get_record('course_modules', 'course', $record_course, 'module', $coursemodule, 'instance', $recordMetaData->dataid);
+ $context = get_context_instance(CONTEXT_MODULE, $cm->id);
+ // add extra fields
+ $comment->title = get_field('search_document', 'title', 'docid', $record->id, 'itemtype', 'record');
+ $comment->dataid = $record->dataid;
+ $comment->groupid = $record->groupid;
+ // make document
+ $documents[] = new DataCommentSearchDocument(get_object_vars($comment), $record_course, $context->id);
+ }
+ else{
+ mtrace('Error : bad or missing item type');
+ }
+} //data_single_document
+
+/**
+* dummy delete function that packs id with itemtype.
+* this was here for a reason, but I can't remember it at the moment.
+*
+*/
+function data_delete($info, $itemtype) {
+ $object->id = $info;
+ $object->itemtype = $itemtype;
+ return $object;
+} //data_delete
+
+/**
+* returns the var names needed to build a sql query for addition/deletions
+*
+*/
+function data_db_names() {
+ //[primary id], [table name], [time created field name], [time modified field name]
+ return array(
+ array('id', 'data_records', 'timecreated', 'timemodified', 'record'),
+ array('id', 'data_comments', 'created', 'modified', 'comment')
+ );
+} //data_db_names
+
+/**
+* this function handles the access policy to contents indexed as searchable documents. If this
+* function does not exist, the search engine assumes access is allowed.
+* When this point is reached, we already know that :
+* - user is legitimate in the surrounding context
+* - user may be guest and guest access is allowed to the module
+* - the function may perform local checks within the module information logic
+* @param path the access path to the module script code
+* @param itemtype the information subclassing (usefull for complex modules, defaults to 'standard')
+* @param this_id the item id within the information class denoted by itemtype. In databases, this id
+* points out an indexed data record page.
+* @param user the user record denoting the user who searches
+* @param group_id the current group used by the user when searching
+* @return true if access is allowed, false elsewhere
+*/
+function data_check_text_access($path, $itemtype, $this_id, $user, $group_id, $context_id){
+ global $CFG;
+
+ // get the database object and all related stuff
+ if ($itemtype == 'record'){
+ $record = get_record('data_records', 'id', $this_id);
+ }
+ elseif($itemtype == 'comment'){
+ $comment = get_record('data_comments', 'id', $this_id);
+ $record = get_record('data_records', 'id', $comment->recordid);
+ }
+ else{
+ // we do not know what type of information is required
+ return false;
+ }
+ $data = get_record('data', 'id', $record->dataid);
+ $context = get_record('context', 'id', $context_id);
+ $cm = get_record('course_modules', 'id', $context->instanceid);
+ // $cm = get_coursemodule_from_instance('data', $data->id, $data->course);
+ // $context = get_context_instance(CONTEXT_MODULE, $cm->id);
+
+ if (!$cm->visible and !has_capability('moodle/course:viewhiddenactivities', $context)) {
+ if (!empty($CFG->search_access_debug)) echo "search reject : hidden database ";
+ return false;
+ }
+
+ //group consistency check : checks the following situations about groups
+ // trap if user is not same group and groups are separated
+ $current_group = get_current_group($course->id);
+ $course = get_record('course', 'id', $data->course);
+ if ((groupmode($course, $cm) == SEPARATEGROUPS) && !ismember($group_id) && !has_capability('moodle/site:accessallgroups', $context)){
+ if (!empty($CFG->search_access_debug)) echo "search reject : separated group owned resource ";
+ return false;
+ }
+
+ //ownership check : checks the following situations about user
+ // trap if user is not owner and has cannot see other's entries
+ if ($itemtype == 'record'){
+ if ($user->id != $record->userid && !has_capability('mod/data:viewentry', $context) && !has_capability('mod/data:manageentries', $context)){
+ if (!empty($CFG->search_access_debug)) echo "search reject : not owned resource ";
+ return false;
+ }
+ }
+
+ //approval check
+ // trap if unapproved and has not approval capabilities
+ // TODO : report a potential capability lack of : mod/data:approve
+ $approval = get_field('data_records', 'approved', 'id', $record->id);
+ if (!$approval && !isteacher($data->course) && !has_capability('mod/data:manageentries', $context)){
+ if (!empty($CFG->search_access_debug)) echo "search reject : unapproved resource ";
+ return false;
+ }
+
+ //minimum records to view check
+ // trap if too few records
+ // TODO : report a potential capability lack of : mod/data:viewhiddenentries
+ $recordsAmount = count_records('data_records', 'dataid', $data->id);
+ if ($data->requiredentriestoview > $recordsAmount && !isteacher($data->course) && !has_capability('mod/data:manageentries', $context)) {
+ if (!empty($CFG->search_access_debug)) echo "search reject : not enough records to view ";
+ return false;
+ }
+
+ //opening periods check
+ // trap if user has not capability to see hidden records and date is out of opening range
+ // TODO : report a potential capability lack of : mod/data:viewhiddenentries
+ $now = usertime(time());
+ if ($data->timeviewfrom > 0)
+ if ($now < $data->timeviewfrom && !isteacher($data->course) && !has_capability('mod/data:manageentries', $context)) {
+ if (!empty($CFG->search_access_debug)) echo "search reject : still not open activity ";
+ return false;
+ }
+ if ($data->timeviewto > 0)
+ if ($now > $data->timeviewto && !isteacher($data->course) && !has_capability('mod/data:manageentries', $context)) {
+ if (!empty($CFG->search_access_debug)) echo "search reject : closed activity ";
+ return false;
+ }
+
+ return true;
+} // data_check_text_access
+?>
View
72 search/documents/document.php
@@ -1,24 +1,58 @@
<?php
- /* Base search document from which other module/block types can
- * extend.
- * */
+/**
+* Global Search Engine for Moodle
+* Michael Champanis (mchampan) [cynnical@gmail.com]
+* review 1.8+ : Valery Fremaux [valery.fremaux@club-internet.fr]
+* 2007/08/02
+*
+* Base search document from which other module/block types can
+* extend.
+**/
- abstract class SearchDocument extends Zend_Search_Lucene_Document {
- public function __construct(&$doc, &$data, $document_type, $course_id, $group_id) {
- $this->addField(Zend_Search_Lucene_Field::Keyword('docid', $doc->docid));
- $this->addField(Zend_Search_Lucene_Field::Text('title', $doc->title));
- $this->addField(Zend_Search_Lucene_Field::Text('author', $doc->author));
- $this->addField(Zend_Search_Lucene_Field::UnStored('contents', $doc->contents));
- $this->addField(Zend_Search_Lucene_Field::UnIndexed('url', $doc->url));
- $this->addField(Zend_Search_Lucene_Field::UnIndexed('date', $doc->date));
-
- //additional data added on a per-module basis
- $this->addField(Zend_Search_Lucene_Field::Binary('data', serialize($data)));
-
- $this->addField(Zend_Search_Lucene_Field::Keyword('doctype', $document_type));
- $this->addField(Zend_Search_Lucene_Field::Keyword('course_id', $course_id));
- $this->addField(Zend_Search_Lucene_Field::Keyword('group_id', $group_id));
+abstract class SearchDocument extends Zend_Search_Lucene_Document {
+ public function __construct(&$doc, &$data, $course_id, $group_id, $user_id, $path) {
+ //document identification and indexing
+ $this->addField(Zend_Search_Lucene_Field::Keyword('docid', $doc->docid));
+ //document type : the name of the Moodle element that manages it
+ $this->addField(Zend_Search_Lucene_Field::Keyword('doctype', $doc->documenttype));
+ //allows subclassing information from complex modules.
+ $this->addField(Zend_Search_Lucene_Field::Keyword('itemtype', $doc->itemtype));
+ //caches the course context.
+ $this->addField(Zend_Search_Lucene_Field::Keyword('course_id', $course_id));
+ //caches the originator's group.
+ $this->addField(Zend_Search_Lucene_Field::Keyword('group_id', $group_id));
+ //caches the originator if any
+ $this->addField(Zend_Search_Lucene_Field::Keyword('user_id', $user_id));
+ // caches the context of this information. i-e, the context in which this information
+ // is being produced/attached. Speeds up the "check for access" process as context in
+ // which the information resides (a course, a module, a block, the site) is stable.
+ $this->addField(Zend_Search_Lucene_Field::UnIndexed('context_id', $doc->contextid));
+
+ //data for document
+ $this->addField(Zend_Search_Lucene_Field::Text('title', $doc->title));
+ $this->addField(Zend_Search_Lucene_Field::Text('author', $doc->author));
+ $this->addField(Zend_Search_Lucene_Field::UnStored('contents', $doc->contents));
+ $this->addField(Zend_Search_Lucene_Field::UnIndexed('url', $doc->url));
+ $this->addField(Zend_Search_Lucene_Field::UnIndexed('date', $doc->date));
+
+ //additional data added on a per-module basis
+ $this->addField(Zend_Search_Lucene_Field::Binary('data', serialize($data)));
+
+ // adding a path allows the document to know where to find specific library calls
+ // for checking access to a module or block content. The Lucene records should only
+ // be responsible to bring back to that call sufficient and consistent information
+ // in order to perform the check.
+ $this->addField(Zend_Search_Lucene_Field::UnIndexed('path', $path));
+ /*
+ // adding a capability set required for viewing. -1 if no capability required.
+ // the capability required for viewing is depending on the local situation
+ // of the document. each module should provide this information when pushing
+ // out search document structure. Although capability model should be kept flat
+ // there is no exclusion some module or block developpers use logical combinations
+ // of multiple capabilities in their code. This possibility should be left open here.
+ $this->addField(Zend_Search_Lucene_Field::UnIndexed('capabilities', $caps));
+ */
} //constructor
- } //SearchDocument
+} //SearchDocument
?>
View
368 search/documents/forum_document.php
@@ -1,135 +1,295 @@
<?php
+/**
+* Global Search Engine for Moodle
+* Michael Champanis (mchampan) [cynnical@gmail.com]
+* review 1.8+ : Valery Fremaux [valery.fremaux@club-internet.fr]
+* 2007/08/02
+*
+* document handling for forum activity module
+* This file contains the mapping between a forum post and it's indexable counterpart,
+*
+* Functions for iterating and retrieving the necessary records are now also included
+* in this file, rather than mod/forum/lib.php
+*
+* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
+* @package search
+* @version 2007110400
+**/
+/* see wiki_document.php for descriptions */
- /* see wiki_document.php for descriptions */
+require_once("$CFG->dirroot/search/documents/document.php");
+require_once("$CFG->dirroot/mod/forum/lib.php");
- require_once("$CFG->dirroot/search/documents/document.php");
- require_once("$CFG->dirroot/mod/forum/lib.php");
+/*
+* a class for representing searchable information
+*
+**/
+class ForumSearchDocument extends SearchDocument {
- class ForumSearchDocument extends SearchDocument {
- public function __construct(&$post, $forum_id, $course_id, $group_id) {
- // generic information
- $doc->docid = $post['id'];
- $doc->title = $post['subject'];
- $doc->author = $post['firstname']." ".$post['lastname'];
- $doc->contents = $post['message'];
- $doc->date = $post['created'];
+ /**
+ * constructor
+ *
+ */
+ public function __construct(&$post, $forum_id, $course_id, $itemtype, $context_id) {
+ // generic information
+ $doc->docid = $post['id'];
+ $doc->documenttype = SEARCH_TYPE_FORUM;
+ $doc->itemtype = $itemtype;
+ $doc->contextid = $context_id;
- $doc->url = forum_make_link($post['discussion'], $post['id']);
-
- // module specific information
- $data->forum = $forum_id;
- $data->discussion = $post['discussion'];
-
- parent::__construct($doc, $data, SEARCH_TYPE_FORUM, $course_id, $group_id);
+ $doc->title = $post['subject'];
+
+ $user = get_record('user', 'id', $post['userid']);
+ $doc->author = fullname($user);
+ $doc->contents = $post['message'];
+ $doc->date = $post['created'];
+ $doc->url = forum_make_link($post['discussion'], $post['id']);
+
+ // module specific information
+ $data->forum = $forum_id;
+ $data->discussion = $post['discussion'];
+
+ parent::__construct($doc, $data, $course_id, $post['groupid'], $post['userid'], PATH_FOR_SEARCH_TYPE_FORUM);
} //constructor
- } //ForumSearchDocument
+} //ForumSearchDocument
- function forum_make_link($discussion_id, $post_id) {
+/**
+* constructs a valid link to a chat content
+* @param discussion_id the discussion
+* @param post_id the id of a single post
+* @return a well formed link to forum message display
+*/
+function forum_make_link($discussion_id, $post_id) {
global $CFG;
+
return $CFG->wwwroot.'/mod/forum/discuss.php?d='.$discussion_id.'#'.$post_id;
- } //forum_make_link
-
- function forum_iterator() {
- //no @ = Undefined index: 82 in moodle/lib/datalib.php on line 2671
- return @get_all_instances_in_courses("forum", get_courses());
- } //forum_iterator
-
- function forum_get_content_for_index(&$forum) {
- $documents = array();
- if (!$forum) return $documents;
-
- $posts = forum_get_discussions_fast($forum->id);
- if (!$posts) return $documents;
+} //forum_make_link
- while (!$posts->EOF) {
- $post = $posts->fields;
+/**
+* search standard API
+*
+*/
+function forum_iterator() {
+ $forums = get_records('forum');
+ return $forums;
+} //forum_iterator
- if (is_array($post)) {
- if (strlen($post['message']) > 0 && ($post['deleted'] != 1)) {
- $documents[] = new ForumSearchDocument($post, $forum->id, $forum->course, $post['groupid']);
- } //if
+/**
+* search standard API
+* @param forum a forum instance
+* @return an array of searchable documents
+*/
+function forum_get_content_for_index(&$forum) {
- if ($children = forum_get_child_posts_fast($post['id'], $forum->id)) {
- while (!$children->EOF) {
- $child = $children->fields;
+ $documents = array();
+ if (!$forum) return $documents;
- if (strlen($child['message']) > 0 && ($child['deleted'] != 1)) {
- $documents[] = new ForumSearchDocument($child, $forum->id, $forum->course, $post['groupid']);
- } //if
+ $posts = forum_get_discussions_fast($forum->id);
+ if (!$posts) return $documents;
- $children->MoveNext();
- } //foreach
- } //if
- } //if
+ $coursemodule = get_field('modules', 'id', 'name', 'forum');
+ $cm = get_record('course_modules', 'course', $forum->course, 'module', $coursemodule, 'instance', $forum->id);
+ $context = get_context_instance(CONTEXT_MODULE, $cm->id);
- $posts->MoveNext();
- } //foreach
+ foreach($posts as $aPost) {
+ $aPost->itemtype = 'head';
+ if ($aPost) {
+ if (strlen($aPost->message) > 0) {
+ $documents[] = new ForumSearchDocument(get_object_vars($aPost), $forum->id, $forum->course, 'head', $context->id);
+ }
+ if ($children = forum_get_child_posts_fast($aPost->id, $forum->id)) {
+ foreach($children as $aChild) {
+ $aChild->itemtype = 'post';
+ if (strlen($aChild->message) > 0) {
+ $documents[] = new ForumSearchDocument(get_object_vars($aChild), $forum->id, $forum->course, 'post', $context->id);
+ }
+ }
+ }
+ }
+ }
+ return $documents;
+} //forum_get_content_for_index
- return $documents;
- } //forum_get_content_for_index
+/**
+* returns a single forum search document based on a forum entry id
+* @param id an id for a single information stub
+* @param itemtype the type of information
+*/
+function forum_single_document($id, $itemtype) {
- //returns a single forum search document based on a forum_entry id
- function forum_single_document($id) {
- $posts = get_recordset('forum_posts', 'id', $id);
- $post = $posts->fields;
-
- $discussions = get_recordset('forum_discussions', 'id', $post['discussion']);
- $discussion = $discussions->fields;
-
- $forums = get_recordset('forum', 'id', $discussion['forum']);
- $forum = $forums->fields;
-
- return new ForumSearchDocument($post, $forum['id'], $forum['course'], $post['groupid']);
- } //forum_single_document
+ // both known item types are posts so get them the same way
+ $post = get_record('forum_posts', 'id', $id);
+ $discussion = get_record('forum_discussions', 'id', $post->discussion);
+ $coursemodule = get_field('modules', 'id', 'name', 'forum');
+ $cm = get_record('course_modules', 'course', $discussion->course, 'module', $coursemodule, 'instance', $discussion->forum);
+ if ($cm){
+ $context = get_context_instance(CONTEXT_MODULE, $cm->id);
+ $post->groupid = $discussion->groupid;
+ return new ForumSearchDocument(get_object_vars($post), $discussion->forum, $discussion->course, $itemtype, $context->id);
+ }
+ return null;
+} //forum_single_document
- function forum_delete($info) {
- return $info;
- } //forum_delete
+/**
+* dummy delete function that aggregates id with itemtype.
+* this was here for a reason, but I can't remember it at the moment.
+*
+*/
+function forum_delete($info, $itemtype) {
+ $object->id = $info;
+ $object->itemtype = $itemtype;
+ return $object;
+} //forum_delete
- //returns the var names needed to build a sql query for addition/deletions
- function forum_db_names() {
+/**
+* returns the var names needed to build a sql query for addition/deletions
+*
+*/
+function forum_db_names() {
//[primary id], [table name], [time created field name], [time modified field name]
- return array('id', 'forum_posts', 'created', 'modified');
- } //forum_db_names
+ return array(
+ array('id', 'forum_posts', 'created', 'modified', 'head', 'parent = 0'),
+ array('id', 'forum_posts', 'created', 'modified', 'post', 'parent != 0')
+ );
+} //forum_db_names
- //reworked faster version from /mod/forum/lib.php
- function forum_get_discussions_fast($forum) {
+/**
+* reworked faster version from /mod/forum/lib.php
+* @param forum_id a forum identifier
+* @return an array of posts
+*/
+function forum_get_discussions_fast($forum_id) {
global $CFG, $USER;
-
+
$timelimit='';
-
if (!empty($CFG->forum_enabletimedposts)) {
- if (!((isadmin() and !empty($CFG->admineditalways)) || isteacher(get_field('forum', 'course', 'id', $forum)))) {
- $now = time();
- $timelimit = " AND ((d.timestart = 0 OR d.timestart <= '$now') AND (d.timeend = 0 OR d.timeend > '$now')";
- if (!empty($USER->id)) {
- $timelimit .= " OR d.userid = '$USER->id'";
+ if (!((isadmin() and !empty($CFG->admineditalways)) || isteacher(get_field('forum', 'course', 'id', $forum_id)))) {
+ $now = time();
+ $timelimit = " AND ((d.timestart = 0 OR d.timestart <= '$now') AND (d.timeend = 0 OR d.timeend > '$now')";
+ if (!empty($USER->id)) {
+ $timelimit .= " OR d.userid = '$USER->id'";
+ }
+ $timelimit .= ')';
}
- $timelimit .= ')';
- }
}
+
+ $query = "
+ SELECT
+ p.id,
+ p.subject,
+ p.discussion,
+ p.message,
+ p.created,
+ d.groupid,
+ p.userid,
+ u.firstname,
+ u.lastname
+ FROM
+ {$CFG->prefix}forum_discussions d
+ JOIN
+ {$CFG->prefix}forum_posts p
+ ON
+ p.discussion = d.id
+ JOIN
+ {$CFG->prefix}user u
+ ON
+ p.userid = u.id
+ WHERE
+ d.forum = '{$forum_id}' AND
+ p.parent = 0
+ $timelimit
+ ORDER BY
+ d.timemodified DESC
+ ";
+ return get_records_sql($query);
+} //forum_get_discussions_fast
- return get_recordset_sql("SELECT p.id, p.subject, p.discussion, p.message,
- p.deleted, d.groupid, u.firstname, u.lastname
- FROM {$CFG->prefix}forum_discussions d
- JOIN {$CFG->prefix}forum_posts p ON p.discussion = d.id
- JOIN {$CFG->prefix}user u ON p.userid = u.id
- WHERE d.forum = '$forum'
- AND p.parent = 0
- $timelimit
- ORDER BY d.timemodified DESC");
- } //forum_get_discussions_fast
-
- //reworked faster version from /mod/forum/lib.php
- function forum_get_child_posts_fast($parent, $forumid) {
+/**
+* reworked faster version from /mod/forum/lib.php
+* @param parent the id of the first post within the discussion
+* @param forum_id the forum identifier
+* @return an array of posts
+*/
+function forum_get_child_posts_fast($parent, $forum_id) {
global $CFG;
+
+ $query = "
+ SELECT
+ p.id,
+ p.subject,
+ p.discussion,
+ p.message,
+ p.created,
+ {$forum_id} AS forum,
+ p.userid,
+ d.groupid,
+ u.firstname,
+ u.lastname
+ FROM
+ {$CFG->prefix}forum_discussions d
+ JOIN
+ {$CFG->prefix}forum_posts p
+ ON
+ p.discussion = d.id
+ JOIN
+ {$CFG->prefix}user u
+ ON
+ p.userid = u.id
+ WHERE
+ p.parent = '{$parent}'
+ ORDER BY
+ p.created ASC
+ ";
+ return get_records_sql($query);
+} //forum_get_child_posts_fast
+
+/**
+* this function handles the access policy to contents indexed as searchable documents. If this
+* function does not exist, the search engine assumes access is allowed.
+* When this point is reached, we already know that :
+* - user is legitimate in the surrounding context
+* - user may be guest and guest access is allowed to the module
+* - the function may perform local checks within the module information logic
+* @param path the access path to the module script code
+* @param itemtype the information subclassing (usefull for complex modules, defaults to 'standard')
+* @param this_id the item id within the information class denoted by itemtype. In forums, this id
+* points out the individual post.
+* @param user the user record denoting the user who searches
+* @param group_id the current group used by the user when searching
+* @return true if access is allowed, false elsewhere
+*/
+function forum_check_text_access($path, $itemtype, $this_id, $user, $group_id, $context_id){
+ global $CFG, $USER;
+
+ include_once("{$CFG->dirroot}/{$path}/lib.php");
- return get_recordset_sql("SELECT p.id, p.subject, p.discussion, p.message, p.deleted,
- $forumid AS forum, u.firstname, u.lastname
- FROM {$CFG->prefix}forum_posts p
- LEFT JOIN {$CFG->prefix}user u ON p.userid = u.id
- WHERE p.parent = '$parent'
- ORDER BY p.created ASC");
- } //forum_get_child_posts_fast
+ // get the forum post and all related stuff
+ $post = get_record('forum_posts', 'id', $this_id);
+ $discussion = get_record('forum_discussions', 'id', $post->discussion);
+ $context = get_record('context', 'id', $context_id);
+ $cm = get_record('course_modules', 'id', $context->instanceid);
+ // $cm = get_coursemodule_from_instance('forum', $discussion->forum, $discussion->course);
+ // $context = get_context_instance(CONTEXT_MODULE, $cm->id);
+ if (!$cm->visible and !has_capability('moodle/course:viewhiddenactivities', $context)){
+ if (!empty($CFG->search_access_debug)) echo "search reject : hidden forum resource ";
+ return false;
+ }
+
+ // approval check : entries should be approved for being viewed, or belongs to the user
+ if (($post->userid != $USER->id) && !$post->mailed && !has_capability('mod/forum:viewhiddentimeposts', $context)){
+ if (!empty($CFG->search_access_debug)) echo "search reject : time hidden forum item";
+ return false;
+ }
+
+ // group check : entries should be in accessible groups
+ $current_group = get_current_group($discussion->course);
+ $course = get_record('course', 'id', $discussion->course);
+ if ($group_id >= 0 && (groupmode($course, $cm) == SEPARATEGROUPS) && ($group_id != $current_group) && !has_capability('mod/forum:viewdiscussionsfromallgroups', $context)){
+ if (!empty($CFG->search_access_debug)) echo "search reject : separated grouped forum item";
+ return false;
+ }
+
+ return true;
+} //forum_check_text_access
?>
View
308 search/documents/glossary_document.php
@@ -1,88 +1,246 @@
<?php
- /* This document illustrates how easy it is to add a module to
- * the search index - the only modifications made were creating
- * this file, and adding the SEARCH_TYPE_GLOSSARY constant to
- * search/lib.php - everything else is automatically handled
- * by the indexer script.
- * */
-
- require_once("$CFG->dirroot/search/documents/document.php");
-
- class GlossarySearchDocument extends SearchDocument {
- public function __construct(&$entry, $glossary_id, $course_id, $group_id) {
- // generic information; required
- $doc->docid = $entry['id'];
- $doc->title = $entry['concept'];
- $doc->date = $entry['timecreated'];
-
- $user = get_recordset('user', 'id', $entry['userid'])->fields;
-
- $doc->author = $user['firstname'].' '.$user['lastname'];
- $doc->contents = $entry['definition'];
- $doc->url = glossary_make_link($entry['id']);
-
- // module specific information; optional
- $data->glossary = $glossary_id;
-
- // construct the parent class
- parent::__construct($doc, $data, SEARCH_TYPE_GLOSSARY, $course_id, $group_id);
+/**
+* Global Search Engine for Moodle
+* Michael Champanis (mchampan) [cynnical@gmail.com]
+* review 1.8+ : Valery Fremaux [valery.fremaux@club-internet.fr]
+* 2007/08/02
+*
+* document handling for glossary activity module
+* This file contains a mapping between a glossary entry and it's indexable counterpart,
+*
+* Functions for iterating and retrieving the necessary records are now also included
+* in this file, rather than mod/glossary/lib.php
+*
+* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
+* @package search
+* @version 2007110400
+**/
+
+require_once("$CFG->dirroot/search/documents/document.php");
+
+/*
+* a class for representing searchable information
+*
+**/
+class GlossarySearchDocument extends SearchDocument {
+
+ /**
+ * document constructor
+ *
+ */
+ public function __construct(&$entry, $course_id, $context_id) {
+ // generic information; required
+ $doc->docid = $entry['id'];
+ $doc->documenttype = SEARCH_TYPE_GLOSSARY;
+ $doc->itemtype = 'standard';
+ $doc->contextid = $context_id;
+
+ $doc->title = $entry['concept'];
+ $doc->date = $entry['timecreated'];
+
+ if ($entry['userid'])
+ $user = get_record('user', 'id', $entry['userid']);
+ $doc->author = ($user ) ? $user->firstname.' '.$user->lastname : '' ;
+ $doc->contents = strip_tags($entry['definition']);
+ $doc->url = glossary_make_link($entry['id']);
+
+ // module specific information; optional
+ $data->glossary = $entry['glossaryid'];
+
+ // construct the parent class
+ parent::__construct($doc, $data, $course_id, -1, $entry['userid'], PATH_FOR_SEARCH_TYPE_GLOSSARY);
} //constructor
- } //GlossarySearchDocument
-
- function glossary_make_link($entry_id) {
+} //GlossarySearchDocument
+
+/*
+* a class for representing searchable information
+*
+**/
+class GlossaryCommentSearchDocument extends SearchDocument {
+
+ /**