Permalink
Browse files

Improved README files for loading wikis into the db, and for testing.

  • Loading branch information...
1 parent 864ec3e commit 9ee3056e11c1c3901dba3d60af315481fd7681f5 Luca de Alfaro committed Aug 23, 2008
View
@@ -13,6 +13,7 @@ If you switch the extension off, then on, the extension will compute
the missing reputation and trust values in a self-healing,
fault-tolerant fashion.
+
PREREQUISITES:
- Install mysql and mediawiki
@@ -37,6 +38,7 @@ PREREQUISITES:
- Make OcamlLdaLibs and install them (see instructions there).
+
INSTALLATION:
1) First, compile the code:
@@ -123,12 +125,32 @@ safely disabled. You can disable these logs via:
$wgTrustLog = "/dev/null";
$wgTrustDebugLog = "/dev/null";
+
DEPLOYMENT:
You should now see a trust tab. Click on it, or edit a revision, and
the trust of all revision text, and the reputation of all authors,
will be computed.
+
+COMPUTING THE TRUST COLORING:
+
+The trust coloring is computed automatically whenever a user edits an
+article, or requests to view the trust informaton for a revision. In
+particular, when a user edits an article, or when someone asks to view
+the trust information, WikiTrust computes the trust coloring not only
+for the newest revision, but for all past revisions of all wiki
+articles that have not been colored yet.
+
+You can also compute the trust coloring from the command line, via the
+command:
+
+ $ ./eval_online_wiki -db_user <username> -db_pass <pwd> -db_name <db_name>
+
+eval_online_wiki computes trust for all revisions for which the trust
+information has not been computed yet.
+
+
RECOMPUTING THE TRUST COLORING:
It is possible at any time to recompute from scratch the trust
View
@@ -1,28 +1,28 @@
-Test harness for on-line code.
+This directory contains code that can be used to load an xml database
+dump into a wiki.
Requirements : mediawiki, mysql, with a mediawiki installation all set up.
-To work, first copy pull_revision.ini.sample into online_feed_test and
-update the values defined there. Point to the db created by mediawiki.
+To work, first copy db_access_data.ini.sample into db_access_data.ini and
+update the values defined there with the information about the
+database created by mediawiki.
-Then, run the sql file ../sql/start_online_analysis.sql, creating the needed tables inside the same db as
-mediawiki:
+To populate the database, use the command:
-$ cat start_online_analysis.sql | mysql --user="root" --password="<mysql-root-password>" wikidb
+ $ ./load_data.py <dump_1.xml> ... <dump_n.xml>
-Now, you are ready to go!
+If you wish to delete the wiki content before loading the files
+(CAREFUL WITH THIS!!), use the --clear_db option:
-Run with:
+ $ ./load_data.py --clear_db <dump_1.xml> ... <dump_n.xml>
-python ./set_test.py
-This will call the online_db code with each revision in the dump file.
-
-For example:
-
- ./set_test.py -v --use_dump /home/ipye/enwiki-20080103-0020009.xml.gz --eval_wiki \
- ../analysis/eval_online_wiki
-
-Note that the other files in this directory are experimental, and are not used right now.
+The file load_and_run.sh.template can be used to test the WikiTrust code.
+To do so, copy it into load_and_run.sh, and run it.
+Note that the script runs eval_online_wiki in debug mode, via
+ocamldebug, so the script is not useful to only for code debugging.
+If you wish to first load a wiki, then color it, you should use
+load_data.py to load the wiki, and eval_online_wiki to color it as
+explained in the README file in the parent directory.
@@ -3,6 +3,6 @@
cat ../sql/remove_wikitrust_tables.sql | mysql --user="root" --password="root_password" wikidb
cat ../sql/create_wikitrust_tables.sql | mysql --user="root" --password="root_password" wikidb
cd ../wikifeed
-./set_test.py ../test-data/wiki_00023.xml
+./load_data.py ../test-data/wiki_00023.xml
cd ../analysis
-ocamldebug -I `ocamlfind query unix` -I `ocamlfind query str` -I `ocamlfind query vec` -I `ocamlfind query mapmin` -I `ocamlfind query hashtbl_bounded` -I `ocamlfind query fileinfo` -I `ocamlfind query intvmap` -I `ocamlfind query extlib` -I `ocamlfind query mysql` -I `ocamlfind query sexplib` -I ../../batch/analysis ./eval_online_wiki -db_user wikiuser -db_pass localwiki -db_name wikidb -log_name ~/temp/color.log -delete_all
+ocamldebug -I `ocamlfind query unix` -I `ocamlfind query str` -I `ocamlfind query vec` -I `ocamlfind query mapmin` -I `ocamlfind query hashtbl_bounded` -I `ocamlfind query fileinfo` -I `ocamlfind query intvmap` -I `ocamlfind query extlib` -I `ocamlfind query mysql` -I `ocamlfind query sexplib` -I ../../batch/analysis ./eval_online_wiki -db_user <username> -db_pass <password> -db_name <dbname> -log_name ~/temp/color.log -delete_all
@@ -49,13 +49,13 @@
## Usage method
def usage():
- print "Usage: python set_test.py [-h, --help] dump1 dump2 ... dumpn "
+ print "Usage: python load_data.py [-h, --help, --clear_db] dump1 dump2 ... dumpn "
## A list of files to load
dumps = []
-def load_dump(dump):
+def load_dump(dump):
global ini_config
@@ -67,19 +67,21 @@ def load_dump(dump):
return
try:
- opts, args = getopt.gnu_getopt(sys.argv[1:], "h", ["help"])
+ opts, args = getopt.gnu_getopt(sys.argv[1:], "h", ["help", "clear_db"])
except getopt.GetoptError:
# print help information and exit:
usage()
sys.exit(2)
+do_clear = False
for a in args:
dumps.append(a)
for o, a in opts:
if o in ("-h", "--help"):
usage()
sys.exit(2)
-
+ if o in ("--clear_db"):
+ do_clear = True
## parse the ini file
ini_config = ConfigParser.ConfigParser()
ini_config.readfp(open(INI_FILE))
@@ -90,11 +92,12 @@ def load_dump(dump):
, db=ini_config.get('db', 'db') )
curs = connection.cursor()
-# clear out the pull db
-curs.execute("delete from text")
-curs.execute("delete from page")
-curs.execute("delete from revision")
-connection.commit()
+# clear out the pull db if requested
+if do_clear:
+ curs.execute("delete from text")
+ curs.execute("delete from page")
+ curs.execute("delete from revision")
+ connection.commit()
for dump in dumps:
load_dump(dump)

0 comments on commit 9ee3056

Please sign in to comment.