Skip to content

Commit 218d7fa

Browse files
David Bengoakaplun
authored andcommitted
BibCheck: new BibCheck module
* Creates a new BibCheck module in python with configurable plugins * Adds a example set of rules in rules.cfg * Creates doi, dates, enum, files, texkey, regexp, mandatory, utf-8, url, trailing_space, regexp_replace, rename_tag and rename_subfield plugins * Creates unit tests for BibCheck and the plugins * Removes old BibCheck web interface and documentation * Adds API so other modules can call bibcheck Tested-by: Javier Martin Montull <javier.martin.montull@cern.ch> Reviewed-by: Javier Martin Montull <javier.martin.montull@cern.ch>
1 parent 5138bed commit 218d7fa

40 files changed

+2157
-393
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,7 @@ modules/websubmit/bin/websubmitadmin
9292
modules/bibcirculation/bin/bibcircd
9393
modules/bibcatalog/bin/bibcatalog
9494
modules/pdfchecker/bin/arxiv-pdf-checker
95+
modules/bibcheck/bin/bibcheck
9596
tags
9697
config.status.lineno
9798
configure.lineno

config/invenio.conf

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2315,6 +2315,9 @@ CFG_CROSSREF_USERNAME =
23152315
## to the Crossref site.
23162316
CFG_CROSSREF_PASSWORD =
23172317

2318+
## CFG_CROSSREF_EMAIL -- crossref query services email
2319+
CFG_CROSSREF_EMAIL =
2320+
23182321
#####################################
23192322
## Part 31: WebLinkback parameters ##
23202323
#####################################

configure.ac

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -588,11 +588,12 @@ AC_CONFIG_FILES([config.nice \
588588
modules/bibcatalog/lib/ticket_templates/Makefile \
589589
modules/bibcheck/Makefile \
590590
modules/bibcheck/doc/Makefile \
591-
modules/bibcheck/doc/admin/Makefile \
592591
modules/bibcheck/doc/hacking/Makefile \
593592
modules/bibcheck/etc/Makefile \
594-
modules/bibcheck/web/Makefile \
595-
modules/bibcheck/web/admin/Makefile \
593+
modules/bibcheck/bin/Makefile \
594+
modules/bibcheck/lib/Makefile \
595+
modules/bibcheck/lib/plugins/Makefile \
596+
modules/bibcheck/bin/bibcheck \
596597
modules/bibcirculation/Makefile \
597598
modules/bibcirculation/bin/Makefile \
598599
modules/bibcirculation/bin/bibcircd \

modules/bibcheck/Makefile.am

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
## This file is part of Invenio.
2-
## Copyright (C) 2009, 2010, 2011 CERN.
2+
## Copyright (C) 2013 CERN.
33
##
44
## Invenio is free software; you can redistribute it and/or
55
## modify it under the terms of the GNU General Public License as
@@ -15,6 +15,6 @@
1515
## along with Invenio; if not, write to the Free Software Foundation, Inc.,
1616
## 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
1717

18-
SUBDIRS = doc etc web
18+
SUBDIRS = doc etc lib bin
1919

2020
CLEANFILES = *~
Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
## This file is part of Invenio.
2-
## Copyright (C) 2009, 2010, 2011 CERN.
2+
## Copyright (C) 2004, 2005, 2006, 2007, 2008, 2010, 2011 CERN.
33
##
44
## Invenio is free software; you can redistribute it and/or
55
## modify it under the terms of the GNU General Public License as
@@ -15,10 +15,7 @@
1515
## along with Invenio; if not, write to the Free Software Foundation, Inc.,
1616
## 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
1717

18-
webdoclibdir = $(libdir)/webdoc/invenio/admin
19-
20-
webdoclib_DATA = bibcheck-admin-guide.webdoc
21-
22-
EXTRA_DIST = $(webdoclib_DATA)
18+
bin_SCRIPTS = bibcheck
19+
EXTRA_DIST = bibcheck.in
2320

2421
CLEANFILES = *~ *.tmp

modules/bibcheck/bin/bibcheck.in

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#!@PYTHON@
2+
## -*- mode: python; coding: utf-8; -*-
3+
##
4+
## This file is part of Invenio.
5+
## Copyright (C) 2013 CERN.
6+
##
7+
## Invenio is free software; you can redistribute it and/or
8+
## modify it under the terms of the GNU General Public License as
9+
## published by the Free Software Foundation; either version 2 of the
10+
## License, or (at your option) any later version.
11+
##
12+
## Invenio is distributed in the hope that it will be useful, but
13+
## WITHOUT ANY WARRANTY; without even the implied warranty of
14+
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
15+
## General Public License for more details.
16+
##
17+
## You should have received a copy of the GNU General Public License
18+
## along with Invenio; if not, write to the Free Software Foundation, Inc.,
19+
## 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
20+
21+
"""
22+
"bibcheck" is used to check a set of records against a
23+
configurable set of rules. A rule consists of a query, a
24+
checker and an amender. The set of records that
25+
match the query are checked with the checker and the records
26+
that don't pass the test will be passed to the amender. An
27+
amender can try to fix the record automatically or request a
28+
human to fix the record.
29+
30+
The checkers and amenders are loaded via a plug-in system, so
31+
it's easy to add new checkers or amenders.
32+
"""
33+
34+
from invenio.bibcheck_task import main as cli_main
35+
36+
if __name__ == '__main__':
37+
try:
38+
cli_main()
39+
except KeyboardInterrupt:
40+
# Exit cleanly
41+
print 'Interrupted'

modules/bibcheck/doc/Makefile.am

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,4 +15,4 @@
1515
## along with Invenio; if not, write to the Free Software Foundation, Inc.,
1616
## 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
1717

18-
SUBDIRS = admin hacking
18+
SUBDIRS = hacking

modules/bibcheck/doc/admin/bibcheck-admin-guide.webdoc

Lines changed: 0 additions & 38 deletions
This file was deleted.

modules/bibcheck/etc/Makefile.am

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717

1818
etcdir = $(sysconfdir)/bibcheck
1919

20-
etc_DATA = sample.cfg
20+
etc_DATA = rules.cfg
2121

2222
EXTRA_DIST = $(etc_DATA)
2323

modules/bibcheck/etc/rules.cfg

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
2+
[check_mandatory_fields]
3+
check=mandatory
4+
check.fields = ["001%%_"]
5+
6+
[check_utf8]
7+
check=utf8
8+
9+
[trailing_space]
10+
check=trailing_space
11+
check.strip = true
12+
check.normalize_spaces = true
13+
check.fields = ["100%%a"]
14+
15+
[check_doi]
16+
check=doi
17+
18+
[check_xref]
19+
check=crossref_checker
20+

0 commit comments

Comments
 (0)