Skip to content

Commit

Permalink
really proud of this commit :-)
Browse files Browse the repository at this point in the history 
z3950 search and import seems to works fine.
Let me explain how :
* a "search z3950" button is added in the addbiblio template.
* when clicked, a popup appears and z3950/search.pl is called
* z3950/search.pl calls addz3950search in the DB
* the z3950 daemon retrieve the records and stores them in z3950results AND in marc_breeding table.
* as long as there as searches pending, the popup auto refresh every 2 seconds, and says how many searches are pending.
* when the user clicks on a z3950 result => the parent popup is called with the requested biblio, and auto-filled

Note :
* character encoding support : (It's a nightmare...) In the z3950servers table, a "encoding" column has been added. You can put "UNIMARC" or "USMARC" in this column. Depending on this, the char_decode in C4::Biblio.pm replaces marc-char-encode by an iso 8859-1 encoding. Note that in the breeding import this value has been added too, for a better support.
* the marc_breeding and z3950* tables have been modified : they have an encoding column and the random z3950 number is stored too for convenience => it's the key I use to list only requested biblios in the popup.
  • Loading branch information
tipaul committed Apr 29, 2003
1 parent a3c119a commit 26543b4
Show file tree
Hide file tree
Showing 16 changed files with 420 additions and 81 deletions.
28 changes: 23 additions & 5 deletions C4/Biblio.pm
View file
@@ -1,6 +1,21 @@
package C4::Biblio;
# $Id$
# $Log$
# Revision 1.45 2003/04/29 16:50:49 tipaul
# really proud of this commit :-)
# z3950 search and import seems to works fine.
# Let me explain how :
# * a "search z3950" button is added in the addbiblio template.
# * when clicked, a popup appears and z3950/search.pl is called
# * z3950/search.pl calls addz3950search in the DB
# * the z3950 daemon retrieve the records and stores them in z3950results AND in marc_breeding table.
# * as long as there as searches pending, the popup auto refresh every 2 seconds, and says how many searches are pending.
# * when the user clicks on a z3950 result => the parent popup is called with the requested biblio, and auto-filled
#
# Note :
# * character encoding support : (It's a nightmare...) In the z3950servers table, a "encoding" column has been added. You can put "UNIMARC" or "USMARC" in this column. Depending on this, the char_decode in C4::Biblio.pm replaces marc-char-encode by an iso 8859-1 encoding. Note that in the breeding import this value has been added too, for a better support.
# * the marc_breeding and z3950* tables have been modified : they have an encoding column and the random z3950 number is stored too for convenience => it's the key I use to list only requested biblios in the popup.
#
# Revision 1.44 2003/04/28 13:07:14 tipaul
# Those fixes solves the "internal server error" with MARC::Record 1.12.
# It was due to an illegal contruction in Koha : we tried to retrive subfields from <10 tags.
Expand Down Expand Up @@ -2137,9 +2152,10 @@ sub getoraddbiblio {
sub char_decode {
# converts ISO 5426 coded string to ISO 8859-1
# sloppy code : should be improved in next issue
my ($string) = @_ ;
my ($string,$encoding) = @_ ;
$_ = $string ;
if (C4::Context->preference("marcflavour") eq "UNIMARC") {
# $encoding = C4::Context->preference("marcflavour") unless $encoding;
if ($encoding eq "UNIMARC") {
s/\xe1/Æ/gm ;
s/\xe2/Ð/gm ;
s/\xe9/Ø/gm ;
Expand Down Expand Up @@ -2201,7 +2217,9 @@ sub char_decode {
s/\xca\x61/å/gm ;
s/\xd0\x43/Ç/gm ;
s/\xd0\x63/ç/gm ;
} else {
# this handles non-sorting blocks (if implementation requires this)
$string = nsb_clean($_) ;
} elsif ($encoding eq "USMARC") {
if(/[\xc1-\xff]/) {
s/\xe1\x61/à/gm ;
s/\xe1\x65/è/gm ;
Expand Down Expand Up @@ -2254,10 +2272,10 @@ sub char_decode {
s/\xe9\x75/ü/gm ;
s/\xea\x41/Å/gm ;
s/\xea\x61/å/gm ;
# this handles non-sorting blocks (if implementation requires this)
$string = nsb_clean($_) ;
}
}
# this handles non-sorting blocks (if implementation requires this)
$string = nsb_clean($_) ;
return($string) ;
}

Expand Down
43 changes: 28 additions & 15 deletions C4/Breeding.pm
View file
Expand Up @@ -33,8 +33,17 @@ C4::Breeding : script to add a biblio in marc_breeding table.
=head1 SYNOPSIS
use C4::Scan;
&ImportBreeding($marcrecords,$);
use C4::Scan;
&ImportBreeding($marcrecords,$overwrite_biblio,$filename,$z3950random);
C<$marcrecord> => the MARC::Record
C<$overwrite_biblio> => if set to 1 a biblio with the same ISBN will be overwritted.
if set to 0 a biblio with the same isbn will be ignored (the previous will be kept)
if set to -1 the biblio will be added anyway (more than 1 biblio with the same ISBN possible in the breeding
C<$encoding> => USMARC
or UNIMARC. used for char_decoding.
If not present, the parameter marcflavour is used instead
C<$z3950random> => the random value created during a z3950 search result.
=head1 DESCRIPTION
Expand All @@ -46,14 +55,15 @@ This module doesn't do anything.
@EXPORT = qw(&ImportBreeding);

sub ImportBreeding {
my ($marcrecords,$overwrite_biblio,$filename) = @_;
my ($marcrecords,$overwrite_biblio,$filename,$encoding,$z3950random) = @_;
my @marcarray = split /\x1D/, $marcrecords;
my $dbh = C4::Context->dbh;
my $searchisbn = $dbh->prepare("select biblioitemnumber from biblioitems where isbn=?");
my $searchissn = $dbh->prepare("select biblioitemnumber from biblioitems where issn=?");
my $searchbreeding = $dbh->prepare("select id from marc_breeding where isbn=?");
my $insertsql = $dbh->prepare("insert into marc_breeding (file,isbn,title,author,marc) values(?,?,?,?,?)");
my $replacesql = $dbh->prepare("update marc_breeding set file=?,isbn=?,title=?,author=?,marc=? where id=?");
my $insertsql = $dbh->prepare("insert into marc_breeding (file,isbn,title,author,marc,encoding,z3950random) values(?,?,?,?,?,?,?)");
my $replacesql = $dbh->prepare("update marc_breeding set file=?,isbn=?,title=?,author=?,marc=?,encoding=?,z3950random=? where id=?");
$encoding = C4::Context->preference("marcflavour") unless $encoding;
# fields used for import results
my $imported=0;
my $alreadyindb = 0;
Expand All @@ -65,11 +75,14 @@ sub ImportBreeding {
$notmarcrecord++;
} else {
my $oldbiblio = MARCmarc2koha($dbh,$marcrecord);
$oldbiblio->{title} = char_decode($oldbiblio->{title});
$oldbiblio->{author} = char_decode($oldbiblio->{author});
$oldbiblio->{title} = char_decode($oldbiblio->{title},$encoding);
$oldbiblio->{author} = char_decode($oldbiblio->{author},$encoding);
# if isbn found and biblio does not exist, add it. If isbn found and biblio exists, overwrite or ignore depending on user choice
# drop every "special" char : spaces, - ...
$oldbiblio->{isbn} =~ s/ |-|\.//g,
$oldbiblio->{isbn} = substr($oldbiblio->{isbn},0,10);
$oldbiblio->{issn} =~ s/ |-|\.//g,
$oldbiblio->{issn} = substr($oldbiblio->{issn},0,10);
# search if biblio exists
my $biblioitemnumber;
if ($oldbiblio->{isbn}) {
Expand All @@ -91,17 +104,17 @@ sub ImportBreeding {
$searchbreeding->execute($oldbiblio->{issn});
($breedingid) = $searchbreeding->fetchrow;
}
if (!$breedingid || $overwrite_biblio) {
if ($breedingid && $overwrite_biblio eq 0) {
$alreadyinfarm++;
} else {
my $recoded;
$recoded = $marcrecord->as_usmarc();
if ($breedingid) {
$replacesql ->execute($filename,substr($oldbiblio->{isbn}.$oldbiblio->{issn},0,10),$oldbiblio->{title},$oldbiblio->{author},$recoded,$breedingid);
} else {
$insertsql ->execute($filename,substr($oldbiblio->{isbn}.$oldbiblio->{issn},0,10),$oldbiblio->{title},$oldbiblio->{author},$recoded);
}
if ($breedingid && $overwrite_biblio eq 1) {
$replacesql ->execute($filename,substr($oldbiblio->{isbn}.$oldbiblio->{issn},0,10),$oldbiblio->{title},$oldbiblio->{author},$recoded,$encoding,$z3950random,$breedingid);
} else {
$insertsql ->execute($filename,substr($oldbiblio->{isbn}.$oldbiblio->{issn},0,10),$oldbiblio->{title},$oldbiblio->{author},$recoded,$encoding,$z3950random);
}
$imported++;
} else {
$alreadyinfarm++;
}
}
}
Expand Down
30 changes: 18 additions & 12 deletions C4/Search.pm
View file
Expand Up @@ -2380,31 +2380,37 @@ and itemtype = 'WEB'";

=item breedingsearch
($count, @results) = &breedingsearch($title);
($count, @results) = &breedingsearch($title,$isbn,$random);
C<$title> contains the title,
C<$isbn> contains isbn or issn,
C<$random> contains the random seed from a z3950 search.
C<$count> is the number of items in C<@results>. C<@results> is an
array of references-to-hash; the keys are the items from the
C<marc_breeding> table of the Koha database.
array of references-to-hash; the keys are the items from the C<marc_breeding> table of the Koha database.
=cut

sub breedingsearch {
my ($title,$isbn) = @_;
my ($title,$isbn,$z3950random) = @_;
my $dbh = C4::Context->dbh;
my $count = 0;
my $query;
my $sth;
my @results;

$query = "Select id,file,isbn,title,author from marc_breeding where ";
if ($title) {
$query .= "title like \"$title%\"";
}
if ($title && $isbn) {
$query .= " and ";
}
if ($isbn) {
$query .= "isbn like \"$isbn%\"";
if ($z3950random) {
$query .= "z3950random = \"$z3950random\"";
} else {
if ($title) {
$query .= "title like \"$title%\"";
}
if ($title && $isbn) {
$query .= " and ";
}
if ($isbn) {
$query .= "isbn like \"$isbn%\"";
}
}
$sth = $dbh->prepare($query);
$sth->execute;
Expand Down
56 changes: 49 additions & 7 deletions C4/Z3950.pm
View file
Expand Up @@ -69,6 +69,7 @@ entering Z39.50 lookup requests.
&getz3950servers
&z3950servername
&addz3950queue
&checkz3950searchdone
);

#------------------------------------------------
Expand Down Expand Up @@ -198,23 +199,23 @@ sub addz3950queue {
if ($server =~ /:/ ) {
push @serverlist, $server;
} elsif ($server eq 'DEFAULT' || $server eq 'CHECKED' ) {
$sth=$dbh->prepare("select host,port,db,userid,password ,name from z3950servers where checked <> 0 ");
$sth=$dbh->prepare("select host,port,db,userid,password ,name,syntax from z3950servers where checked <> 0 ");
$sth->execute;
while ( my ($host, $port, $db, $userid, $password,$servername) = $sth->fetchrow ) {
push @serverlist, "$servername/$host\:$port/$db/$userid/$password";
while ( my ($host, $port, $db, $userid, $password,$servername,$syntax) = $sth->fetchrow ) {
push @serverlist, "$servername/$host\:$port/$db/$userid/$password/$syntax";
} # while
} else {
$sth=$dbh->prepare("select host,port,db,userid,password from z3950servers where id=? ");
$sth=$dbh->prepare("select host,port,db,userid,password,syntax from z3950servers where id=? ");
$sth->execute($server);
my ($host, $port, $db, $userid, $password) = $sth->fetchrow;
push @serverlist, "$server/$host\:$port/$db/$userid/$password";
my ($host, $port, $db, $userid, $password,$syntax) = $sth->fetchrow;
push @serverlist, "$server/$host\:$port/$db/$userid/$password/$syntax";
}
}

my $serverlist='';

$serverlist = join(" ", @serverlist);
chop $serverlist;
# chop $serverlist;

# FIXME - Is this test supposed to test whether @serverlist is
# empty? If so, then a) there are better ways to do that in
Expand Down Expand Up @@ -266,6 +267,32 @@ sub addz3950queue {

} # sub addz3950queue

=item &checkz3950searchdone
$numberpending= & &checkz3950searchdone($random);
Returns the number of pending z3950 requests
C<$random> is the random z3950 query number.
=cut
sub checkz3950searchdone {
my ($z3950random) = @_;
my $dbh = C4::Context->dbh;
# first, check that the deamon already created the requests...
my $sth = $dbh->prepare("select count(*) from z3950queue,z3950results where z3950queue.id = z3950results.queryid and z3950queue.identifier=?");
$sth->execute($z3950random);
my ($result) = $sth->fetchrow;
if ($result eq 0) { # search not yet begun => should be searches to do !
return "??";
}
# second, count pending requests
$sth = $dbh->prepare("select count(*) from z3950queue,z3950results where z3950queue.id = z3950results.queryid and z3950results.enddate is null and z3950queue.identifier=?");
$sth->execute($z3950random);
($result) = $sth->fetchrow;
return $result;
}

1;
__END__
Expand All @@ -279,6 +306,21 @@ Koha Developement team <info@koha.org>
#--------------------------------------
# $Log$
# Revision 1.9 2003/04/29 16:50:51 tipaul
# really proud of this commit :-)
# z3950 search and import seems to works fine.
# Let me explain how :
# * a "search z3950" button is added in the addbiblio template.
# * when clicked, a popup appears and z3950/search.pl is called
# * z3950/search.pl calls addz3950search in the DB
# * the z3950 daemon retrieve the records and stores them in z3950results AND in marc_breeding table.
# * as long as there as searches pending, the popup auto refresh every 2 seconds, and says how many searches are pending.
# * when the user clicks on a z3950 result => the parent popup is called with the requested biblio, and auto-filled
#
# Note :
# * character encoding support : (It's a nightmare...) In the z3950servers table, a "encoding" column has been added. You can put "UNIMARC" or "USMARC" in this column. Depending on this, the char_decode in C4::Biblio.pm replaces marc-char-encode by an iso 8859-1 encoding. Note that in the breeding import this value has been added too, for a better support.
# * the marc_breeding and z3950* tables have been modified : they have an encoding column and the random z3950 number is stored too for convenience => it's the key I use to list only requested biblios in the popup.
#
# Revision 1.8 2003/04/29 08:09:45 tipaul
# z3950 support is coming...
# * adding a syntax column in z3950 table = this column will say wether the z3950 must be called with PerferedRecordsyntax => USMARC or PerferedRecordsyntax => UNIMARC. I tried some french UNIMARC z3950 servers, and some only send USMARC, some only UNIMARC, some can answer with both.
Expand Down

0 comments on commit 26543b4

Please sign in to comment.