Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
replication skips HTTPFile->digest if device can reject bad MD5s
Rereading a large file is expensive.  If we can monitor
and observe our storage nodes for MD5 rejectionability, we
can rely on that instead of having to have anybody reread
the entire file to calculate its MD5.
  • Loading branch information
Eric Wong authored and dormando committed Mar 30, 2012
1 parent bd624ed commit 55b29e5
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions lib/MogileFS/Worker/Replicate.pm
Expand Up @@ -705,6 +705,12 @@ sub http_copy {
if ($1 >= 200 && $1 <= 299) {
if ($digest) {
my $alg = $rfid->class->checksumname;

if ($ddev->{reject_bad_md5} && ($alg eq "MD5")) {
# dest device would've rejected us with a error,
# no need to reread the file
return 1;
}
my $durl = "http://$dhostip:$dport$dpath";
my $actual = MogileFS::HTTPFile->at($durl)->digest($alg);
if ($actual ne $digest) {
Expand Down

0 comments on commit 55b29e5

Please sign in to comment.