open checksum file in utf8 mode and strip BOM

[mwarin]

Fix for issue #156, triggered by https://hathitrust.atlassian.net/issues/GS-11436.

[aelkiss]

I don't love having to do it, but I think the fix is fine, and it's good to be generous in what we accept. It might be good to ensure that the use open ... is lexically scoped to this block and won't inadvertently break stuff elsewhere?

[mwarin]

I think this proves the lexicalness of its scope:

use strict;                              
use warnings;                            
                                         
no_pragma();                             
use_pragma();                            
no_pragma();                             
                                         
# Should print the unicode as expected   
sub use_pragma {                         
    use open ':std', ':encoding(UTF-8)'; 
    print "Using pragma\n";              
    open(F, "utf_chars.txt") or die;     
    while (<F>) {                        
        print;                           
    }                                    
    close(F);                            
    print "-----\n";                     
}                                        
                                         
# Should print the unicode as garbled goo
sub no_pragma {                          
    print "NOT using pragma\n";          
    open(F, "utf_chars.txt") or die;     
    while (<F>) {                        
        print;                           
    }                                    
    close(F);                            
    print "-----\n";                     
}                                        

Just make a utf_chars.txt and put some €s in it or something, and watch it print the same stuff differently depending on the scope.