Check for duplicates in data set

[mesmith75]

If you try and submit a job with duplicated files in the inputdata the splitter throws out an incomprehensible error. It should fail but we can add an explicit check for duplicates to make it easier to diagnose the problem