Issues/4101 wrong filename in output

[gmloose]

Changelog Entry

To be copied to the draft changelog by merger:

• Fix wrong filename in output due to name conflict
  The logic that determines a potential filename conflict contains a flaw. If one of the outputs is a directory that contains (at least) two files with identical names except for a "trailing underscore part", then, depending on the order in which these files are visited, the filename with the "trailing underscore part" may be considered a duplicate of that without. For example, in my case there are files named table.f4 and table.f4_TSM0. If table.f4 was already visited, then table.f4_TSM0 is considered a duplicate of `table.f4. This PR corrects this incorrect behavior.

Reviewer Checklist

• Make sure it is coming from issues/XXXX-fix-the-thing in the Toil repo, or from an external repo.
  • If it is coming from an external repo, make sure to pull it in for CI with:

    contrib/admin/test-pr otheruser theirbranchname issues/XXXX-fix-the-thing
    

  • If there is no associated issue, create one.
• Read through the code changes. Make sure that it doesn't have:
  • Addition of trailing whitespace.
  • New variable or member names in camelCase that want to be in snake_case.
  • New functions without type hints.
  • New functions or classes without informative docstrings.
  • Changes to semantics not reflected in the relevant docstrings.
  • New or changed command line options for Toil workflows that are not reflected in docs/running/{cliOptions,cwl,wdl}.rst
  • New features without tests.
• Comment on the lines of code where problems exist with a review comment. You can shift-click the line numbers in the diff to select multiple lines.
• Finish the review with an overall description of your opinion.

Merger Checklist

• Make sure the PR passes tests.
• Make sure the PR has been reviewed since its last modification. If not, review it.
• Merge with the Github "Squash and merge" feature.
  • If there are multiple authors' commits, add Co-authored-by to give credit to all contributing authors.
• Copy its recommended changelog entry to the Draft Changelog.
• Append the issue number in parentheses to the changelog entry.

[mr-c]

Thanks!

I wonder how we can create a unit test for this...

[gmloose]

I'm still trying to reproduce it in a small example. It looks like I'm making some progress. So hold on ...

[adamnovak]

This looks like it ought to work. I've pulled it in for testing.

It would be nice to have a test, but we shouldn't not fix the bug if we don't get one.

[DailyDreaming]

LGTM too. Thanks for submitting the fix.

[gmloose]

I added a test that seem to properly work. It fails on current master and passes on this branch. I tested it on two different systems in a virtualenv using Python 3.8

[mr-c]

I added a test that seem to properly work. It fails on current master and passes on this branch. I tested it on two different systems in a virtualenv using Python 3.8

Thanks! I've pushed that fix up for testing.