Glob sort helper doesn't work when numbers in filenames are padded with spaces

[prashnts]

Hey, thanks for this library! There's an issue with the implementation of pims.utils.sort.natural_keys implementation where the sorted path names are not correct.

Reproducing

Use a glob pathspec with filenames with numbers padded with spaces.
Example: /path/to/files/img 1.tif, /path/to/files/img 42.tif etc.

You can use the following snippet to reproduce this:

>>> fake_names = [f'/data/meh/img-{x:5d}.tiff' for x in range(20)]
>>> sorted(fake_names, key=pims.utils.sort.natural_keys)
['/data/meh/img-   10.tiff',
 '/data/meh/img-   11.tiff',
 '/data/meh/img-   12.tiff',
 '/data/meh/img-   13.tiff',
 '/data/meh/img-   14.tiff',
 '/data/meh/img-   15.tiff',
 '/data/meh/img-   16.tiff',
 '/data/meh/img-   17.tiff',
 '/data/meh/img-   18.tiff',
 '/data/meh/img-   19.tiff',
 '/data/meh/img-    0.tiff',
 '/data/meh/img-    1.tiff',
 '/data/meh/img-    2.tiff',
 '/data/meh/img-    3.tiff',
 '/data/meh/img-    4.tiff',
 '/data/meh/img-    5.tiff',
 '/data/meh/img-    6.tiff',
 '/data/meh/img-    7.tiff',
 '/data/meh/img-    8.tiff',
 '/data/meh/img-    9.tiff']

Expected result:

['/data/meh/img-    0.tiff',
 '/data/meh/img-    1.tiff',
 '/data/meh/img-    2.tiff',
 '/data/meh/img-    3.tiff',
 '/data/meh/img-    4.tiff',
 '/data/meh/img-    5.tiff',
 '/data/meh/img-    6.tiff',
 '/data/meh/img-    7.tiff',
 '/data/meh/img-    8.tiff',
 '/data/meh/img-    9.tiff',
 '/data/meh/img-   10.tiff',
 '/data/meh/img-   11.tiff',
 '/data/meh/img-   12.tiff',
 '/data/meh/img-   13.tiff',
 '/data/meh/img-   14.tiff',
 '/data/meh/img-   15.tiff',
 '/data/meh/img-   16.tiff',
 '/data/meh/img-   17.tiff',
 '/data/meh/img-   18.tiff',
 '/data/meh/img-   19.tiff']

I can make a pull-request with a fix implemented if you're interested.

Note that the regular sorted call without using this key function works correctly. Tested in python 3.6 on macOS.

Thanks!

[nkeim]

Thanks for the excellent bug report! Yes, if you can make a PR that would be awesome; this should certainly be fixed. On Sep 1, 2018, at 8:21 AM, Prashant Sinha <notifications@github.com<mailto:notifications@github.com>> wrote: Hey, thanks for this library! There's an issue with the implementation of pims.utils.sort.natural_keys implementation where the sorted path names are not correct. Reproducing Use a glob pathspec with filenames with numbers padded with spaces. Example: /path/to/files/img 1.tif, /path/to/files/img 42.tif etc. You can use the following snippet to reproduce this:

>> fake_names = [f'/data/meh/img-{x:5d}.tiff' for x in range(20)] >> sorted(fake_names, key=pims.utils.sort.natural_keys)

['/data/meh/img- 10.tiff', '/data/meh/img- 11.tiff', '/data/meh/img- 12.tiff', '/data/meh/img- 13.tiff', '/data/meh/img- 14.tiff', '/data/meh/img- 15.tiff', '/data/meh/img- 16.tiff', '/data/meh/img- 17.tiff', '/data/meh/img- 18.tiff', '/data/meh/img- 19.tiff', '/data/meh/img- 0.tiff', '/data/meh/img- 1.tiff', '/data/meh/img- 2.tiff', '/data/meh/img- 3.tiff', '/data/meh/img- 4.tiff', '/data/meh/img- 5.tiff', '/data/meh/img- 6.tiff', '/data/meh/img- 7.tiff', '/data/meh/img- 8.tiff', '/data/meh/img- 9.tiff'] Expected result: ['/data/meh/img- 0.tiff', '/data/meh/img- 1.tiff', '/data/meh/img- 2.tiff', '/data/meh/img- 3.tiff', '/data/meh/img- 4.tiff', '/data/meh/img- 5.tiff', '/data/meh/img- 6.tiff', '/data/meh/img- 7.tiff', '/data/meh/img- 8.tiff', '/data/meh/img- 9.tiff', '/data/meh/img- 10.tiff', '/data/meh/img- 11.tiff', '/data/meh/img- 12.tiff', '/data/meh/img- 13.tiff', '/data/meh/img- 14.tiff', '/data/meh/img- 15.tiff', '/data/meh/img- 16.tiff', '/data/meh/img- 17.tiff', '/data/meh/img- 18.tiff', '/data/meh/img- 19.tiff'] I can make a pull-request with a fix implemented if you're interested. Note that the regular sorted call without using this key function works correctly. Tested in python 3.6 on macOS. Thanks! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#309>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AC2NbZb252wgWhPzdoCiN4CpatqYyvygks5uWqXXgaJpZM4WWQjW>.

[tacaswell]

While I agree we should fix this bug, I strongly encourage you to switch to zero-padded file names (or at least filenames without spaces in them).

[prashnts]

I agree @tacaswell -- however, this dataset was obtained by a colleague. @nkeim I'll send a PR asap, would also add a few test clauses for this. Thank you both!