Problem inside Python 'check_arg_types'

[PABourdin]

Hello,

since about 2 days, I have a problem running "git restore-mtime". Before then, it worked well. I suspect this is an issue with a very recent update of my Python version to 3.7.13. Let me show the error:

$ cd $HOME/build/pencil-code/pencil-code/ && git restore-mtime src/
Traceback (most recent call last):
  File "/usr/lib/git-core/git-restore-mtime", line 157, in <module>
    lsfileslist.add(os.path.relpath(line.strip(), workdir))
  File "/opt/pyenv/versions/3.7.13/lib/python3.7/posixpath.py", line 485, in relpath
    genericpath._check_arg_types('relpath', path, start)
  File "/opt/pyenv/versions/3.7.13/lib/python3.7/genericpath.py", line 155, in _check_arg_types
    raise TypeError("Can't mix strings and bytes in path components") from None
TypeError: Can't mix strings and bytes in path components

PS: This happens when using ubuntu 16.04 (default on Travis!). I switched to using ubuntu 20.04 now and it works again...
It would be great if you could have a look, since many people use Travis builds in their default configuration.

Thanks and best greetings!

[MestreLion]

What version of git-restore-mtime are you using, and how did you obtain it? (Ubuntu repository, github, etc?)

[MestreLion]

Also, do you know what exact python version you had previously and was working ok? It should give some clues on the bug

[MestreLion]

I suspect this was already fixed in 8ec84de, available since v2020.09. Care to test if this happens with latest release?

[PABourdin]

We used always the latest git-restore-mtime because our Travis build does an apt install git-restore-mtime on each run.

The python version that was previously working for us on ubuntu 16.04 is unknown to me and we can not see this, because Travis did not show their Python version until the time when the build failed for us.

[kilobyte]

In that traceback, I see you're not even using a system Python but some weird third-party build; that kind of mix notoriously tends to confuse versions of installed packages. For an ecosystem as complex as Python, I doubt that's a good idea.

There's no supported version of Ubuntu anymore that ships Python 3.7; I see that Debian buster (oldstable, soon to become oldoldstable) still has that. If someone bothers to test platforms that old, I'd recommend using 3.7 on buster instead. Or, perhaps 3.6 on Ubuntu bionic, or 3.8 on focal.

But really, what's the point of using most recent software on an ancient platform? There was that quote about new wine in old bottles... You might have better luck using git-restore-mtime from those times (unless you require a new feature, but it's not a fast moving project), which can be installed from Ubuntu repositories.

[PABourdin]

@kilobyte Well, that is not my choice, as I said I am running on the default build environment of Travis CI, which is a sub firm of GitHub. So, I guess they are important enough to care for their setup?

Otherwise, I am fine, I change the default build settings now to use 20.04 (which works).

[MestreLion]

As @kilobyte said, mixing Python versions can be problematic, but in this case this is not exactly the culprit: git-restore-mtime is fully compatible with even Python 3.6 (the one that ships in Ubuntu 18.04 bionic, which, believe it or not, I still use in my main workstation (hey, it still has 3 months of support left! lol)

The problem in your case is: the bug you're experiencing have already been reported in late 2016 at #23 and fixed in 2017 when PR #26 was merged, but the version available in Ubuntu 16.04 repositories is still the First official release from 2015!!!

So, apart from moving to 20.04, you have some other choices:

• Get the latest version of git-restore-mtime, or any version since 2017, when this bug was fixed. The one from Ubuntu 18.04 should work fine in 16.04.
• Revert your 16.04 python back to 2.7, as it was originally, since this bug was only triggered in Python 3.

[MestreLion]

@kilobyte Well, that is not my choice, as I said I am running on the default build environment of Travis CI

Python 3.7 was released in 2018, so I doubt this is the default Travis environment for Ubuntu 16.04. If you've upgraded it to a 2018 python, grab a 2018 git-restore-mtime

[PABourdin]

@MestreLion I am just wondering why this bug then occurred only since 2 days... My suspicion is that Travis updated their build environment (in particular their Python version) - which is ubuntu 16.04 by default (as you can verify here: https://docs.travis-ci.com/user/reference/linux ) and probably some Python 3.5 version before. And since just two days they started using the newer Python 3.7.13 version for a reason that is unknown to me, but might be a valid reason...

Unfortunately, I am not allowed to load any other version than available in the official packet resources. At least you know now that you can point users to use newer ubuntu 20.04 in case they experience the same issue on Travis.

Let this be our conclusion and close this bug?

[kilobyte]

In other words, Travis changed their build environment to use some random third-party Python builds but didn't update the rest of the system to match.

Unfortunately, I am not allowed to load any other version than available in the official packet resources.

... which is exactly what Travis is doing, with predictable fallout.

Let this be our conclusion and close this bug?

With the culprit identified, and workarounds available, +1 from me.

[MestreLion]

probably some Python 3.5 version before.

I believe it was 2.7, as that bug also happened in Python 3.5.

they started using the newer Python 3.7.13 version for a reason that is unknown to me

Possibly cos 2.7 is EOL'd since 2020? And 3.7 might be the current oldest non-EOL Python.

Still, I'm perplexed that Travis still have 16.04 available, let alone it being the default (it has no support since 2021!!!), and even more perplexed they upgraded its system python to an incompatible version.

Unfortunately, I am not allowed to load any other version than available in the official packet resources.

Well, they upgraded Python to 2018 while at the same time locking you to 2016 repositories??? This is a truly nonsense scenario.

Let this be our conclusion and close this bug?

Yup!

[MestreLion]

At least you know now that you can point users to use newer ubuntu 20.04 in case they experience the same issue on Travis.

Just a final remark here: the problem was not the ancient Ubuntu 16.04, you don't need a modern distro (or python) to run this. The problem was the ancient git-restore-mtime of 2015, when compatibility with Python 3 was still full of rough edges..