Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Require the system encoding to be UTF-8 #6305
As seen in #6295, python 3's
One solution to that problem would be to explicitly specify encodings on all
Require a preferred encoding of
If folks are using other encodings for valid reasons, hopefully they will file issues; if not, they will have a workaround.
referenced this pull request
Aug 4, 2018
Also, to be clear: I'm not certain that this is the very best idea. It's possible that:
Is there another option to force the system to use UTF 8? I think I remember seeing a way to do this in Python. Although I’m not sure the ramifications and if that’s safe.
The simplest option to me seems to be requiring UTF-8 because it reduces the complexity of development and simplifies our assumptions. If someone using Pants complains after trying that this breaks their usage, then we could invest in supporting their encoding. Alternative of supporting multiple encodings now may be a premature optimization.
Thanks for looking into this all!
My instinct is that we should just require everyone to use UTF-8 so that we can assume UTF-8 everywhere. Simplifies our world a lot.
In particular, I'm scared about the idea of things crossing the python/rust boundary with other encodings; we very strongly assume utf-8 on that boundary.
Adding an explicit encoding to every
In Python 3.7 we can actually force UTF-8, by settings the env var
So perhaps we should require Python 3.7 for running Pants once we're fully on py3, and until then make this check more strict so that it fails for any preferred encoding that isn't UTF-8?
Definitely agreed having to specify
Requiring Python 3.7 seems like a step up in terms of restrictiveness compared to only requiring UTF-8. Because Python 3.7 came out only a month ago and some libraries still have issues with it, I’m not very comfortable with requiring 3.7.
P.S. Benjy’s comment brings up our need to decide which python 3 languages we’ll support. The most common for libraries in 2018 is to support 3.5+. Which version we choose determines which new features we can use, such as 3.7’s dataclasses. This might also not be the best venue for deciding.