[Core] Explicit encoding for opening files and update pylint

[Michaelvll]

Fixes #3152

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)
• All smoke tests: pytest tests/test_smoke.py
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: bash tests/backward_comaptibility_tests.sh

[romilbhardwaj]

Thanks @Michaelvll! Looks like we need to be careful about all open() calls.

As an alternative, can we somehow globally set to use utf-8 for all calls? That reduces chances of bugs if a developer forgets to add encoding='utf-8'.

Looks like the UTF-8 mode aims to do so, and can be forced by setting PYTHONUTF8=1 environment variable. Any way we can use this?

[Michaelvll]

Thanks @Michaelvll! Looks like we need to be careful about all open() calls.

As an alternative, can we somehow globally set to use utf-8 for all calls? That reduces chances of bugs if a developer forgets to add encoding='utf-8'.

Looks like the UTF-8 mode aims to do so, and can be forced by setting PYTHONUTF8=1 environment variable. Any way we can use this?

Thanks for pointing that out @romilbhardwaj! It seems that explicitly specifying the encoding for every open is a recommended practice according to the PEP standard: https://peps.python.org/pep-0597/ and also a warning in pylint: https://pylint.readthedocs.io/en/latest/user_guide/messages/warning/unspecified-encoding.html

It might be a good chance for us to add the encoding everywhere in our code and updating our pylint version. Wdyt?

[romilbhardwaj]

Thanks @Michaelvll! Adding to pylint check is a good idea. LGTM.