chore: popen using utf8 encoding

[fatelei]

Describe the changes you have made:

Reference any relevant issue (Fixes #000)

fix #209 using utf8 encoding instead of cp1255

• I have performed a self-review of my code:

I have tested the code on the following OS:

• Windows
• MacOS
• Linux

AI Language Model (if applicable)

• GPT4
• GPT3
• Llama 7B
• Llama 13B
• Llama 34B
• Huggingface model (Please specify which one)

[fatelei]

if we want to customize env not affect system env, copy it may be better

[ericrallen]

@fatelei You’re right. I deleted my last review question after looking at things a bit more closely.

Going to test this one out in a bit so we can get it merged.

Thanks for the contribution!

[ericrallen]

Based on my limited testing, this seems to work as expected.

I've never encountered the error that other folks were getting, though.

[fatelei]

Based on my limited testing, this seems to work as expected.

I've never encountered the error that other folks were getting, though.

what about unit tests

[ericrallen]

@fatelei If you can think of a good way to recreate a situation that triggers the error and then add a unit test where we make sure that interpreter can still execute code, that would be great.

I haven't had a chance to put brainpower towards it because I'm not sure what the underlying environment setting that causes an issue in the first place is.

I guess we could just set PYTHONIOENCODING to something else in the test, but then we'd just be testing if setting the encoding works rather than if setting it and passing the environment to the subprocess fixes the underlying issue with the system having a different encoding.

[KillianLucas]

This seems like a good idea— utf-8 handles diverse output chars including the European chars the issues seemed to be having difficulty with. Pushing to pip soon so I'm going to merge this.

I think in the future a good unit test would try out a bunch of different non-English chars, but something tells me that might only be useful once we can be testing things across a bunch of different OS/terminals, not just the github ubuntu non-GUI one.

Great work @fatelei and thanks @ericrallen for reviewing!