Ipython generates .pyc files even thought PYTHONDONTWRITEBYTECODE is set to 1 in Windows 10 and Ubuntu (Linux)

[alphaCTzo7G]

I run ipython on both linux and windows. Recently I have had situations where my code has changed, but the bytecode doesn’t update or change. So I had to turn off the bytecode generation, otherwise debugging becomes a headache.

I have set %env PYTHONDONTWRITEBYTECODE =1 in a file 00-startup.py under .ipython>profile_default>startup folder (https://stackoverflow.com/a/154617/4752883).

In linux, setting PYTHONDONTWRITEBYTECODE seems to work fine for the last few days, in that it doesnt generate bytecode.

On both linux and windows, I have checked using the %env magic parameter that PYTHONDONTWRITEBYTECODE is set correctly. However, ipython continues to generate bytecode in the ipython running under Windows 10.

Does ipython not respect PYTHONDONTWRITEBYTECODE on Windows or Linux?

[takluyver]

IPython doesn't do anything with bytecode directly - all your code is passed through to Python to be executed, so it's still Python considering that environment variable.

One possibility is that Python on Windows checks for the environment variable when it starts, so setting it in the process doesn't affect anything. I guess you'd have to read the source code to be certain whether that's what's going on, though.

[alphaCTzo7G]

Thanks for the points. I will try to figure out a solution and post it here so it may help others.

I wrote down my test plan below.. and also had some questions. Would you know the answers to the highlight questions below?

I understand that .pyc files get generated only when I do _imports_, not when I do a %run in ipython (http://effbot.org/pyfaq/how-do-i-create-a-pyc-file.htm). Is this correct?

However, regardless of whether PYTHONDONWRITEBYTECODE is set to TRUE or not, it seems to be intermittent, in Windows 10

So my first plan is to get a repeatable test case first, where I can atleast generate the bytecode in Windows10 repeatably. Rightnow, I cant even generate the Bytecode repeatably, even if I dont have PYTHONDONTWRITEBYTECODE set.

These are the tests I am thinking of

1. Set the environment variable ‘PYTHONDONTWRITEBYTECODE=1” in windows environmental parameters directly
   a) under Control Panel>System>User variables
   b) under Control Panel>System>Systemvariables
2. import sys; sys.dont_write_bytecode=True
   a) In ipython (Q: what directory should I place this .py file in?)
   b) In python. (Q: you mentioned that ipython calls python. Does it pass any specific .py to python before loading python, so I can put the above import statements in there? If not, should I just put it in the ~/.ipython/startup/ directory as a 01-startup.**py** file?)
3. Set the value PYTHONDONTWRITEBYTECODE=1 in the first file (in the starteup folder in ipython) that ipython. (Q: Is it correct that ipython loads the files in the ~/.ipython/startup/00-startup.ipy first during startup?)
4. Figure out how to sent the -B flag to python when ipython starts https://docs.python.org/3/using/cmdline.html#id1 (Q: I check the %run magic in ipython, but didn’t see an option to set the -B flag. When ipython calls python3 is there a way to send the -B flag to python3?)

If these don’t, work my plan is to pretty much trigger a pyclean to clean up any .pyc files in the code directory, on each write, for a short term solution, and start looking into the source code.

Q: As for digging into the ipython source code do you have an entry where I should start looking as to where the PYTHONDONTWRITEBYTECODE is set? I can debug from there.

[alphaCTzo7G]

Just wanted to update with my findings so far:

1. %env PYTHONDONTWRITEBYTECODE =True if loaded by ipython during startup does get set in the environment variables. I can see it using %env. However ipython still will generate bytecode. If you load ipython and then try to set %env PYTHONDONTWRITEBYTECODE=1, ipython/python doesnt respect it either.
2. There is no difference in the bytecode generation between linux and windows 10.
3. If I use import sys; sys.dont_write_bytecode=True in 00-startup.py under ipython\startup, it does work in both Linux and Windows 10
   I wanted to use the same configuration files for both linux and windows so I added import sys; sys.dont_write_bytecode=True in 00-startup.py under .ipython\startup.

If there is a demand, I will check why %env PYTHONDONTWRITEBYTECODE = True is not respected in the ipython

Here are some experiments (folders) I ran. Each folder is a single experiment and has a README.txt with the experiment and the results details. These are primarily run on Windows 10, however, I did check the conclusions on my Ubuntu 16.04 machine as well: https://github.com/alphaCTzo7G/stackexchange/tree/master/python/ipython_windows_bytecode02192018

[takluyver]

As for digging into the ipython source code do you have an entry where I should start looking as to where the PYTHONDONTWRITEBYTECODE is set? I can debug from there.

We don't set or read it at all, as far as I know. It's used by Python itself (which IPython runs on top of).