python path for menuconfig.py (IDFGH-4417)

[masterxq]

Hi,
I'm using Debian buster with python 3.8.4. And i have trouble with the path of menuconfig.py lib.

The file:

esp-idf/tools/kconfig_new/menuconfig.py

Wants to call the menuconfig.py from the python kconfig lib. (Talking about two different files with same name here...)

This code:

esp-idf/tools/kconfig_new/menuconfig.py

Line 33 in 178b122

cmd += [os.path.join(get_python_lib(), 'menuconfig.py')]

gives me the path:

/usr/lib/python3.8/menuconfig.py

And this is wrong in my case!

It should be:

/home/user/.local/lib/python3.8/site-packages/menuconfig.py

Which also will be is in list of get_python_lib(). But there is no check if the file exists.

What I did for being able to use the master is:

def find_libfile(filename):
    for syspath in sys.path:
        res = os.path.join(syspath, filename)
        if os.path.isfile(res) and not os.path.samefile(res, __file__):
            return res
    raise Error("Can't find file %s" % filename)

Care! If you don't compare with file you will go into endless loop calling the same file :)

and calling:
cmd += [find_libfile('menuconfig.py')]

If the code is ok, you can use it. I'm not a Python programmer... But it works at least in my case...

[dobairoland]

Hi @masterxq. Your issue makes me wonder if you are in the environment set by . export.sh?

/home/user/.local/lib/python3.8/site-packages This is the local user path where you are installing with pip install --user ....

Your path should be something like this /home/user/.espressif/python_env/idfX.Y_py3.8_env/ where X, Y depend on the IDF version.

Have you run install.sh and . export.sh?

[masterxq]

Hello @danicampora,
ofc you are correct maybe i should have mentioned that i not running install.sh or export.sh. Sorry for that. I like clean systems under my control... EG. this also helps me being able to compile in the IDE, where i can not run export.sh for getting the correct environment variables.
But still, shouldn't the software not be able to work no matter in what path the libs are installed as long as there pathes are exported correctly?
Thanks for reply

[dobairoland]

I like clean systems under my control.

In my opinion, installing packages into a virtual environment for one purpose is much clearer than to install them globally and manage compatibility between all your Python applications. Please note that export.sh doesn't make any permanent change in your system remaining after closing the given terminal window. install.sh creates files in ~/.espressif but makes no real system install one might assume based on the name.

But still, shouldn't the software not be able to work no matter in what path the libs are installed as long as there pathes are exported correctly?

There are big differences between working in a virtual Python environment. The path-related functions work differently, or some of them are not available. I don't particularly like the idea of looking into all PATH directories because we might find a false positive match (or incompatible version). I'm willing to apply a patch if that is not breaking the supported way of working with ESP-IDF. But currently I don't see how it would possible without putting a bunch of if-else branches about virtualenvs, OS types & versions, Python versions which never work for everyone as should be.

this also helps me being able to compile in the IDE

Are you using a different IDE from those we are supporting? We have plugins for Eclipse and Vscode. In other cases, you could run export.sh in a terminal and start the IDE. There should no changes in system variables after you close the IDE and the terminal.

[masterxq]

Hi again,
thank you for explanation.
Please note, that i don't want to say you a doing bad practice. Or that I don't trust in your software. But sometimes it can hard to differ because there much software and installers that using non default pathes and other unwanted behavior. It's often more easy to do it yourself than understand what the script exactly is doing that you want to use.

I have some workstations where many users programming and using the espressif-idf. There i prefer to install libraries we are using into an global path for having less maintain task per user and ofc same level of software.

I see your point but this is the only place where this workflow will break. As i think multiply user are doing the same way as i do, even if its not the best practice in every situation, I hope you will support other ways of handle the espressif too. So that i and others don't need to handle with custom patches all the way.

My suggestion is:

1. Rename the caller menuconfig.py (espressif)
2. Keep default behavior.
3. If the path of menuconfig.py (library) is not pointing to an existing file, do some magic for check other pathes.
4. Make the call

This would keep the wanted way in every situation where it is possible and allow users who want to go another way to do it without writing patches^^

My argumentation is:

The default user will always use the install and export script. Users who don't want to use this are mostly able to maintain there systems on there own...

Using KDevelop and there is a shell but never tested if the shell is using same environment as the the IDE. But there are special environments you can use for this functionality. What i do and what always worked well in the past, and still does because i not run menuconfig in the IDE (just building and debug). It's an excellent IDE that i want continue to use :)

Regards

[dobairoland]

Hi @masterxq. I've got an idea. Does the following patch work for you?

-    cmd += [os.path.join(get_python_lib(), 'menuconfig.py')]
+    cmd += ['-m', 'menuconfig']

[masterxq]

Really nice solution. It's working perfectly!

Thanks for your effort!

[dobairoland]

Great! I'll submit a fix.