cmake: west: invoke west using same python as rest of build system

[tejlmand]

When running CMake, then Python3 will be used.
This is detected through FindPython3, with a preference for using the
python or python3 in path, if any of those matches the required Python
minimal version in Zephyr.

It is also possible for users to specify a different Python, as example
by using:
cmake -DPYTHON_PREFER=/usr/bin/python3.x

However, when running west as native command, then west will be
invoked on linux based on the python defined in:
west launcher, which could be: #!/usr/bin/python3.y

Thus there could be mismatch in Pythons used for west and the python
used for other scripts.

This is even worse on windows, where a user might experience:

>.\opt\bin\Scripts\west.exe --version
Traceback (most recent call last):
  File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  ...
  File "C:\Python37\lib\socket.py", line 49, in <module>
    import _socket
ImportError: Module use of python38.dll conflicts with this version of
Python.

when testing out a newer Python, but the python in path is still a 3.7.

By importing west into zephyr_module.py and by using
python -c "from west.app.main import main; main(['<command>'])"
we ensure the same python is used in all python scripts.

Also it allows the user to control the python to use for west.

It also ensures that the west version being tested,m is also the version
being used, where old code would test the version imported by python,
but using the west in path (which could be a different version)

If the west version installed in the current Python, and west invocation
is using a different Python interpreter, then an additional help text
is printed, to easier assist users with debugging.

Signed-off-by: Torsten Rasmussen Torsten.Rasmussen@nordicsemi.no

[zephyrbot]

All checks are passing now.

Tip: The bot edits this comment instead of posting a new one, so you can check the comment's history to see earlier messages.

[tejlmand]

Example of improved help text, if wanting to build with python3.7 when python3 points to another version.

west build -bnrf52840dk_nrf52840 -dbuild_west -- -DPYTHON_PREFER=/usr/bin/python3.7
...
CMake Error at /projects/github/ncs/zephyr/cmake/host-tools.cmake:38 (message):
  The detected west version is unsupported.

    The version was found to be 0.7.0:
      
    But the minimum supported version is 0.7.1
    Please upgrade with:
        /usr/bin/python3.7 -mpip install --upgrade west

    Note:
    The Python version used by west is:  /usr/bin/python3
    The Python version used by CMake is: /usr/bin/python3.7
    This might be correct, but please verify your installation.

[tejlmand]

@mbolivar due to compliance test, I discovered that I wasn't testing the error message properly, when not in a west workspace.
Code now redirects sys.stderr to be able to correctly check the message, so please re-review.

[mbolivar-nordic]

I was testing this out before adding my +1, but this breaks west build for me.

Example steps to reproduce issues:

1. west build -b nrf9160dk_nrf9160 -s samples/hello_world/: this runs cmake to generate a build system, but it doesn't actually build anything (west build normally invokes cmake in build tool mode to finish the job). Perhaps the zcmake changes need to be guarded by a kwarg that can be turned off when running cmake in build tool mode.
2. Verbose builds don't seem to work at all:

$ west -v build -b nrf9160dk_nrf9160 -s samples/hello_world/ 
ZEPHYR_BASE=/home/mbolivar/zp/zephyr (origin: configfile)
cmake version 3.17.3 is OK; minimum version is 3.13.1
Running CMake: /usr/bin/cmake -DWEST_PYTHON=/home/mbolivar/.virtualenvs/west-dev/bin/python --build /home/mbolivar/zp/zephyr/build -- -v
CMake Error: The source directory "/home/mbolivar/zp/zephyr/-v" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
FATAL ERROR: command exited with status 1: /usr/bin/cmake -DWEST_PYTHON=/home/mbolivar/.virtualenvs/west-dev/bin/python --build /home/mbolivar/zp/zephyr/build -- -v

[tejlmand]

I was testing this out before adding my +1, but this breaks west build for me.

Fixed, see also: #26060 (comment)

[mbolivar-nordic]

Code looks fine and my basic tests are now passing. Thanks!

[tejlmand]

@nashif could you take a look ?

[sjg20]

Why this blank line?

[tejlmand]

That is probably due to moving code into this file, and then out again, as discussed:
#26060 (comment)

unfortunately this must have resulted in a blank line in that process.
You seem to be the first who noticed.