poetry install .venv from pyenv-win make problem for pywin32 install

[leolcao]

My idea is to use pywin32 create windows service, but meet problem with poetry install.

Background Environment:

• windows 10 22H2
• python for windows 3.10.11
• pyenv-win 3.1.1
• poetry 1.6.1 installed by using pipx 1.2.0
• pyproject.toml, the related part:

[tool.poetry.group.dev.dependencies]
pywin32 = { version = '306', platform = 'win32' }

• pyenv installed the python 3.12.0

• poetry config --list

virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = true
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false

Steps:

• poetry install -vvv # this step will use python 3.12.0 installed by pyenv

Loading configuration file C:\Users\leo.cao\AppData\Roaming\pypoetry\config.toml
Adding repository tinghua (https://pypi.tuna.tsinghua.edu.cn/simple) and setting it as the default one
Adding repository aliyun (https://mirrors.aliyun.com/pypi/simple) and setting it as primary
Deactivating the PyPI repository
The currently activated Python version 3.10.11 is not supported by the project (3.12.0).
Trying to find and use a compatible version.
Trying python3
Using python3 (3.12.0)
Creating virtualenv py-service in D:\LeoCao\Projects\llc\techstack.git\1.python\0.register-service\.venv
Using virtualenv: D:\LeoCao\Projects\llc\techstack.git\1.python\0.register-service\.venv
Installing dependencies from lock file
...

• Check the site info in the .venv
  .\.venv\Scripts\python.exe -m site

sys.path = [
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service',
    'D:\\LeoCao\\Programs\\pyenv\\pyenv-win\\versions\\3.12.0\\python312.zip',
    'D:\\LeoCao\\Programs\\pyenv\\pyenv-win\\versions\\3.12.0\\DLLs',
    'D:\\LeoCao\\Programs\\pyenv\\pyenv-win\\versions\\3.12.0\\Lib',
    'D:\\LeoCao\\Programs\\pyenv\\pyenv-win\\versions\\3.12.0',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv\\Lib\\site-packages',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv\\Lib\\site-packages\\win32',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv\\Lib\\site-packages\\win32\\lib',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv\\Lib\\site-packages\\Pythonwin',
]
USER_BASE: 'C:\\Users\\leo.cao\\AppData\\Roaming\\Python' (exists)
USER_SITE: 'C:\\Users\\leo.cao\\AppData\\Roaming\\Python\\Python312\\site-packages' (doesn't exist)
ENABLE_USER_SITE: False

• Run the python service to install windows service

poetry run python ./src/python_windows_service.py install in the evaluated powershell:

Installing service PythonWindowsService
copying host exe 'D:\LeoCao\Projects\llc\techstack.git\1.python\0.register-service\.venv\Lib\site-packages\win32\pythonservice.exe' -> 'D:\LeoCao\Projects\llc\techstack.git\1.python\0.register-service\.venv\pythonservice.exe'
copying helper dll **'D:\LeoCao\Projects\llc\techstack.git\1.python\0.register-service\.venv\Lib\site-packages\pywin32_system32\pywintypes312.dll' -> 'D:\LeoCao\Programs\pyenv\pyenv-win\versions\3.12.0\pywintypes312.dll'**

Now, if I run python ./src/python_windows_service.py debug will show nothing, and if I go to windows service dialog and start the service manually, it will throw error 1053.

Then, I use miniconda to create the new .env, and do above steps again, and find everything is good.

I find the difference between poetry install and miniconda way.

• The folder and files structure in the .venv is different:
  • Poetry install will not copy the original python version files, only few files
  • the location of 'python.exe' is different: ./venv/python.exe Vs ./venv/Scripts/python.exe
• The key point is that pywintypes312.dll is copied into the pyenv-win python location. But in miniconda way, this file will be copied into .venv/ folder, just alongside the ./venv/python.exe.

Expectations

poetry install can copy all of files from pyenv installed version, and not link to the location of pyenv folders, so that, pywin32 will work

[dimbleby]

poetry is responsible for unpacking the pywin32 wheel into the virtual environment where you are installing that wheel. Nothing more.

what happens when you run your code is between you and pywin32.

I see no poetry bug.

[leolcao]

eh, let me use another way to explain~

Now, let's just forget pywin32, only compare poetry created .venv from pyenv-win and miniconda created .venv.

If i use miniconda create the .env by using conda create -p .venv python=3.12.0, the folder .venv will like below(partially):

.venv/
  DLLs/
  Libs/
  Library
  libs/
  Scripts/
    pip.exe
    pip3.exe
  Tools/
  share/
  python.exe
  pythonew.exe
  ...

and if I check the site for this .venv by using .\.venv\python.exe -m site, it will shows output like this:

sys.path = [
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv\\python312.zip',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv\\DLLs',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv\\Lib',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv\\Lib\\site-packages',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv\\Lib\\site-packages\\win32',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv\\Lib\\site-packages\\win32\\lib',
    'D:\\LeoCao\\Projects\\llc\\techstack.git\\1.python\\0.register-service\\.venv\\Lib\\site-packages\\Pythonwin',
]
USER_BASE: 'C:\\Users\\leo.cao\\AppData\\Roaming\\Python' (exists)
USER_SITE: 'C:\\Users\\leo.cao\\AppData\\Roaming\\Python\\Python312\\site-packages' (doesn't exist)
ENABLE_USER_SITE: True

And poetry installed .venv from pyenv-win, the folder structure is like below(partially):

.venv/
  Lib/
    site-packages/
      ...
  Scripts/
     python.exe
     pip.exe
    ...
  pyvenv.cfg

So maybe it is not poetry's issue, it is only my finding the folder structure and python sites output is the key to make pywin32 running successfully.

With the conda created .venv, the pywin32 install command will do right copy:

copying helper dll **'D:\LeoCao\Projects\llc\techstack.git\1.python\0.register-service\.venv\Lib\site-packages\pywin32_system32\pywintypes312.dll' -> D:\LeoCao\Projects\llc\techstack.git\1.python\0.register-service\.venv\pywintypes312.dll'

Thanks for any clue~

[dimbleby]

poetry just uses virtualenv to create environments, I expect you can reproduce the same without poetry by calling virtualenv directly: and if you think there's a problem with those environments then you should talk to the virtualenv project

[leolcao]

Ok I will. thanks any way~

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.