Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Editable package install fails after a previous uninstall #2074

Closed
amirhosseindavoody opened this issue Feb 29, 2024 · 17 comments · Fixed by #2259
Closed

Editable package install fails after a previous uninstall #2074

amirhosseindavoody opened this issue Feb 29, 2024 · 17 comments · Fixed by #2259
Assignees
@charliermarsh
Labels
bug Something isn't working needs-mre Needs more information for reproduction

Comments

@amirhosseindavoody
Copy link

Steps to reproduce

  • Activate a conda environment.
  • Install a local package that I have using uv pip install -e .
  • Uninstall the local package using uv pip uninstall -e .
  • Install the local package again using uv pip install -e .

I get the following error:

error: Cannot uninstall package; RECORD file not found at: path/to/site-packages/mylib-0.1.dist-info/RECORD

If I manually remove path/to/site-packages/mylib-0.1.dist-info directory, then I am able to install my local package again.
@charliermarsh charliermarsh added the bug Something isn't working label Feb 29, 2024
@charliermarsh charliermarsh self-assigned this Feb 29, 2024
@charliermarsh
Copy link
Member

Thanks! Probably my bug from the most recent release.

@charliermarsh
Copy link
Member

Do you get this consistently? I wasn't able to reproduce.

@charliermarsh charliermarsh added the needs-mre Needs more information for reproduction label Feb 29, 2024
@amirhosseindavoody
Copy link
Author

I tried this with a fresh environment and the same error happened.

Some more details about my environment setup

  • I am using pixi as the package manager.
  • Then I install uv for my specific environment using pixi add uv
  • Then I use pixi shell to activate the environment.
  • Now I run the uv pip install -e \path\to\my\local\package
  • The local package is using pyproject.toml for setup. Something similar to this

@charliermarsh
Copy link
Member

Thanks, will try again.

@charliermarsh
Copy link
Member

This still worked for me even when running through pixi. Is there anything else you can share about your OS, etc.?

@amirhosseindavoody
Copy link
Author

My OS is Suse 12.

I don't know what else might help.

If this is not reproducible we can close this and others will open something if it happens again.

@charliermarsh
Copy link
Member

I believe it's a real problem if you're seeing it consistently. I just need to find a way to repro.

@iwane-pl
Copy link

iwane-pl commented Mar 1, 2024 •
edited
Loading

We've run on the same error when installing lxml, for example. The procedure was

PS C:\Users\me> C:\Python3.12\python.exe -m venv venv  
PS C:\Users\me> .\venv\Scripts\activate.ps1                                                                                    
(venv) PS C:\Users\me> uv pip install lxml==5.1.0                                                                              
Resolved 1 package in 4ms                                                                                                       
Installed 1 package in 50ms                                                                                                     
 + lxml==5.1.0                                                                                                                  
(venv) PS C:\Users\me> uv pip install lxml==4.9.3                                                                              
Resolved 1 package in 4ms                                                                                                       
Installed 1 package in 51ms                                                                                                     
 - lxml==5.1.0                                                                                                                  
 + lxml==4.9.3                                                                                                                  
(venv) PS C:\Users\me> uv pip install lxml==5.1.0                                                                              
Resolved 1 package in 4ms                                                                                                       
error: Cannot uninstall package; RECORD file not found at: C:\Users\me\venv\Lib\site-packages\lxml-5.1.0.dist-info\RECORD

OS info:

# Windows Server 2016
> (Get-CimInstance Win32_OperatingSystem).version
10.0.14393    
> python --version                                                                                        
Python 3.12.2
> uv --version                                                                                            
uv 0.1.12 (f68b2d1d5 2024-02-28)

Don't ask why we install it that way :) (hint: multiple requirement files in sequence)
Didn't happen on Linux, though.

EDIT: Still present on 0.1.13, despite #2091

@konstin
Copy link
Member

konstin commented Mar 1, 2024

I tried on my windows desktop but i can't reproduce it there

PS C:\Users\Konsti\projects\uv> python.exe -m venv venv 
PS C:\Users\Konsti\projects\uv> .\venv\Scripts\activate.ps1            
(venv) PS C:\Users\Konsti\projects\uv> powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
Downloading uv 0.1.13 (x86_64-pc-windows-msvc)
Installing to C:\Users\Konsti\.cargo\bin
  uv.exe
Everything's installed!
(venv) PS C:\Users\Konsti\projects\uv> uv --version
uv 0.1.13 (9ce5170e6 2024-02-29)
(venv) PS C:\Users\Konsti\projects\uv>  uv pip install lxml==5.1.0                                 

Resolved 1 package in 228ms
Downloaded 1 package in 366ms
Installed 1 package in 65ms
 + lxml==5.1.0
(venv) PS C:\Users\Konsti\projects\uv> uv pip install lxml==4.9.3                         
Resolved 1 package in 170ms
Downloaded 1 package in 473ms
Installed 1 package in 59ms
 - lxml==5.1.0
 + lxml==4.9.3
(venv) PS C:\Users\Konsti\projects\uv> uv pip install lxml==5.1.0                 
Resolved 1 package in 2ms
Installed 1 package in 68ms
 - lxml==4.9.3
 + lxml==5.1.0

@iwane-pl
Copy link

iwane-pl commented Mar 1, 2024 •
edited
Loading

🤔
I tried on other Windows versions too. Same uv (0.1.13) and Python (3.12.2).

# Win10

PS C:\Users\me> (Get-CimInstance Win32_OperatingSystem).version
10.0.19045
PS C:\Users\me> C:\Python3.12\python.exe -m venv venv
PS C:\Users\me> .\venv\Scripts\activate.ps1
(venv) PS C:\Users\me> uv pip install lxml==5.1.0
Resolved 1 package in 84ms
Downloaded 1 package in 150ms
Installed 1 package in 82ms
 + lxml==5.1.0
(venv) PS C:\Users\me> uv pip install lxml==4.9.3
Resolved 1 package in 47ms
Downloaded 1 package in 156ms
Installed 1 package in 107ms
 - lxml==5.1.0
 + lxml==4.9.3
(venv) PS C:\Users\me> uv pip install lxml==5.1.0 
Resolved 1 package in 6ms
Installed 1 package in 75ms
 - lxml==4.9.3
 + lxml==5.1.0


# WinServer2019

PS C:\Users\me> (Get-CimInstance Win32_OperatingSystem).version
10.0.17763
PS C:\Users\me> C:\Python3.12\python.exe -m venv venv
PS C:\Users\me> .\venv\Scripts\activate.ps1
(venv) PS C:\Users\me> uv pip install lxml==5.1.0
Resolved 1 package in 84ms
Downloaded 1 package in 199ms
Installed 1 package in 66ms
 + lxml==5.1.0
(venv) PS C:\Users\me> uv pip install lxml==4.9.3
Resolved 1 package in 33ms
Downloaded 1 package in 224ms
Installed 1 package in 68ms
 - lxml==5.1.0
 + lxml==4.9.3
(venv) PS C:\Users\me> uv pip install lxml==5.1.0
Resolved 1 package in 7ms
error: Cannot uninstall package; RECORD file not found at: C:\Users\me\venv\Lib\site-packages\lxml-5.1.0.dist-info\RECORD 

# WinServer2022

PS C:\Users\me> (Get-CimInstance Win32_OperatingSystem).version
10.0.20348
PS C:\Users\me> C:\Python3.12\python.exe -m venv venv  
PS C:\Users\me> .\venv\Scripts\activate.ps1
(venv) PS C:\Users\me> uv pip install lxml==5.1.0
Resolved 1 package in 187ms
Downloaded 1 package in 158ms
Installed 1 package in 88ms
 + lxml==5.1.0
(venv) PS C:\Users\me> uv pip install lxml==4.9.3
Resolved 1 package in 35ms
Downloaded 1 package in 142ms
Installed 1 package in 87ms
 - lxml==5.1.0
 + lxml==4.9.3
(venv) PS C:\Users\me> uv pip install lxml==5.1.0
Resolved 1 package in 21ms
Installed 1 package in 110ms
 - lxml==4.9.3
 + lxml==5.1.0

It seems like it reproduces on older Windows versions. Updated Win10 and WinServer2022 seem unaffected, while WinServer2016 and 2019 fail.

@gozdal
Copy link

gozdal commented Mar 6, 2024 •
edited
Loading

As @iwane-pl noticed, this happens on Windows 2016 and 2019 but not on Windows 10 and 2022.
The difference is that after an uninstall an empty directory is left befind. E.g. if you have lxml==5.1.0 installed and you downgrade to lxml==4.9.3 then site-packages will have

venv\Lib\site-packages\lxml-4.9.3.dist-info
venv\Lib\site-packages\lxml-5.1.0.dist-info

Where 5.1.0 dir will be empty and hence the error

error: Cannot uninstall package; RECORD file not found at: venv\Lib\site-packages\lxml-5.1.0.dist-info\RECORD

Curiously enough it happens only on selected Windows versions.

@charliermarsh
Copy link
Member

My guess is that this is due to delayed file deletion...? E.g., we delete a file, then test if a directory is empty, and delete the directory if so; but I believe there can be temporary inconsistencies, such that when you delete a file, it may still "exist" for some period of time, or something like that.

@gozdal
Copy link

gozdal commented Mar 6, 2024

It's 100% reproducible, so I'd say it's not a race. Could try Process Monitor tomorrow to see what's happening.

@charliermarsh
Copy link
Member

Yeah what I'm describing could still be consistent with it being 100% reproducible.

@charliermarsh
Copy link
Member

I don't have a system to test this on, so the best I can do is try a refactor and ask someone here to test it.

@charliermarsh
Copy link
Member

Good news, I'm able to reproduce this on Windows 2019 in GitHub Actions! #2259

@charliermarsh charliermarsh mentioned this issue Mar 7, 2024
Merged
@charliermarsh
Copy link
Member

Ok, I believe this is fixed by #2259. If you're able to reproduce after the next release, please let me know.

charliermarsh added a commit that referenced this issue Mar 7, 2024
@charliermarsh
Close RECORD after reading entries during uninstall (#2259)
59f4639 
## Summary

It turns out that by keeping the `RECORD` file open, older versions of
Windows mark it for deletion, but don't allow it to be deleted until
it's closed. As such, we end up leaving the `.dist-info` directory
around, since it appears non-empty; but once the program terminates, we
_do_ delete `RECORD`, leaving it empty. This then creates the impression
that a package exists where it does not.

Closes #2074.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@charliermarsh charliermarsh

Labels
bug Something isn't working needs-mre Needs more information for reproduction
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Close RECORD after reading entries during uninstall astral-sh/uv
5 participants
@gozdal @charliermarsh @konstin @amirhosseindavoody @iwane-pl