Dependency Install Failure During Plugin Install Doesn't Fail The Plugin Install

[QuinnDamerell]

The problem

I had a community member reachout to me and say my plugin wasn't working after being installed. He sent his OctoPrint logs and it looked like one of the dependency libs OctoEverywehre uses wasn't installed on his system. I asked him to resinstall the plugin and send the install window output, and this is what he got:

Installing plugin from uploaded file...
/home/pi/oprint/bin/python -m pip --disable-pip-version-check install file:///tmp/tmprvq8z421.zip --no-cache-dir
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /tmp/tmprvq8z421.zip
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: OctoPrint in /home/pi/oprint/lib/python3.9/site-packages (from OctoEverywhere==1.11.2) (1.8.6)
Requirement already satisfied: certifi in /home/pi/oprint/lib/python3.9/site-packages (from OctoEverywhere==1.11.2) (2022.9.24)
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x75453a30>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/dnspython/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x754a4a18>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/dnspython/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x754a4568>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/dnspython/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x754a4028>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/dnspython/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x754a4be0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/dnspython/
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x75453c40>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/dnspython/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x75453d90>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/dnspython/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x75453e80>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/dnspython/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x75453f70>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/dnspython/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x75453fe8>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/dnspython/
ERROR: Could not find a version that satisfies the requirement dnspython (from octoeverywhere) (from versions: none)
ERROR: No matching distribution found for dnspython
Looks like the plugin was already installed. Forcing a reinstall.
/home/pi/oprint/bin/python -m pip --disable-pip-version-check install file:///tmp/tmprvq8z421.zip --no-cache-dir --ignore-installed --force-reinstall --no-deps
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /tmp/tmprvq8z421.zip
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: OctoEverywhere
Building wheel for OctoEverywhere (setup.py): started
Building wheel for OctoEverywhere (setup.py): finished with status 'done'
Created wheel for OctoEverywhere: filename=OctoEverywhere-1.11.2-py3-none-any.whl size=136125 sha256=f38b07f0105b122b3e56b11dfa9d1475d267cbdee5b5c65bdaaacae2d54639dc
Stored in directory: /tmp/pip-ephem-wheel-cache-sf50_750/wheels/33/35/04/9d47d6cd7ec4a71de807900a1ee213fe0a807b1eb00be1b057
Successfully built OctoEverywhere
Installing collected packages: OctoEverywhere
Successfully installed OctoEverywhere-1.11.2
Done!

What it looks like is, for some reason, the dnspython package failed to install from PyPi due to a domain lookup failure (ironic). But when it failed to install, the system didn't stop the plugin install or tell the user of the failure. Upon restart, my plugin fails to load due to the missing import.

Ideally, I think if this happens, the plugin should not be installed since it won't be in a good state, with the missing dependency.

Did the issue persist even in safe mode?

Yes, it did persist

If you could not test in safe mode, please state why

No response

Version of OctoPrint

Latest

Operating system running OctoPrint

OctoPi

Printer model & used firmware incl. version

No response

Browser and version of browser, operating system running browser

No response

Checklist of files to include below

• Systeminfo Bundle (always include!)
• Contents of the JavaScript browser console (always include in cases of issues with the user interface)
• Screenshots and/or videos showing the problem (always include in case of issues with the user interface)
• GCODE file with which to reproduce (always include in case of issues with GCODE analysis or printing behaviour)

Additional information & file uploads

No response

[QuinnDamerell]

(I don't have a system info bundle, as this bug report came through a support ticket)

[jneilliii]

I've had similar issues recently with my Google Drive plugins due to dependency issues. IIRC when that does happen the lines that show error are in red, but you're right the plugin still shows successful install.

[cp2004]

This is the interesting part, and what makes it a bug in OctoPrint (initially when I was reading I thought this was pip's problem):

Looks like the plugin was already installed. Forcing a reinstall.

From a quick look at the code, OctoPrint is probably picking up the 'Requirement already satisfied' lines (for the first couple of deps) and considering that pip has ignored installing everything, so we need a force-reinstall. But this is of course not the case.

[QuinnDamerell]

Yeah, I should have also said, for this user it was a “fresh” install in OctoPrint’s eyes, since the plug-in wasn’t loaded. But the package was on disk and OctoPrint was setup to use it, it just failed to load due to the missing package.

[foosel]

This should now be fixed as of the above commit. The code now also checks if a line starting with Error: is included in the output, and only if not considers a reinstall is needed.

This is a bit of a weird thing. I've so far only seen pip output neither Successfully installed nor Could not install lines in case of "look, it's already installed". So the logic so far was, check for a result line, if no result line is present and we have a "Requirement already satisifed" in there, assume that's about the plugin we were just asked to install and reinstall it.

In the log up there however, there is no result line either. I would have expected a final "Could not install OctoEverywhere" line due to the failure in installing the dependency, but that never happened. So that's a bit weird and unexpected. I really wish there was a well defined programmatic pip API, but there isn't and we are stuck with trying to parse output. Anyhow, I've added this as a test case to the unit test suite.

[QuinnDamerell]

That amazing! Thanks! I agree that it’s really strange that pip doesn’t have an api. I have seen a lot of PY projects that talk about how they would love to interface directly with pip, but have to rely on the cli.

Anyways, thanks again!