Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add extra dependencies to docker container #1073

Merged
merged 9 commits into from
Dec 13, 2021

Conversation

nniehoff
Copy link
Contributor

Fixes: #1068

This PR adds the optional dependencies described in the docs to poetry as optional dependencies and extras to configure for Nautobot. A custom extra all will combine all of the optional dependencies. The docker container is updated to include the all extra to bundle all optional components in the docker container.

pyproject.toml Outdated Show resolved Hide resolved
pyproject.toml Outdated Show resolved Hide resolved
nautobot/docs/installation/nautobot.md Outdated Show resolved Hide resolved
nautobot/docs/configuration/authentication/sso.md Outdated Show resolved Hide resolved
@@ -6,13 +6,10 @@ This guide explains how to implement LDAP authentication using an external serve

### Install System Packages

!!! danger
FIXME(jathan): With `wheel` packages asserted, let's make doubly sure these development libraries even need to be installed anymore.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were you able to confirm that these libraries are still needed in the current installation? I see you've added them to the Dockerfile - were there failures without adding them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes there definitely were

Copy link
Contributor

@u1735067 u1735067 Nov 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I may add some inputs: on Alpine, while openldap-dev is required to build the package, only libldap seems to be required at runtime (and outside of ldap, postgresql-dev zlib-dev jpeg-dev libffi-dev at build time, libpq zlib libjpeg libffi at run time)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@u1735067 thanks for the input, unfortunately, we don't support alpine. I'd be curious if you are able to test using the Debian containers for runtime dependencies.


```no-highlight
$ echo django-storages >> $NAUTOBOT_ROOT/local_requirements.txt
$ echo "nautobot[storages]" >> $NAUTOBOT_ROOT/local_requirements.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels weird to me to be putting nautobot into local_requirements.txt in any way shape or form. Maybe this change should be reverted? Compare to similar verbiage in authentication/ldap.md that you haven't updated in this way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but I think there is a fundamental difference between pip install django-storages and pip install nautobot[remote_storage] I believe the latter will follow the version requirements specified by the nautobot package which are driven by the pyproject.toml when the package is built. Installing django-storages could potentially have version conflicts.

That said it's still odd adding nautobot in an additional requirements file. This might bring the larger issue up in that local_requirements.txt could potentially drive dependency issues with Nautobot itself. Should we recommend some kind of poetry environment be setup locally, I'm not sure how that would look exactly...

@glennmatthews
Copy link
Contributor

The black update brings in required format changes to one test file - you can see the way I addressed them in #1069 if you want to just steal the changes from there.

@glennmatthews glennmatthews added this to the v1.2.0 milestone Nov 18, 2021
docker/Dockerfile Show resolved Hide resolved
@@ -104,13 +108,20 @@ pycryptodome = "~3.10.1"
pyuwsgi = "~2.0.19.1.post0"
# YAML parsing and rendering
PyYAML = "~5.4.1"
# Social authentication core
social-auth-core = {extras = ["openidconnect", "saml"], version = "~4.1.0", optional = true}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does marking social-auth-core as optional actually make sense given that social-auth-app-django is non-optional and it in turn requires social-auth-core? Is this just for the optional openidconnect and saml extras? I ask because the poetry.lock here looks suspicious (https://github.com/nautobot/nautobot/pull/1073/files#diff-f53a023eedfa3fbf2925ec7dc76eecdc954ea94b7e47065393dbad519613dc89R1731) - note the empty dependencies for openid and saml:

[extras]
all = ["mysqlclient", "django-auth-ldap", "django-storages"]
ldap = ["django-auth-ldap"]
mysql = ["mysqlclient"]
openid = []
remote_storage = ["django-storages"]
saml = []

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for finding this Glenn! it looks like we need to wait for python-poetry/poetry#3913 to have the separate openid/saml extras, for now I'll combine them into a single sso

@glennmatthews
Copy link
Contributor

Fixes #888.

Copy link
Contributor

@glennmatthews glennmatthews left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not critical for this PR, but for sake of eventual consistency, we should give NAPALM a similar treatment.

@nniehoff
Copy link
Contributor Author

Not critical for this PR, but for sake of eventual consistency, we should give NAPALM a similar treatment.

Oops sorry I missed it, I think it's definitely in the spirit of this PR, I'll get it updated ASAP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Embed optional dependencies in the official Docker image (LDAP, SSO, storage, ..)
4 participants