Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force importing yaml from the system dist-packages #2448

Merged
merged 2 commits into from Mar 3, 2023

Conversation

renanrodrigo
Copy link
Member

When users install a new pyyaml using pip, it will end up in pythonpath with a higher priority.
This PR introduces some logic to import yaml only directly from the system dist-packages.
This fixes two bugs:
LP: #2007234 : If yaml is not present, then a UserFacingError is thrown - this can only happen if the python3-yaml deb is not present or was directly modified, and as it is a dependency, it does not make sense to support anything like this.
LP: #2007241 : The system yaml won't have a version which is incompatible with the python version, so the safe_load bug is prevented.

Test Steps

Added unit tests and integration tests to verify the fix.

Checklist

  • I have updated or added any unit tests accordingly
  • I have updated or added any integration tests accordingly
  • I have updated or added any documentation accordingly

Does this PR require extra reviews?

  • Yes
  • No
  • I hope not

"missing-yaml-module",
"""\
Couldn't import the YAML module from /usr/lib/python3/dist-packages.
Make sure the 'python3-yaml' package is installed correctly.""",
Copy link
Contributor

@CalvoM CalvoM Feb 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be unnecessary to add the command to install the python3-yaml package i.e. sudo apt install python3-yaml?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes unecessary - it is a dependency of ubuntu-advantage-tools - if this is not installed correctly then they messed it up, and a simple apt install may not be sufficient...

Copy link
Collaborator

@orndorffgrant orndorffgrant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow thanks for figuring all of this out!
Looks good, but I can't get the new integration test to fail with e.g. install_from=archive - ideally the test would fail on the current version and pass on this PR though

@renanrodrigo
Copy link
Member Author

@orndorffgrant hmmm well I'll investigate and see why it does not break... Thanks for the heads up

@renanrodrigo renanrodrigo force-pushed the yaml-very-safe-load branch 2 times, most recently from 873418d to 4d786ae Compare February 24, 2023 19:13
@renanrodrigo
Copy link
Member Author

renanrodrigo commented Feb 24, 2023

@orndorffgrant surprisingly, Python itself ensures retro compatibility and it does not fail until 3.10!
SO the bug ocurrences are vast on Jammy/Kinetic because there is where it matters.
I edited the test to run on Jammy+, where you can see the failure and fix flow.

Copy link
Collaborator

@orndorffgrant orndorffgrant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update! LGTM

uaclient/yaml.py Outdated Show resolved Hide resolved
Signed-off-by: Renan Rodrigo <renanrodrigo@canonical.com>
Check for the existence of yaml in the system libs before going the
Return an error when no module is found.

Using the system yaml guarantees that safe_load and safe_dump works.

LP: #2007234
LP: #2007241

Signed-off-by: Renan Rodrigo <renanrodrigo@canonical.com>
@renanrodrigo
Copy link
Member Author

CI failures unrelated

@renanrodrigo renanrodrigo merged commit 19d408e into main Mar 3, 2023
@renanrodrigo renanrodrigo deleted the yaml-very-safe-load branch March 3, 2023 01:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants