Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin Jinja2 and MarkupSafe to fix issues building on Focal #211

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

hloeung
Copy link
Contributor

@hloeung hloeung commented Oct 19, 2022

See #210

wheelhouse.txt Outdated Show resolved Hide resolved
wheelhouse.txt Outdated Show resolved Hide resolved
@hloeung
Copy link
Contributor Author

hloeung commented Oct 19, 2022

Built charm on Jammy - https://paste.ubuntu.com/p/tP9X63bdWX/
Built charm on Focal - https://paste.ubuntu.com/p/GByT7j5Fcn/
Built charm on Bionic - https://paste.ubuntu.com/p/cs9J5HdSrp/
Built charm on Xenial - https://paste.ubuntu.com/p/SrGPv8tRjH/

@hloeung hloeung marked this pull request as draft October 20, 2022 04:24
@hloeung hloeung marked this pull request as ready for review October 20, 2022 04:25
@hloeung hloeung changed the title Pin Jinja2, setuptools, and setuptools-scm to fix issues with Focal Pin Jinja2, MarkupSafe, setuptools, and setuptools-scm to fix issues with Focal Oct 20, 2022
@hloeung
Copy link
Contributor Author

hloeung commented Oct 28, 2022

@ajkavanagh , free to take another look at this PR?

Copy link
Contributor

@ajkavanagh ajkavanagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for your work on the patch. I am a bit concerned with the re-pinning of various modules as this will change the builds on existing stable charms (should they be rebuilt), which would be a significant change in behaviour of the charm. e.g. the unpins of setuptools and setuptool-scm seems particularly problematic.

I wonder if, tactically, you just override these in your charm's wheelhouse.txt?

tests/charm-minimal/wheelhouse.txt Outdated Show resolved Hide resolved
wheelhouse.txt Outdated Show resolved Hide resolved
@ajkavanagh
Copy link
Contributor

@hloeung I think this PR may be closed? I had a chat with @fnordahl and I think the issues you were seeing were to to with a juju bug?

@hloeung
Copy link
Contributor Author

hloeung commented Oct 31, 2022

@hloeung I think this PR may be closed? I had a chat with @fnordahl and I think the issues you were seeing were to to with a juju bug?

@ajkavanagh different problem actually. The one @fnordahl & @darkalia saw and ran into was the Juju bug where it was selecting the charm with the highest revno and not one that was built for the specific series that they wanted.

@fnordahl
Copy link
Member

fnordahl commented Nov 1, 2022

Just to be clear, I've seen no issues fixed by this proposed change, and I repeat my stance that fine grained pinning does not belong in layer-basic. The wide range of different approaches to reactive charming within the teams we know about (not to mention those we don't know about) makes this impractical.

If you have specific pinning needs, doing so in individual charms or a different layer shared among a fleet of charms would be the preferred approach.

@hloeung
Copy link
Contributor Author

hloeung commented Nov 1, 2022

The first issue is that charms fail to build on Focal. Here's quick steps I just came up with to reproduce:

From a clean Focal VM or LXD container:

sudo snap install charmcraft --classic
sudo snap install charm --classic
sudo apt-get install python3-venv
charm create haw-test   # using charm here instead of charmcraft init to create a bare reactive charm instead of operator.
cd haw-test

Now edit metadata.yaml removing these lines:

provides:
  provides-relation:
    interface: interface-name
requires:
  requires-relation:
    interface: interface-name
peers:
  peer-relation:
    interface: interface-name

Then create a charmcraft.yaml file with:

type: charm
bases:
  - build-on:
    - name: ubuntu
      channel: "20.04"
    run-on:
    - name: ubuntu
      channel: "20.04"
parts:
  charm:
    source: "."
    plugin: "reactive"
    build-snaps: [charm]

Now try build it with:

charmcraft pack --destructive-mode

Unfortunately, it would fail with ModuleNotFoundError: No module named 'markupsafe'. Snippet of output being:

2022-11-01 00:41:36.959 :: build: Processing layer: layer:basic
2022-11-01 00:41:37.097 :: build: Processing layer: haw-test (from .)
2022-11-01 00:41:53.874 :: build: Ignoring pip: markers 'python_version < "3.8"' don't match your environment
2022-11-01 00:41:53.874 :: Ignoring Jinja2: markers 'python_version >= "3.0" and python_version <= "3.4"' don't match your environment
2022-11-01 00:41:53.874 :: Ignoring Jinja2: markers 'python_version == "2.7" or python_version == "3.5"' don't match your environment
2022-11-01 00:41:53.874 :: Ignoring PyYAML: markers 'python_version >= "3.0" and python_version <= "3.4"' don't match your environment
2022-11-01 00:41:53.874 :: Ignoring MarkupSafe: markers 'python_version < "3.6"' don't match your environment
2022-11-01 00:41:53.874 :: Ignoring MarkupSafe: markers 'python_version == "3.6"' don't match your environment
2022-11-01 00:41:53.874 :: Ignoring setuptools: markers 'python_version < "3.8"' don't match your environment
2022-11-01 00:41:53.874 :: Ignoring setuptools-scm: markers 'python_version < "3.8"' don't match your environment
2022-11-01 00:41:53.874 :: Ignoring wheel: markers 'python_version < "3.8"' don't match your environment
2022-11-01 00:41:53.874 :: Collecting pip<22.1
2022-11-01 00:41:53.874 ::   Downloading pip-22.0.4.tar.gz (2.1 MB)
2022-11-01 00:41:53.874 ::   Installing build dependencies: started
2022-11-01 00:41:53.874 ::   Installing build dependencies: finished with status 'done'
2022-11-01 00:41:53.874 ::   Getting requirements to build wheel: started
2022-11-01 00:41:53.874 ::   Getting requirements to build wheel: finished with status 'done'
2022-11-01 00:41:53.874 ::     Preparing wheel metadata: started
2022-11-01 00:41:53.874 ::     Preparing wheel metadata: finished with status 'done'
2022-11-01 00:41:53.874 ::   Saved /home/ubuntu/snap/charmcraft/common/tmpkzk9z42m/pip-22.0.4.tar.gz
2022-11-01 00:41:53.874 ::   Downloading Jinja2-3.1.2.tar.gz (268 kB)
...
2022-11-01 00:41:53.875 ::       File "/home/ubuntu/snap/charmcraft/common/pip-download-4j6lk0r8/Jinja2/src/jinja2/environment.py", line 14, in <module>
2022-11-01 00:41:53.875 ::         from markupsafe import Markup
2022-11-01 00:41:53.875 ::     ModuleNotFoundError: No module named 'markupsafe'

So we'll want to pin Jinja2, create the wheelhouse.txt file with:

Jinja2==2.11;python_version >= '3.6' and python_version <= '3.8' # py38, focal
Jinja2;python_version > '3.8'

Re-try build. That should build successfully with the bundled wheelhouse Python packages:

unzip -l haw-test_ubuntu-20.04-amd64.charm | grep wheelhouse | sort -k 4
     1682  2022-11-01 00:45   wheelhouse.txt
   257890  2022-11-01 00:46   wheelhouse/Jinja2-2.11.0.tar.gz
    18668  2022-11-01 00:45   wheelhouse/MarkupSafe-2.1.1.tar.gz
   269377  2022-11-01 00:45   wheelhouse/PyYAML-5.3.1.tar.gz
   610181  2022-11-01 00:46   wheelhouse/charmhelpers-1.2.1.tar.gz
   356307  2022-11-01 00:46   wheelhouse/charms.reactive-1.5.1.tar.gz
    40330  2022-11-01 00:46   wheelhouse/flit_core-3.7.1.tar.gz
  1622835  2022-11-01 00:46   wheelhouse/netaddr-0.7.19.tar.gz
    84848  2022-11-01 00:46   wheelhouse/packaging-21.3.tar.gz
   127505  2022-11-01 00:46   wheelhouse/pbr-5.11.0.tar.gz
  2090742  2022-11-01 00:45   wheelhouse/pip-22.0.4.tar.gz
    20466  2022-11-01 00:46   wheelhouse/pyaml-21.10.1.tar.gz
  1999906  2022-11-01 00:46   wheelhouse/pyparsing-3.0.9.tar.gz
  2512885  2022-11-01 00:45   wheelhouse/setuptools-62.1.0.tar.gz
    61305  2022-11-01 00:46   wheelhouse/setuptools_scm-6.4.2.tar.gz
    15164  2022-11-01 00:46   wheelhouse/tomli-2.0.1.tar.gz
    66376  2022-11-01 00:46   wheelhouse/wheel-0.37.1.tar.gz

The other issue is that some of our charms use include_system_packages: true. This wasn't a problem until fb767dc. Here's quick steps to reproduce:

Edit layer.yaml adding:

options:
  basic:
    include_system_packages: true

Build charm:

charmcraft pack --destructive-mode

That should build successfully with the bundled wheelhouse Python packages:

$ unzip -l haw-test_ubuntu-20.04-amd64.charm | grep wheelhouse | sort -k 4
     1682  2022-11-01 01:29   wheelhouse.txt
   257890  2022-11-01 01:30   wheelhouse/Jinja2-2.11.0.tar.gz
    18668  2022-11-01 01:29   wheelhouse/MarkupSafe-2.1.1.tar.gz
   269377  2022-11-01 01:29   wheelhouse/PyYAML-5.3.1.tar.gz
   610181  2022-11-01 01:30   wheelhouse/charmhelpers-1.2.1.tar.gz
   356307  2022-11-01 01:30   wheelhouse/charms.reactive-1.5.1.tar.gz
    40330  2022-11-01 01:30   wheelhouse/flit_core-3.7.1.tar.gz
  1622835  2022-11-01 01:30   wheelhouse/netaddr-0.7.19.tar.gz
    84848  2022-11-01 01:30   wheelhouse/packaging-21.3.tar.gz
   127505  2022-11-01 01:30   wheelhouse/pbr-5.11.0.tar.gz
  2090742  2022-11-01 01:29   wheelhouse/pip-22.0.4.tar.gz
    20466  2022-11-01 01:30   wheelhouse/pyaml-21.10.1.tar.gz
  1999906  2022-11-01 01:30   wheelhouse/pyparsing-3.0.9.tar.gz
  2512885  2022-11-01 01:29   wheelhouse/setuptools-62.1.0.tar.gz
    61305  2022-11-01 01:29   wheelhouse/setuptools_scm-6.4.2.tar.gz
    15164  2022-11-01 01:30   wheelhouse/tomli-2.0.1.tar.gz
    66376  2022-11-01 01:30   wheelhouse/wheel-0.37.1.tar.gz

Now using that newly built charm, try to deploy to Focal:

juju deploy ./haw-test_ubuntu-20.04-amd64.charm --series focal haw-test-focal-dd

It fails with:

2022-11-01 01:43:07 DEBUG unit.haw-test-focal-dd/0.install logger.go:60 Setting up virtualenv (20.0.17-1ubuntu0.4) ...
2022-11-01 01:43:08 DEBUG unit.haw-test-focal-dd/0.install logger.go:60 Processing triggers for man-db (2.9.1-1) ...
2022-11-01 01:43:11 DEBUG unit.haw-test-focal-dd/0.install logger.go:60 created virtual environment CPython3.8.10.final.0-64 in 451ms
2022-11-01 01:43:11 DEBUG unit.haw-test-focal-dd/0.install logger.go:60   creator CPython3Posix(dest=/var/lib/juju/agents/unit-haw-test-focal-dd-0/.venv, clear=False, global=True)
2022-11-01 01:43:11 DEBUG unit.haw-test-focal-dd/0.install logger.go:60   seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, pkg_resources=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1.debian.1)
2022-11-01 01:43:11 DEBUG unit.haw-test-focal-dd/0.install logger.go:60   activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
2022-11-01 01:43:14 DEBUG unit.haw-test-focal-dd/0.install logger.go:60 Looking in links: wheelhouse
2022-11-01 01:43:14 DEBUG unit.haw-test-focal-dd/0.install logger.go:60 Processing ./wheelhouse/pip-22.0.4.tar.gz
2022-11-01 01:43:15 DEBUG unit.haw-test-focal-dd/0.install logger.go:60   Installing build dependencies: started
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60   ERROR: Command errored out with exit status 1:
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60    command: /var/lib/juju/agents/unit-haw-test-focal-dd-0/.venv/bin/python /var/lib/juju/agents/unit-haw-test-focal-dd-0/.venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-odg07h6m/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links wheelhouse -- setuptools wheel
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60        cwd: None
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60   Complete output (40 lines):
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60   Looking in links: wheelhouse
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60   Processing ./wheelhouse/setuptools-62.1.0.tar.gz
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60     Getting requirements to build wheel: started
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60     Getting requirements to build wheel: finished with status 'error'
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60     ERROR: Command errored out with exit status 1:
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60      command: /var/lib/juju/agents/unit-haw-test-focal-dd-0/.venv/bin/python /tmp/tmpq0di12ca get_requires_for_build_wheel /tmp/tmp39w94f_4
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60          cwd: /tmp/pip-install-c8kb4gvs/setuptools
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60     Complete output (30 lines):
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60     Traceback (most recent call last):
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/tmpq0di12ca", line 280, in <module>
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         main()
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/tmpq0di12ca", line 263, in main
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         json_out['return_val'] = hook(**hook_input['kwargs'])
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/tmpq0di12ca", line 114, in get_requires_for_build_wheel
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         return hook(config_settings)
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/pip-install-c8kb4gvs/setuptools/setuptools/build_meta.py", line 177, in get_requires_for_build_wheel
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         return self._get_build_requires(
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/pip-install-c8kb4gvs/setuptools/setuptools/build_meta.py", line 159, in _get_build_requires
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         self.run_setup()
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/pip-install-c8kb4gvs/setuptools/setuptools/build_meta.py", line 174, in run_setup
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         exec(compile(code, __file__, 'exec'), locals())
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "setup.py", line 87, in <module>
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         dist = setuptools.setup(**setup_params)
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/pip-install-c8kb4gvs/setuptools/setuptools/__init__.py", line 87, in setup
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         return distutils.core.setup(**attrs)
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/pip-install-c8kb4gvs/setuptools/setuptools/_distutils/core.py", line 109, in setup
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         _setup_distribution = dist = klass(attrs)
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/pip-install-c8kb4gvs/setuptools/setuptools/dist.py", line 462, in __init__
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         _Distribution.__init__(
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/pip-install-c8kb4gvs/setuptools/setuptools/_distutils/dist.py", line 293, in __init__
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         self.finalize_options()
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/pip-install-c8kb4gvs/setuptools/setuptools/dist.py", line 885, in finalize_options
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         for ep in sorted(loaded, key=by_order):
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/pip-install-c8kb4gvs/setuptools/setuptools/dist.py", line 884, in <lambda>
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         loaded = map(lambda e: e.load(), filtered)
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60       File "/tmp/pip-install-c8kb4gvs/setuptools/setuptools/_vendor/importlib_metadata/__init__.py", line 196, in load
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60         return functools.reduce(getattr, attrs, module)
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60     AttributeError: type object 'Distribution' has no attribute '_finalize_feature_opts'
2022-11-01 01:43:18 WARNING unit.haw-test-focal-dd/0.install logger.go:60     ----------------------------------------
2022-11-01 01:43:31 DEBUG unit.haw-test-focal-dd/0.install logger.go:60   Installing build dependencies: finished with status 'error'
2022-11-01 01:43:31 WARNING unit.haw-test-focal-dd/0.install logger.go:60 ERROR: Command errored out with exit status 1: /var/lib/juju/agents/unit-haw-test-focal-dd-0/.venv/bin/python /var/lib/juju/agents/unit-haw-test-focal-dd-0/.venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-ep8vjrs7/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links wheelhouse -- setuptools wheel Check the logs for full command output.
2022-11-01 01:43:31 WARNING unit.haw-test-focal-dd/0.install logger.go:60 Traceback (most recent call last):
2022-11-01 01:43:31 WARNING unit.haw-test-focal-dd/0.install logger.go:60   File "/var/lib/juju/agents/unit-haw-test-focal-dd-0/charm/hooks/install", line 8, in <module>
2022-11-01 01:43:31 WARNING unit.haw-test-focal-dd/0.install logger.go:60     basic.bootstrap_charm_deps()
2022-11-01 01:43:31 WARNING unit.haw-test-focal-dd/0.install logger.go:60   File "/var/lib/juju/agents/unit-haw-test-focal-dd-0/charm/lib/charms/layer/basic.py", line 202, in bootstrap_charm_deps
2022-11-01 01:43:31 WARNING unit.haw-test-focal-dd/0.install logger.go:60     _update_if_newer(pip, pre_install_pkgs)
2022-11-01 01:43:31 WARNING unit.haw-test-focal-dd/0.install logger.go:60   File "/var/lib/juju/agents/unit-haw-test-focal-dd-0/charm/lib/charms/layer/basic.py", line 366, in _update_if_newer
2022-11-01 01:43:31 WARNING unit.haw-test-focal-dd/0.install logger.go:60     check_call([pip, 'install', '-U', '--no-index', '-f', 'wheelhouse',
2022-11-01 01:43:31 WARNING unit.haw-test-focal-dd/0.install logger.go:60   File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
2022-11-01 01:43:31 WARNING unit.haw-test-focal-dd/0.install logger.go:60     raise CalledProcessError(retcode, cmd)
2022-11-01 01:43:31 WARNING unit.haw-test-focal-dd/0.install logger.go:60 subprocess.CalledProcessError: Command '['/var/lib/juju/agents/unit-haw-test-focal-dd-0/.venv/bin/pip', 'install', '-U', '--no-index', '-f', 'wheelhouse', 'pip']' returned non-zero exit status 1.
2022-11-01 01:43:31 ERROR juju.worker.uniter.operation runhook.go:140 hook "install" (via explicit, bespoke hook script) failed: exit status 1

@hloeung
Copy link
Contributor Author

hloeung commented Nov 1, 2022

@fnordahl I think you're right, I think going forward, for charms using include_system_packages: true we'll have to pin setuptools & setuptools-scm there within the charm. But can we land the Jinja2 pinning so we can at least build the charms using Focal (Launchpad per-series builds)?

@hloeung hloeung changed the title Pin Jinja2, MarkupSafe, setuptools, and setuptools-scm to fix issues with Focal Pin Jinja2 and MarkupSafe to fix issues building on Focal Nov 1, 2022
@hloeung
Copy link
Contributor Author

hloeung commented Nov 1, 2022

@ajkavanagh, @fnordahl, okay PR updated to just resolve the build issues on/with Focal.

Copy link
Contributor

@ajkavanagh ajkavanagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I'm generally happy with this as it's fairly minimal and does allow a minimal charm to have sensible defaults for Jinja2/markupsafe on Focal. As @fnordahl has some key thoughts on this, I'd also like his +1 on this. Thanks again for your work on this.

@fnordahl
Copy link
Member

fnordahl commented Nov 6, 2022

Happy to see a more focused proposal.

Not going to block this change, but here are my further comments:

  1. The root of the Jinja2 build issue on focal is a problem with the setuptools version/package as shipped with Focal (ref No control over virtualenv used to build wheels juju/charm-tools#646). So the proper fix for this issue is to extend charm-tools to allow controlling the virtualenv used for downloading/building wheels, or perhaps just upgrading the Python packages in the virtualenv (ref --upgrade-deps in https://docs.python.org/3/library/venv.html).
  2. I'm not thrilled by the use of == pins here as previously stated strict pinning belongs in individual charm build.lock or other methods. I.e. what do you do if a upstream Jinja 2.x security fix is released?

@ajkavanagh
Copy link
Contributor

2. I'm not thrilled by the use of == pins here as previously stated strict pinning belongs in individual charm build.lock or other methods. I.e. what do you do if a upstream Jinja 2.x security fix is released?

I think this is a valid critique and should be addressed. @hloeung could you update the PR to use "<" pins which would allow security fixes, please?

With charms.reactive, charm-tools, layer-basic and the rest of the charms.reactive charm building system being in maintenance mode effectively, we don't want to do invasive or breaking changes in these tools. This means being very conservative about changes and ensuring that existing work flows continue to work (as best we can) but recognising that there is a better way of building charms now.

wheelhouse.txt Outdated Show resolved Hide resolved
@hloeung
Copy link
Contributor Author

hloeung commented Nov 7, 2022

Updated the PR to use "<" pins as advised.

@hloeung hloeung marked this pull request as draft December 15, 2022 06:54
@hloeung hloeung marked this pull request as ready for review December 15, 2022 06:54
@hloeung
Copy link
Contributor Author

hloeung commented Dec 15, 2022

Trying to get this moving along... @ajkavanagh ?

@fnordahl
Copy link
Member

@hloeung juju/charm-tools#650 has merged now, try using charm snap from edge channel and pass the --upgrade-buildvenv-core-deps argument, i.e.:

type: charm

parts:
  charm:
    source: src/
    plugin: reactive
    reactive-charm-build-arguments:
      - --upgrade-buildvenv-core-deps
    build-snaps:
      - charm/3.x/edge

The new version will of course be promoted to stable, but most likely not until next year.

Copy link
Contributor

@ajkavanagh ajkavanagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm approving this as Jinja is pinned by version and MarkupSafe is a dependency of that; either we unpin everything or we continue to hold things per python version as this review is doing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants