Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse charm URLs consistantly for local charms #974

Merged

Conversation

jack-w-shaw
Copy link
Member

@jack-w-shaw jack-w-shaw commented Oct 25, 2023

Port forward #969

Parse charm URLs consistantly for local charms

Local charm URLs are confused in pylibjuju because often a charm
url-like string is passed into deploy, to explicitly specify a local
charm. These 'urls' were of the form 'local:/path/to/charm'

Local URLs were parsed accordingly.

However, the above is in no sense a url really so should be treated as
such. This also means that pylibjuju is unable to parse real local charm
urls returned to the client from the server when, say, uploading a local
charm.

Drop support for local charms like this. Instead parse local charm urls the
same way we parse charmhub charm url.

This has the side-effect, however, that we can no longer treat all
objects to deploy as URLs. Now, they are either URLs (ch or cs charms)
or paths or paths prepended with 'local:' (local charms)

As such, in deploy code and refresh code, we often need to branch on
is_local_charm. This is to ensure we don't attempt to parse paths as
URLs. Everywhere we URL.parse user input, we ensure the entity is not a
path

This resolves #961

QA Steps

tox -e unit -- tests/unit/test_url.py
tox -e integration -- tests/integration/test_model.py::test_deploy_local_bundle_dir
tox -e integration -- tests/integration/test_model.py::test_deploy_local_bundle_file
tox -e integration -- tests/integration/test_model.py::test_deploy_bundle_local_charms
tox -e integration -- tests/integration/test_model.py::test_deploy_local_charm

All CI tests need to pass.

(In a python interpreter)

>>> from juju.url import URL
>>> u = URL.parse("local:focal/ubuntu-12")
>>> print(u.name)
ubuntu
>>> print(u.revision)
12
>>> URL.parse("local:/path/to/charm")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jack/juju/python-libjuju/juju/url.py", line 50, in parse
    c = parse_v2_url(u, s, default_store)
  File "/home/jack/juju/python-libjuju/juju/url.py", line 138, in parse_v2_url
    raise JujuError("charm or bundle URL {} malformed".format(s))
juju.errors.JujuError: charm or bundle URL local:/path/to/charm malformed

Ensure the example scripts still function

$ python examples/deploy_bundle.py
$ python examples/deploy_local_bundle_with_resources.py

Verify the following example script runs successfully:

from juju import jasyncio
from juju.model import Model


async def main():
    model = Model()
    print('Connecting to model')
    await model.connect()
    try:
        print('path="/home/jack/charms/ubuntu"')
        await depl(model, path="/home/jack/charms/ubuntu")
        print('switch="/home/jack/charms/ubuntu"')
        await depl(model, switch="/home/jack/charms/ubuntu")
        print('switch="local:/home/jack/charms/ubuntu"')
        await depl(model, switch="local:/home/jack/charms/ubuntu")
    finally:
        print('Disconnecting from model')
        await model.disconnect()

async def depl(model, **kwargs):
    try:
        app = await model.deploy("ubuntu")
        await model.block_until(lambda: all(u[0].workload_status == 'active' for u in app.units))
        await app.refresh(**kwargs)
    finally:
        await app.remove()
        await model.block_until(lambda: not len(model.applications))

if __name__ == '__main__':
    jasyncio.run(main())

@@ -541,7 +532,7 @@ async def resolve(self, url, architecture,
raise JujuError('revision and channel are mutually exclusive when deploying a bundle. Please choose one.')

if app_name is None:
app_name = url.name
app_name = charm_url.name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the charm_url is returned as a string and blowing up here, and needs to be fixed before further review. See https://github.com/juju/python-libjuju/actions/runs/6642890817/job/18048743466?pr=974#step:6:1553

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good spot. Thanks, this is a fairly easy fix, _resolve_charm is only called a few times, and when we do we don't assume it's a string mostly

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I've decided to change it to return a charm url

@jack-w-shaw jack-w-shaw force-pushed the JUJU-4779_parse_local_charms_properly-master branch from bac5900 to a953d07 Compare October 26, 2023 14:54
Copy link
Contributor

@cderici cderici left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI fail is unrelated, this can land 👍

@jack-w-shaw
Copy link
Member Author

/merge

@cderici
Copy link
Contributor

cderici commented Oct 26, 2023

Might need a rebase

Local charm URLs are confused in pylibjuju because often a charm
url-like string is passed into deploy, to explicitly specify a local
charm. These 'urls' were of the form 'local:/path/to/charm'

Local URLs were parsed accordingly.

However, the above is in no sense a url really so should be treated as
such. This also means that pylibjuju is unable to parse real local charm
urls returned to the client from the server when, say, uploading a local
charm.

Drop support for local charms like this. Instead parse local charm urls the
same way we parse charmhub charm url.

This has the side-effect, however, that we can no longer treat all
objects to deploy as URLs. Now, they are either URLs (ch or cs charms)
or paths or paths prepended with 'local:' (local charms)

As such, in deploy code and refresh code, we often need to branch on
is_local_charm. This is to ensure we don't attempt to parse paths as
URLs. Everywhere we URL.parse user input, we ensure the entity is not a
path
@jack-w-shaw jack-w-shaw force-pushed the JUJU-4779_parse_local_charms_properly-master branch from a953d07 to 0356f00 Compare October 26, 2023 16:24
@jack-w-shaw
Copy link
Member Author

/merge

@jujubot jujubot merged commit 03332e6 into juju:master Oct 26, 2023
7 of 8 checks passed
@cderici cderici mentioned this pull request Oct 26, 2023
jujubot added a commit that referenced this pull request Oct 26, 2023
#980

## What's Changed
* Repository Maintenance Improvements by @cderici in #922
* Stale bot to not bother feature requests by @cderici in #926
* Fix linter issues by @cderici in #928
* Fix docstring typo by @DanielArndt in #927
* Fix asyncio on README by @marceloneppel in #930
* Fix integration/test_application.test_action by @cderici in #932
* Update 3.2 facade clients by @cderici in #931
* [JUJU-4488] Add licence headers to source files on 3.x by @cderici in #934
* Update async tests to use builtin python suite by @DanielArndt in #935
* Pass correct charm url to series selector by @cderici in #942
* Green CI cleanup for python-libjuju by @cderici in #939
* Bring forward support for nested assumes expressions on 3x by @cderici in #943
* Release 3.2.2.0 notes by @cderici in #945
* Cleanup release process for 3.x by @cderici in #946
* Use new DeployFromRepository endpoint for deploy by @cderici in #949
* Handle pending upload resources deployfromrepository by @cderici in #953
* Optimize connection teardown by @cderici in #952
* Use `log.warning` instead of the deprecated `warn` by @sed-i in #954
* Find controller name by endpoint on 3.x track by @cderici in #966
* Optimize & fix unit removal by @cderici in #967
* Allow switch kwarg in refresh to switch to local charms by @jack-w-shaw in #971
* Parse charm URLs consistantly for local charms by @jack-w-shaw in #974
* Juju config directory location fix on 3.x by @cderici in #976
* [JUJU-4779] Ensure valid charm origin for local charm switches by @jack-w-shaw in #978
* Application refresh with resources on 3.x by @cderici in #973

#### Notes & Discussion

JUJU-4851

[JUJU-4488]: https://warthogs.atlassian.net/browse/JUJU-4488?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-4779]: https://warthogs.atlassian.net/browse/JUJU-4779?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Local charm urls are not parsed correctly
3 participants