From b6ad4cb55cdcce415a735a553781d00c16dc00b8 Mon Sep 17 00:00:00 2001 From: "John R. Lenton" Date: Thu, 26 Mar 2020 11:10:48 +0000 Subject: [PATCH 01/10] README: a significant rework This tries to massage the README into something more focused on the big picture, giving only a minimal introduction to the salient points, and linking off to other (TBD) resources for further information. --- README.md | 154 +++++++++++++++++++++++++----------------------------- 1 file changed, 72 insertions(+), 82 deletions(-) diff --git a/README.md b/README.md index 347bfc8ef..cfa80b377 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@ # The Operator Framework -The Operator Framework provides a simple, lightweight, and powerful way of encapsulating operational experience in code. +The Operator Framework provides a simple, lightweight, and powerful way of +writing juju charms, the best way to encapsulate operational experience in code. The framework will help you to: @@ -11,79 +12,60 @@ The framework will help you to: ## Getting Started -The following overall structure for your charm directory is recommended: +Charms written using the operator framework are just Python code. The intention +is for it to feel very natural for somebody used to coding in Python, and +reasonably easy to pick up for somebody whose might be a domain expert but +not necessarily a pythonista themselves. -``` -. -├── config.yaml -├── metadata.yaml -├── mod/ -├── lib/ -│ └── ops -> ../mod/operator/ops -├── src/ -│ └── charm.py -└── hooks/ - ├── install -> ../src/charm.py - └── start -> ../src/charm.py # for k8s charms per below -``` - -The `mod/` directory should contain the operator framework dependency as a git -submodule: +The dependencies of the operator framework are kept as minimal as possible; +currently that's Python 3.5 or greater, and `PyYAML` (both are included by +default in Ubuntu's cloud images from 16.04 on). -``` -git submodule add https://github.com/canonical/operator mod/operator -``` +If you're new to the world of juju and charms, you should probably dive into our +[tutorial](/TBD). -Then symlink from the git submodule for the operator framework into the `lib/` -directory of your charm so it can be imported at run time: +If you know about juju, and have written charms that didn't use the operator +framework (be it with reactive or without), we have an [introduction to the +operator framework](/TBD) just for you. -``` -ln -s ../mod/operator/ops lib/ops -``` +If you've gone through the above already and just want a refresher, or are +really impatient and just want to dive in, feel free to carry on down. -Other dependencies included as git submodules can be added in the `mod/` -directory and symlinked into `lib/` as well. +## An Unrelenting Introduction -You can sync subsequent changes from the framework and other submodule -dependencies by running: +Operator framework charms are just Python code. The entry point to your charm is +a particular Python file. It could be anything that makes sense to your project, +but let's assume this is `src/charm.py`. This file must be executable (and it +must have the appropriate shebang line). -``` -git submodule update -``` +This file must be symlinked from `hooks/install` (or `hooks/start` if it's a +charm targeting k8s on juju < 3.8). -Those cloning and checking out the source for your charm for the first time -will need to run: +You also need the usual `metadata.yaml` and `config.yaml` files, and any Python +dependencies (maybe using some kind of virtualenv). In other words, your project +might look like this: ``` -git submodule update --init +. +├── config.yaml +├── metadata.yaml +├── env/ +│ └── # ... your Python dependencies, including the operator framework itself +├── src/ +│ └── charm.py +└── hooks/ + └── install -> ../src/charm.py ``` -Your `src/charm.py` is the entry point for your charm logic. It should be set -to executable and use Python 3.6 or greater. At a minimum, it needs to define -a subclass of `CharmBase` and pass that into the framework's `main` function: +`src/charm.py` here is the entry point to your charm code. At a minimum, it +needs to define a subclass of `CharmBase` and pass that into the framework's +`main` function: ```python -import sys -sys.path.append('lib') # noqa: E402 - +#!../env/bin/python3 from ops.charm import CharmBase from ops.main import main - -class MyCharm(CharmBase): - pass - - -if __name__ == "__main__": - main(MyCharm) -``` - -This charm does nothing, because the `MyCharm` class passed to the operator -framework's `main` function is empty. Functionality can be added to the charm -by instructing it to observe particular Juju events when the `MyCharm` object -is initialized. For example, - -```python class MyCharm(CharmBase): def __init__(self, *args): super().__init__(*args) @@ -91,43 +73,51 @@ class MyCharm(CharmBase): def on_start(self, event): # Handle the start event here. + +if __name__ == "__main__": + main(MyCharm) ``` -Every standard event in Juju may be observed that way, and you can also easily -define your own events in your custom types. +That should be enough for you to be able to run -> The second argument to `observe` can be either the handler as a bound -> method, or the observer itself if the handler is a method of the observer -> that follows the conventional naming pattern. That is, in this case, we -> could have called just `self.framework.observe(self.on.start, self)`. +``` +$ juju deploy . +``` -The `hooks/` directory must contain a symlink to your `src/charm.py` entry -point so that Juju can call it. You only need to set up the `hooks/install` link -(`hooks/start` for K8s charms, until [lp#1854635](https://bugs.launchpad.net/juju/+bug/1854635) -is resolved), and the framework will create all others at runtime. +Happy charming! -Once your charm is ready, upload it to the charm store and deploy it as -normal with: +## Testing your charms -``` -# Replace ${CHARM} with the name of the charm. -charm push . cs:~${USER}/${CHARM} -# Replace ${VERSION} with the version created by `charm push`. -charm release cs:~${USER}/${CHARM}-${VERSION} -charm grant cs:~${USER}/${CHARM}-${VERSION} everyone -# And now deploy your charm. -juju deploy cs:~${USER}/$CHARM +The operator framework provides a testing harness, so that you can test that +your charm does the right thing when presented with different scenarios, without +having to have a full deployment to do so. `pydoc3 ops.testing` has the details +for that, including this example: + +```python +harness = Harness(MyCharm) +# Do initial setup here +relation_id = harness.add_relation('db', 'postgresql') +# Now instantiate the charm to see events as the model changes +harness.begin() +harness.add_relation_unit(relation_id, 'postgresql/0', remote_unit_data={'key': 'val'}) +# Check that charm has properly handled the relation_joined event for postgresql/0 +self.assertEqual(harness.charm. ...) ``` -Alternatively, to deploy directly from local disk, run: +# Talk to us -``` -juju deploy . -``` +If you need help, have ideas, or would just like to chat with us, reach out on +IRC: we're in [#smooth-operator](irc://chat.freenode.net/%23smooth-operator) on +freenode (or try the [webchat](https://webchat.freenode.net/#smooth-operator)). + +We also pay attention to juju's [discourse](https://discourse.jujucharms.com/), +but currently we don't actively post there; most discussion at this stage is on +IRC. # Operator Framework development -If you want to work in the framework *itself* you will need the following depenencies installed in your system: +If you want to work in the framework *itself* you will need the following +depenencies installed in your system: - Python >= 3.5 - PyYAML From 3ba21fd24ce868f5c9f32dedcbc8a4fe3fdc4f9a Mon Sep 17 00:00:00 2001 From: "John R. Lenton" Date: Fri, 27 Mar 2020 14:42:16 +0000 Subject: [PATCH 02/10] README: fix typo (thanks @facundobatista) --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index cfa80b377..99b96d3b4 100644 --- a/README.md +++ b/README.md @@ -117,7 +117,7 @@ IRC. # Operator Framework development If you want to work in the framework *itself* you will need the following -depenencies installed in your system: +dependencies installed in your system: - Python >= 3.5 - PyYAML From 06d242ff31b4a5cc6e7e829f9e8f99c6974def86 Mon Sep 17 00:00:00 2001 From: "John R. Lenton" Date: Fri, 27 Mar 2020 14:43:12 +0000 Subject: [PATCH 03/10] tbd_examples: started working through the examples. These won't live here! But it's a good place to stage them for now. --- tbd_examples.md | 130 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 130 insertions(+) create mode 100644 tbd_examples.md diff --git a/tbd_examples.md b/tbd_examples.md new file mode 100644 index 000000000..008f6ed44 --- /dev/null +++ b/tbd_examples.md @@ -0,0 +1,130 @@ +# Examples of common operations in a charm + +## Configuration + +Configuration can be accesed on the `config` object of the `model` +property on the charm object. On first access it will run `juju +config-get`, and cache the results transparently. + +
Example + +```python +from ops.charm import CharmBase + +class MyCharm(CharmBase): + # ... + + def on_some_event(self, event): + if self.model.config['foo'] == 'xyzzy': + # yak yak yak +``` +
+ +### Handling changes to configuration + +Configuration changes are reported via a `ConfigChangedEvent`, as +expected. + +
Example + +```python +from ops.charm import CharmBase + +class MyCharm(CharmBase): + # ... + + def on_config_changed(self, event): + # self.model.config will already have the new values +``` +
+ +### Validate configuration, blocking charm until fixed + +If the charm has a way to validate its config, it can do that and set +the charm's status to blocked if it fails. + +
Example + +```python +from ops.charm import CharmBase +from ops.model import BlockedStatus + +class MyCharm(CharmBase): + # ... + + def on_config_changed(self, event): + try: + # call my custom validator that raises a custom exception on error + self._validate_config() + except MyValidationError as e: + self.status = BlockedStatus("fix yo stuff: {}".format(e)) +``` +
+ +## Writing to juju logs + +The operator framework ties together Python's standard library's +logging facility with juju's. Logging should just work™ (file bugs). + +> ⚠️The tieing-together is done in the framework's `main`; if you somehow +> manage to not use that, you need to reimplement that for it to work. + +
Example + +```python +from ops.charm import CharmBase + +import logging + +logger = logging.getLogger() + +class MyCharm(CharmBase): + # ... + + def on_frobnicated(self, event): + logger.warning("'tis the end of times") +``` +
+ +## Relation data + +
Example + +```python + +``` +
+ +### Getting relation data + +### Setting relation data + +### Getting subordinate/juju-info relation data + +## Determining if the unit is the application “leader” + +## Changing “leader” settings + +## Rendering a pod spec (for k8s charms) + +## Simple actions: + +### Decoupled from charm, such as triggering a backup + +### Coupled to charm state, such as shutting down the service (charm needs to know so it doesn’t restart it) + +## Getting network information + +### IP address for daemons to listen on + +### IP address to publish to clients + +## Open/Close ports [not implemented yet] + +## Use of shared code and libraries + +### How shared code extends config.yaml, metadata.yaml, actions.yaml + +### How to embed Python dependencies from pypi (ie. reactive wheelhouse) + +## Storage From 59bf73a0e362f69ea177acb3f6e744f3ab418f9c Mon Sep 17 00:00:00 2001 From: "John R. Lenton" Date: Tue, 12 May 2020 16:22:08 +0100 Subject: [PATCH 04/10] Revert "tbd_examples: started working through the examples." This reverts commit 06d242ff31b4a5cc6e7e829f9e8f99c6974def86. --- tbd_examples.md | 130 ------------------------------------------------ 1 file changed, 130 deletions(-) delete mode 100644 tbd_examples.md diff --git a/tbd_examples.md b/tbd_examples.md deleted file mode 100644 index 008f6ed44..000000000 --- a/tbd_examples.md +++ /dev/null @@ -1,130 +0,0 @@ -# Examples of common operations in a charm - -## Configuration - -Configuration can be accesed on the `config` object of the `model` -property on the charm object. On first access it will run `juju -config-get`, and cache the results transparently. - -
Example - -```python -from ops.charm import CharmBase - -class MyCharm(CharmBase): - # ... - - def on_some_event(self, event): - if self.model.config['foo'] == 'xyzzy': - # yak yak yak -``` -
- -### Handling changes to configuration - -Configuration changes are reported via a `ConfigChangedEvent`, as -expected. - -
Example - -```python -from ops.charm import CharmBase - -class MyCharm(CharmBase): - # ... - - def on_config_changed(self, event): - # self.model.config will already have the new values -``` -
- -### Validate configuration, blocking charm until fixed - -If the charm has a way to validate its config, it can do that and set -the charm's status to blocked if it fails. - -
Example - -```python -from ops.charm import CharmBase -from ops.model import BlockedStatus - -class MyCharm(CharmBase): - # ... - - def on_config_changed(self, event): - try: - # call my custom validator that raises a custom exception on error - self._validate_config() - except MyValidationError as e: - self.status = BlockedStatus("fix yo stuff: {}".format(e)) -``` -
- -## Writing to juju logs - -The operator framework ties together Python's standard library's -logging facility with juju's. Logging should just work™ (file bugs). - -> ⚠️The tieing-together is done in the framework's `main`; if you somehow -> manage to not use that, you need to reimplement that for it to work. - -
Example - -```python -from ops.charm import CharmBase - -import logging - -logger = logging.getLogger() - -class MyCharm(CharmBase): - # ... - - def on_frobnicated(self, event): - logger.warning("'tis the end of times") -``` -
- -## Relation data - -
Example - -```python - -``` -
- -### Getting relation data - -### Setting relation data - -### Getting subordinate/juju-info relation data - -## Determining if the unit is the application “leader” - -## Changing “leader” settings - -## Rendering a pod spec (for k8s charms) - -## Simple actions: - -### Decoupled from charm, such as triggering a backup - -### Coupled to charm state, such as shutting down the service (charm needs to know so it doesn’t restart it) - -## Getting network information - -### IP address for daemons to listen on - -### IP address to publish to clients - -## Open/Close ports [not implemented yet] - -## Use of shared code and libraries - -### How shared code extends config.yaml, metadata.yaml, actions.yaml - -### How to embed Python dependencies from pypi (ie. reactive wheelhouse) - -## Storage From 739949f0e0904573a99adf6200a86c411c8ccbd6 Mon Sep 17 00:00:00 2001 From: "John R. Lenton" Date: Tue, 12 May 2020 16:52:42 +0100 Subject: [PATCH 05/10] more tweaks to the README --- README.md | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 99b96d3b4..b3ceca3d2 100644 --- a/README.md +++ b/README.md @@ -110,18 +110,25 @@ If you need help, have ideas, or would just like to chat with us, reach out on IRC: we're in [#smooth-operator](irc://chat.freenode.net/%23smooth-operator) on freenode (or try the [webchat](https://webchat.freenode.net/#smooth-operator)). -We also pay attention to juju's [discourse](https://discourse.jujucharms.com/), -but currently we don't actively post there; most discussion at this stage is on -IRC. +We also pay attention to juju's [discourse], but currently we don't actively +post there outside of our little corner of the [docs]; most discussion at this +stage is on IRC. + +[discourse]: https://discourse.juju.is/c/charming +[docs]: https://discourse.juju.is/c/docs/operator-framework # Operator Framework development -If you want to work in the framework *itself* you will need the following +If you want to work in the framework *itself* you will need some extra dependencies installed in your system: -- Python >= 3.5 -- PyYAML -- autopep8 -- flake8 + pip install -r requirements-dev.txt Then you can try `./run_tests`, it should all go green. + +If you want to build the documentation you can do + + pip install -r docs/requirements.txt + ./build_docs + + From d9a7c2f35eabe11cf29306454318342191746d23 Mon Sep 17 00:00:00 2001 From: "John R. Lenton" Date: Tue, 12 May 2020 17:06:04 +0100 Subject: [PATCH 06/10] still more tweaks to the README --- README.md | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index b3ceca3d2..d8fc7e788 100644 --- a/README.md +++ b/README.md @@ -14,8 +14,8 @@ The framework will help you to: Charms written using the operator framework are just Python code. The intention is for it to feel very natural for somebody used to coding in Python, and -reasonably easy to pick up for somebody whose might be a domain expert but -not necessarily a pythonista themselves. +reasonably easy to pick up for somebody who might be a domain expert but not +necessarily a pythonista themselves. The dependencies of the operator framework are kept as minimal as possible; currently that's Python 3.5 or greater, and `PyYAML` (both are included by @@ -29,7 +29,7 @@ framework (be it with reactive or without), we have an [introduction to the operator framework](/TBD) just for you. If you've gone through the above already and just want a refresher, or are -really impatient and just want to dive in, feel free to carry on down. +really impatient and need to dive in, feel free to carry on down. ## An Unrelenting Introduction @@ -38,8 +38,8 @@ a particular Python file. It could be anything that makes sense to your project, but let's assume this is `src/charm.py`. This file must be executable (and it must have the appropriate shebang line). -This file must be symlinked from `hooks/install` (or `hooks/start` if it's a -charm targeting k8s on juju < 3.8). +This file must be symlinked from `dispatch` (as of juju 3.8; otherwise from +`hooks/install`, and also from `hooks/start` if it's for k8s). You also need the usual `metadata.yaml` and `config.yaml` files, and any Python dependencies (maybe using some kind of virtualenv). In other words, your project @@ -53,10 +53,11 @@ might look like this: │ └── # ... your Python dependencies, including the operator framework itself ├── src/ │ └── charm.py -└── hooks/ - └── install -> ../src/charm.py +└── dispatch -> src/charm.py ``` +> 🛈 once `charmcraft build` works the layout will get simpler! + `src/charm.py` here is the entry point to your charm code. At a minimum, it needs to define a subclass of `CharmBase` and pass that into the framework's `main` function: @@ -104,31 +105,35 @@ harness.add_relation_unit(relation_id, 'postgresql/0', remote_unit_data={'key': self.assertEqual(harness.charm. ...) ``` -# Talk to us +## Talk to us If you need help, have ideas, or would just like to chat with us, reach out on -IRC: we're in [#smooth-operator](irc://chat.freenode.net/%23smooth-operator) on -freenode (or try the [webchat](https://webchat.freenode.net/#smooth-operator)). +IRC: we're in [#smooth-operator] on freenode (or try the [webchat]). We also pay attention to juju's [discourse], but currently we don't actively post there outside of our little corner of the [docs]; most discussion at this stage is on IRC. +[webchat]: https://webchat.freenode.net/#smooth-operator +[#smooth-operator]: irc://chat.freenode.net/%23smooth-operator [discourse]: https://discourse.juju.is/c/charming [docs]: https://discourse.juju.is/c/docs/operator-framework -# Operator Framework development +## Operator Framework development -If you want to work in the framework *itself* you will need some extra -dependencies installed in your system: +If you want to work in the framework *itself* you will need Python >= 3.5 and +the dependencies declared in `requirements-dev.txt` installed in your system. +Or you can use a virtualenv: + virtualenv --python=python3 env + source env/bin/activate pip install -r requirements-dev.txt Then you can try `./run_tests`, it should all go green. -If you want to build the documentation you can do +If you want to build the documentation you'll need the requirements from +`docs/requirements.txt`, or in your virtualenv pip install -r docs/requirements.txt - ./build_docs - +and then you can run `./build_docs`. From 47b4e674e57a9bc58de440ac713b43ff21bae7d2 Mon Sep 17 00:00:00 2001 From: "John R. Lenton" Date: Wed, 13 May 2020 15:24:26 +0100 Subject: [PATCH 07/10] s/Unrelenting/Quick/ in the README --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d8fc7e788..34f4a6a33 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ operator framework](/TBD) just for you. If you've gone through the above already and just want a refresher, or are really impatient and need to dive in, feel free to carry on down. -## An Unrelenting Introduction +## An Quick Introduction Operator framework charms are just Python code. The entry point to your charm is a particular Python file. It could be anything that makes sense to your project, From 6580fbdea298a87ae0a141ca885b839e437d6727 Mon Sep 17 00:00:00 2001 From: "John R. Lenton" Date: Wed, 13 May 2020 15:24:57 +0100 Subject: [PATCH 08/10] "An Quick"? Really? --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 34f4a6a33..dff3be116 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ operator framework](/TBD) just for you. If you've gone through the above already and just want a refresher, or are really impatient and need to dive in, feel free to carry on down. -## An Quick Introduction +## A Quick Introduction Operator framework charms are just Python code. The entry point to your charm is a particular Python file. It could be anything that makes sense to your project, From c065bf53c3513157dd9b07aa44c733e5eb07b568 Mon Sep 17 00:00:00 2001 From: "John R. Lenton" Date: Thu, 14 May 2020 16:17:15 +0100 Subject: [PATCH 09/10] README: some more tweaks and improvements via peer review --- README.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index dff3be116..7e5ec3474 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # The Operator Framework The Operator Framework provides a simple, lightweight, and powerful way of -writing juju charms, the best way to encapsulate operational experience in code. +writing Juju charms, the best way to encapsulate operational experience in code. The framework will help you to: @@ -21,10 +21,10 @@ The dependencies of the operator framework are kept as minimal as possible; currently that's Python 3.5 or greater, and `PyYAML` (both are included by default in Ubuntu's cloud images from 16.04 on). -If you're new to the world of juju and charms, you should probably dive into our +If you're new to the world of Juju and charms, you should probably dive into our [tutorial](/TBD). -If you know about juju, and have written charms that didn't use the operator +If you know about Juju, and have written charms that didn't use the operator framework (be it with reactive or without), we have an [introduction to the operator framework](/TBD) just for you. @@ -38,8 +38,9 @@ a particular Python file. It could be anything that makes sense to your project, but let's assume this is `src/charm.py`. This file must be executable (and it must have the appropriate shebang line). -This file must be symlinked from `dispatch` (as of juju 3.8; otherwise from -`hooks/install`, and also from `hooks/start` if it's for k8s). +This file must be symlinked from `dispatch` (supported by Juju 2.8; otherwise +from `hooks/install` and `hooks/upgrade-charm`, and `hooks/start` if it is a k8s +charm). You also need the usual `metadata.yaml` and `config.yaml` files, and any Python dependencies (maybe using some kind of virtualenv). In other words, your project @@ -63,7 +64,6 @@ needs to define a subclass of `CharmBase` and pass that into the framework's `main` function: ```python -#!../env/bin/python3 from ops.charm import CharmBase from ops.main import main @@ -100,7 +100,8 @@ harness = Harness(MyCharm) relation_id = harness.add_relation('db', 'postgresql') # Now instantiate the charm to see events as the model changes harness.begin() -harness.add_relation_unit(relation_id, 'postgresql/0', remote_unit_data={'key': 'val'}) +harness.add_relation_unit(relation_id, 'postgresql/0') +harness.update_relation_data(relation_id, 'postgresql/0', {'key': 'val'}) # Check that charm has properly handled the relation_joined event for postgresql/0 self.assertEqual(harness.charm. ...) ``` @@ -110,7 +111,7 @@ self.assertEqual(harness.charm. ...) If you need help, have ideas, or would just like to chat with us, reach out on IRC: we're in [#smooth-operator] on freenode (or try the [webchat]). -We also pay attention to juju's [discourse], but currently we don't actively +We also pay attention to Juju's [discourse], but currently we don't actively post there outside of our little corner of the [docs]; most discussion at this stage is on IRC. From 820a2d393849c6d3b8f2ae67c4ec45e6b31f4203 Mon Sep 17 00:00:00 2001 From: chipaca Date: Thu, 14 May 2020 17:53:07 +0100 Subject: [PATCH 10/10] Comment out the TBD links --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 7e5ec3474..fdc2de606 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,7 @@ The dependencies of the operator framework are kept as minimal as possible; currently that's Python 3.5 or greater, and `PyYAML` (both are included by default in Ubuntu's cloud images from 16.04 on). + ## A Quick Introduction Operator framework charms are just Python code. The entry point to your charm is