Merge develop into 3.0-dqlite #14652
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Bumps [dawidd6/action-homebrew-bump-formula](https://github.com/dawidd6/action-homebrew-bump-formula) from b1f4659559bcc32ab942f0b0f851e3c856d05253 to 3.8.0. This release includes the previously tagged commit. - [Release notes](https://github.com/dawidd6/action-homebrew-bump-formula/releases) - [Commits](dawidd6/action-homebrew-bump-formula@b1f4659...dd221ff) --- updated-dependencies: - dependency-name: dawidd6/action-homebrew-bump-formula dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
ported it to bash
…after unregister.
The series list from the metadata isn't what is available for deploy. That comes from the Charmhub API response default-release bases (which in turn comes from charmcraft and the manifest.yaml). I also remove the ad-hoc isKubernetes function here and use corecharm.IsKubernetes instead. Unfortunately due to the existing TODO in convert.go:104, we don't get the manifest.yaml and can't pass that to charmMeta / IsKubernetes, so IsKubernetes returns the wrong value for container charms, e.g., the likes of snappass-test. So snappass-test says it "supports: focal" instead of what I'd expect, "supports: kubernetes". I think we'll need to fix that in a follow-up change once ManifestYAML is provided by the Charmhub API. This commit also removes the unused Supports field from bundleInfoOutput (it was never set so always omitted).
When Juju users have to input interactive credentials for hidden file based attributes they previously had no way to enter this data. This change now allows file names to be entered with the user input echoed.
Published candidate and edge versions and included them as part of this process.
Hopefully this helps 'future us' understand what the build process is intended to be.
dd 2-0-stable and 2-0-edge make targets. Add .jujuignore and pack statements to */charmcraft.yaml for extra directories. Update README.md
Both 2-0-edge and 2-0-stable now have new revisions and versions, record that information into README.md so that people can follow what is going on. (note that 2-0-edge is now a conflicting version string)
…ile-credentials juju#14551 When Juju users have to input interactive credentials for hidden file based attributes they previously had no way to enter this data. This change now allows file names to be entered with the user input echoed. ## Checklist - [x] Code style: imports ordered, good names, simple structure, etc - [x] Comments saying why design decisions were made - [x] Go unit tests, with comments saying what you're testing - [x] [Integration tests](https://github.com/juju/juju/tree/develop/tests), with comments saying what you're testing - [x] [doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) added or updated in changed packages ## QA steps - Create a new lxd cloud with `juju add-cloud` - Add credentials for the new cloud with `juju add-credentials` choosing the certificate input method during an interactive session. Make sure that filename can be entered for the certificate data and what the user types can be seen. ## Documentation changes N/A ## Bug reference https://bugs.launchpad.net/juju/+bug/1988239
They're no longer being used in our ci since it has been replaced with the migration tests in our model test suite
juju#14611 Agent config for API addresses includes local Fan addresses on the controller when Fan network is set up. These are not usable for controller communication. Here we include such addresses as those filtered as bridges, meaning they will not be in agent configuration, and never tried for controller communication ## QA steps Prior to this patch, connecting to Mongo on an AWS controller, one can observe these addresses in the _controllers_ collection: ``` juju:PRIMARY> db.controllers.find({"_id": "apiHostPortsForAgents"}).pretty() { "_id" : "apiHostPortsForAgents", "apihostports" : [ [ { "value" : "54.159.29.145", "addresstype" : "ipv4", "networkscope" : "public", "port" : 17070, "spaceid" : "0" }, { "value" : "172.31.16.134", "addresstype" : "ipv4", "networkscope" : "local-cloud", "port" : 17070, "spaceid" : "0" }, { "value" : "252.16.134.1", "addresstype" : "ipv4", "networkscope" : "local-fan", "port" : 17070 }, { "value" : "127.0.0.1", "addresstype" : "ipv4", "networkscope" : "local-machine", "port" : 17070 }, { "value" : "::1", "addresstype" : "ipv6", "networkscope" : "local-machine", "port" : 17070 } ] ], "txn-revno" : NumberLong(5), "txn-queue" : [ "6321c73701cb932137c859d6_7e30d7b3" ] } ``` Bootstrapping with this patch will result in the same document without the "local-fan" scoped address. ## Bug reference https://bugs.launchpad.net/juju/+bug/1942804
…er-test-unregister-test juju#14623 This PR fixes the test-controller-test-unregister-test tests, the original test does not consider that `controllers.yaml` file may not exist. The test is now rewritten using the fake `controllers.yaml` file created in the test folder. The test is moved from the controller suite to the cli suite. ## Checklist - [x] Code style: imports ordered, good names, simple structure, etc - [x] Comments saying why design decisions were made ## QA steps *Commands to run to verify that the change works.* ```sh cd tests ./main.sh -v cli run_unregister ```
…idd6/action-homebrew-bump-formula-dd221ff435f42fa8102b5871bb1929af9d76476c juju#14485 Bumps [dawidd6/action-homebrew-bump-formula](https://github.com/dawidd6/action-homebrew-bump-formula) from b1f4659559bcc32ab942f0b0f851e3c856d05253 to 3.8.0. This release includes the previously tagged commit. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/dawidd6/action-homebrew-bump-formula/commit/ad9fa45815e933d9ef7d83c699560316946b4b61"><code>ad9fa45</code></a> Specify Access Token scopes in README</li> <li><a href="https://github.com/dawidd6/action-homebrew-bump-formula/commit/4e7716c8925b1988bb8616e5fd1b59d75c6fa011"><code>4e7716c</code></a> README: add another example</li> <li><a href="https://github.com/dawidd6/action-homebrew-bump-formula/commit/a3cb816c8ae5c3f48686e9d556b00c43c1109891"><code>a3cb816</code></a> main: add no-audit back</li> <li><a href="https://github.com/dawidd6/action-homebrew-bump-formula/commit/8a8b416db997f3dadd11efe464d3ebecde96fe6d"><code>8a8b416</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/dawidd6/action-homebrew-bump-formula/issues/22">#22</a> from dawidd6/dawidd6-patch-1</li> <li>See full diff in <a href="https://github.com/dawidd6/action-homebrew-bump-formula/compare/b1f4659559bcc32ab942f0b0f851e3c856d05253...dd221ff435f42fa8102b5871bb1929af9d76476c">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>
…storage_test juju#14625 Covered by new bash based tests. ## Checklist *If an item is not applicable, use `~strikethrough~`.* - ~[ ] Code style: imports ordered, good names, simple structure, etc~ - ~[ ] Comments saying why design decisions were made~ - ~[ ] Go unit tests, with comments saying what you're testing~ - ~[ ] [Integration tests](https://github.com/juju/juju/tree/develop/tests), with comments saying what you're testing~ - ~[ ] [doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) added or updated in changed packages~
juju#14615 Currently, only model super users can SSH/SCP for k8s models because the current implementation uses the model credential to call the exec API. This PR ensures the model operator creates a set of RBAC resources for exec purposes and changes the SSH/SCP command to use the new service account token. This allows any users having a model admin role to use SSH/SCP for k8s models. - [x] Code style: imports ordered, good names, simple structure, etc - [x] Comments saying why design decisions were made - [x] Go unit tests, with comments saying what you're testing - [ ] ~[Integration tests](https://github.com/juju/juju/tree/develop/tests), with comments saying what you're testing~ - [ ] ~[doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) added or updated in changed packages~ ## QA steps ```console $ juju bootstrap microk8s k1 $ juju add-model t1 $ mkubectl -ncontroller-k1 get sa/model-exec NAME SECRETS AGE model-exec 1 3m55s $ mkubectl -nt1 get sa/model-exec NAME SECRETS AGE model-exec 1 3m51s $ juju deploy snappass-test $ juju ssh -m k1:controller 0 # env | grep HOSTNAME HOSTNAME=controller-0 $ juju ssh -m k1:t1 snappass-test/0 # env | grep HOSTNAME HOSTNAME=snappass-test-0 $ juju add-user u-read $ juju add-user u-write $ juju add-user u-admin $ juju grant u-write write t1 $ juju grant u-read read t1 $ juju grant u-admin admin t1 $ juju logout --force Logged out. You are no longer logged into any controllers. $ juju login -u u-read -c k1 please enter password for u-read on k1: Welcome, u-read. You are now logged into "k1". Current model set to "admin/t1". $ juju ssh -m k1:admin/t1 snappass-test/0 ERROR permission denied (unauthorized access) $ juju logout --force Logged out. You are no longer logged into any controllers. $ juju login -u u-write -c k1 please enter password for u-write on k1: Welcome, u-write. You are now logged into "k1". Current model set to "admin/t1". $ juju ssh -m k1:admin/t1 snappass-test/0 ERROR permission denied (unauthorized access) $ juju logout --force Logged out. You are no longer logged into any controllers. $ juju login -u u-admin -c k1 please enter password for u-admin on k1: Welcome, u-admin. You are now logged into "k1". Current model set to "admin/t1". $ juju ssh -m k1:admin/t1 snappass-test/0 # env | grep HOSTNAME HOSTNAME=snappass-test-0 ``` ## Documentation changes No ## Bug reference https://bugs.launchpad.net/juju/+bug/1989160
Split model migration tests;
…-credentials-py-from-juju-repo juju#14630 This PR removes assess_add_credentials.py, because we switch to the bash-based version of the add_remove_credential test. ## Checklist - [x] Code style: imports ordered, good names, simple structure, etc - [x] Comments saying why design decisions were made ## QA steps Check that the file is removed.
…ss-persistent-storage-python-tests juju#14627 Covered by [bash ](juju#14575 tests . ## Checklist *If an item is not applicable, use `~strikethrough~`.* - ~[ ] Code style: imports ordered, good names, simple structure, etc~ - ~[ ] Comments saying why design decisions were made~ - ~[ ] Go unit tests, with comments saying what you're testing~ - ~[ ] [Integration tests](https://github.com/juju/juju/tree/develop/tests), with comments saying what you're testing~ - ~[ ] [doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) added or updated in changed packages~
juju#14636 Secrets are now included in a model migration. ## Checklist - [X] Code style: imports ordered, good names, simple structure, etc - [X] Comments saying why design decisions were made - [X] Go unit tests, with comments saying what you're testing - ~[ ] [Integration tests](https://github.com/juju/juju/tree/develop/tests), with comments saying what you're testing~ - ~[ ] [doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) added or updated in changed packages~ ## QA steps *Commands to run to verify that the change works.* bootstrap 2 controllers on c1, create a model and a secret and share it with another app run secret-get from the other app's unit to ensure a consumer record is created migrate the model to c2 run secret-get again to see the secret has been migrated
Due to a new endpoints service made by Juju for statefulset deployments our dashboard proxying mechanism has stopped working in k8s as we get confused about what service to use. This change ignores the endpoints service when considering what service to proxy onto for the client.
[JUJU-1830] [CONTRIBUTING.md] Add CLA section + other small fixes
juju#14634 Due to a new endpoints service made by Juju for statefulset deployments our dashboard proxying mechanism has stopped working in k8s as we get confused about what service to use. This change ignores the endpoints service when considering what service to proxy onto for the client. ## Checklist - [x] Code style: imports ordered, good names, simple structure, etc - [x] Comments saying why design decisions were made - [x] Go unit tests, with comments saying what you're testing - [x] [Integration tests](https://github.com/juju/juju/tree/develop/tests), with comments saying what you're testing - [x] [doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) added or updated in changed packages ## QA steps ```sh 1. Bootstrap Juju to a kubernetes cluster 2. Switch to the Juju controller model. `juju switch controller` 3. Deploy the dashboard charm. `juju deploy juju-dashboard-k8s --channel=edge` 4. Relate the dashboard to the controller. `juju relate juju-dashboard-k8s controller` 5. Test the dashboard command works with. `juju dashboard` ``` ## Documentation changes N/A ## Bug reference N/A
juju#14639 Merge 2.9 juju#14613 juju#14609 juju#14605 juju#14603 juju#14601 juju#14600 juju#14606 juju#14592 juju#14575 juju#14561 There were a lot of ssh client conflicts. We no longer need to support the old workflow so the cloud and model facade calls can be removed. Also changed the minimum 2.9 client version to 2.9.35 Conflicts ``` # Conflicts: # acceptancetests/assess_storage.py # api/client/sshclient/facade_test.go # apiserver/facades/client/sshclient/facade.go # apiserver/facades/client/sshclient/facade_test.go # apiserver/facades/client/sshclient/register.go # apiserver/facades/schema.json # caas/broker.go # caas/kubernetes/provider/proxy/setup.go # caas/kubernetes/provider/secrets.go # caas/kubernetes/provider/secrets_test.go # cmd/juju/application/deployer/bundlehandler_test.go # cmd/juju/ssh/ssh_container.go # cmd/juju/ssh/ssh_container_test.go ``` ## Checklist - [X] Code style: imports ordered, good names, simple structure, etc - [X] Comments saying why design decisions were made - [X] Go unit tests, with comments saying what you're testing - ~[ ] [Integration tests](https://github.com/juju/juju/tree/develop/tests), with comments saying what you're testing~ - ~[ ] [doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) added or updated in changed packages~ ## QA steps I manually checked that juju ssh worked for k8s and machine models.
juju#14650 Merge 2.9 juju#14639 juju#14641 juju#14638 Main conflicts in: cmd/juju/application apserver/facades/agent/machinemanager due to the removal of series from the api params and the new facade v8. ## Checklist - [X] Code style: imports ordered, good names, simple structure, etc - [X] Comments saying why design decisions were made - [X] Go unit tests, with comments saying what you're testing - ~[ ] [Integration tests](https://github.com/juju/juju/tree/develop/tests), with comments saying what you're testing~ - ~[ ] [doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) added or updated in changed packages~ ## QA steps See PRs
…ential-detect juju#14651 Add trace log for kubeconfig file detecting;
jujubot
added a commit
that referenced
this pull request
Feb 10, 2023
#15177 The following brings the 3.0-dqlite feature branch into the develop branch. ### Changes This brings in the dqlite database to sit along side the mongo database. Currently, only leases are implemented in Juju using dqlite, more controller base configuration and data will be subsequently moved over to dqlite once this branch has landed. #### Leases/Raft The whole raft implementation has been removed from Juju completely. This includes the following workers: - raft backstop - raft clusterer - raft log - raft transport - global clock updater In addition, the raft API implementation has also been removed. Instead, the lease has changed to handle the store (dqlite db) directly, improving readability and complexity. ### Jujud The `jujud` agent is now built using musl (specifically musl-gcc). This allows `juju` to be built statically embedding `dqlite` in the same process. There are still some rough edges when building and testing and when this lands, we expect to see some churn to polish any of those issues. Using `go test` is expected to still work as is, this is a last-minute change so that we can utilize sqlite directly for local tests. If you require to test with dqlite (linux only), then running `-tags="dqlite"` with builds/tests/installs is required. All CI jobs are required to run with the dqlite tag. Some notes: 1. `CGO_ENABLED=1` and `CGO_LDFLAGS_ALLOW="(-Wl,-wrap,pthread_create)|(-Wl,-z,now)"` are required if you're using dqlite directly. 2. You are expected to install musl directly on your system if you want to build, using `make musl-install`. This will require sudo. 3. For development purposes we will download dqlite `.a` files from an s3 bucket to facilitate the setup process. The tar file is sha256 summed to ensure no MITM. You can build these locally if you want to bypass s3 using `make dqlite-build-lxd`. This will spin up an lxd container to build. **Do not attempt** to run `make dqlite-build` locally, unless you know what you're doing. 4. To access dqlite from a controller, use `make repl`, this will open up a pseudo repl when you can then explore the database with. `.open <db name>` and then you can use SQL from there. 5. Cross compilation to other architectures can be done using `GOARCH` and `GOOS` before `make install` or `make build`. There are probably some things I've forgotten, expect a discourse post soon, which will highlight the development flow. ---- Two conflicts when merging. The resolution was to bring in the secret backends for the manifold tests and the controller config type changed for `DefaultMigrationMinionWaitMax`. ``` CONFLICT (content): Merge conflict in cmd/jujud/agent/machine/manifolds_test.go CONFLICT (content): Merge conflict in controller/config.go ``` c141b2e (upstream/3.0-dqlite) Merge pull request #15159 from SimonRichardson/system-install-musl-by-default 83656e2 Merge pull request #15156 from SimonRichardson/change-log-ddl 125c19d Fix static-analysis pipeline (#15168) 5abfa24 Merge pull request #15140 from SimonRichardson/allow-testing-on-mac 1dc60f6 (3.0-dqlite) Merge pull request #15153 from SimonRichardson/content-addressable-deps 5a1cd24 Merge pull request #15150 from jack-w-shaw/JUJU-2615_symlink_sudo 4502d63 Merge pull request #15148 from SimonRichardson/better-install-method 88941dd Merge pull request #15134 from SimonRichardson/bootstrap-dqlite-agent-tests 2551ffc Merge pull request #15130 from SimonRichardson/build-jujud-snap 0180a53 (origin/3.0-dqlite, manadart/3.0-dqlite) Merge pull request #15123 from SimonRichardson/fix-manifold-lease-expiry-tests fdf9cc7 Merge pull request #15115 from SimonRichardson/remove-jujud-main-test-file bf58843 Merge pull request #15113 from SimonRichardson/remove-api-raftlease-api-client f9419c0 Merge pull request #15112 from SimonRichardson/fix-initializing-state-twice 334d557 Merge pull request #15108 from SimonRichardson/github-action-go-build 2ee6e1a Merge pull request #15107 from SimonRichardson/cross-building-jujud 5a93305 Merge pull request #15087 from SimonRichardson/ensure-placement-of-file da95dc0 Merge pull request #15086 from SimonRichardson/more-sudo-changes 7b86376 Merge pull request #15085 from SimonRichardson/sudo-apt-get c4d4eb6 Merge pull request #15057 from SimonRichardson/dqlite-local-build 0ac79b3 Merge pull request #15061 from manadart/develop-into-3.0-dqlite adc20f7 Merge pull request #15043 from SimonRichardson/allow-overriding-arch-machine 8c02f22 Merge pull request #15048 from SimonRichardson/static-analysis-fix 4547c06 Merge pull request #15050 from manadart/dqlite-address-option d51b324 Merge pull request #15049 from manadart/dqlite-bootstrap-options 3801b78 Merge pull request #15047 from manadart/develop-into-3.0-dqlite 22d5247 Merge pull request #15037 from SimonRichardson/standardise-dqlite-build 25640a2 Merge pull request #15036 from SimonRichardson/remove-batch-fsm-controller-config dfa4cb1 Merge pull request #15041 from manadart/dqlite-fix-mock caf9481 Merge pull request #15034 from manadart/develop-into-3.0-dqlite c91985d Merge pull request #15035 from SimonRichardson/remove-typed-lease-error 42d17be Merge pull request #15009 from SimonRichardson/allow-repl-via-juju-ssh d798238 Merge pull request #15002 from manadart/dqlite-use-lease-store e4f0d39 Merge pull request #14918 from manadart/3.0-dqlite-lease-store 8315fb7 Merge pull request #14986 from manadart/dqlite-build-from-tags a73b947 Merge pull request #14927 from manadart/3.0-dqlite-lease-store-interface 1657a1d Merge pull request #14910 from manadart/3.0-dqlite-db-supply 27b23f3 Merge pull request #14909 from manadart/3.0-into-3.0-dqlite 6adff35 Merge pull request #14756 from manadart/develop-into-3.0-dqlite 37d81ff Merge pull request #14717 from manadart/develop-into-3.0-dqlite fe2edb8 Merge pull request #14671 from manadart/3.0-simplify-dbaccessor 1a09836 Merge pull request #14604 from manadart/3.0-bootstrap-controller-db 5ad011e Merge pull request #14652 from manadart/develop-into-3.0-dqlite 1c3d250 Merge pull request #14591 from manadart/develop-into-3.0-dqlite 229cd3e Merge pull request #14578 from manadart/3.0-dqlite-simplify 9d715ba Merge pull request #14565 from manadart/develop-into-3.0-dqlite 92ffd88 Merge pull request #14466 from manadart/develop-into-3.0-dqlite 57f67ce Merge pull request #14336 from manadart/develop-into-3.0-dqlite 648d354 Merge pull request #14364 from manadart/update-musl 198621d Merge pull request #14241 from manadart/develop-into-3.0-dqlite 0360db6 Merge pull request #14153 from manadart/develop-into-3.0-dqlite 17950b2 Merge pull request #14053 from manadart/develop-into-3.0-dqlite 9452026 Merge pull request #14016 from manadart/develop-into-3.0-dqlite 741baca Merge pull request #13963 from manadart/develop-into-3.0-dqlite 5449603 Merge pull request #13969 from manadart/dqlite-manifolds 7b612a0 Merge pull request #13944 from SimonRichardson/dqlite-develop
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Zero-conflict merge bringing forward: