/juju

Add macaroon authentication and bundle deployment (chicago-cubs merge) #3590

Merged
merged 192 commits into from Oct 23, 2015

Conversation

Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
@rogpeppe
Owner

rogpeppe commented Oct 23, 2015

This merges the chicago-cubs feature branch into trunk.
This adds macaroon authentication support (along
with much API server cleanup) and support
for bundle deployment.

(Review request: http://reviews.vapour.ws/r/2991/)

frankban and others added some commits Aug 19, 2015

@frankban
API endpoint for retrieving bundle changes. 
Implement the GetBundleChanges API endpoint on the Client.
This is used to generate end return the list of changes
required to deploy a bundle (from the GUI perspective),
given a bundle YAML content.
7d589f5
@frankban
Simplify the GetBundleChanges client and decouple structs used in the… 
… API calls.
f12db18
@jujubot
Merge pull request #3035 from frankban/changes-endpoint 
API endpoint for retrieving bundle changes.

Implement the GetBundleChanges API endpoint on the Client.
This is used to generate end return the list of changes
required to deploy a bundle (from the GUI perspective),
given a bundle YAML content.

(Review request: http://reviews.vapour.ws/r/2416/)
0eb685e
@frankban
Improve GetBundleChanges API tests.
c6b7f24
@jujubot
Merge pull request #3079 from frankban/changes-endpoint-fixes 
Improve GetBundleChanges API tests.

This is already reviewed at http://reviews.vapour.ws/r/2416/ but I forgot to commit the changes.

(Review request: http://reviews.vapour.ws/r/2460/)
950e74f
Use ParseReference rather than InferURL
c7fd86c
@frankban
Migrate to unstable dependencies.
f97f910
@frankban
Add a comment for the UploadCharm test helper.
3c7ccb3
@jujubot
Merge pull request #3111 from frankban/unstable-packages 
Migrate to unstable dependencies.

This branch also includes:
- fix tests making use of `charmstoretesting`, which does not exist anymore;
- sort imports;
- fix httpbakery usage to use the v1 package;
- fix cleaning up of some tests making use of mongo sessions;
- update dependencies file.

Please note that this is proposed for merging into the `guibundles` feature branch:
we plan to freeze unstable packages before including this work in core trunk.

(Review request: http://reviews.vapour.ws/r/2491/)
6c4a7c3
Missing colon; test matches
cbf4fb0
@jujubot
Merge pull request #3086 from makyo/resolve-url-1 
Use ParseReference rather than InferURL

InferURL is [deprecated](https://github.com/juju/charm/blob/v5/url.go#L240) in favor of ParseReference, and future work around charm URLs will be taking place within ParseReference.

(Review request: http://reviews.vapour.ws/r/2467/)
6d3fb2e
@cmars
Comments sketching out macaroon-based auth.
f70848a
@cmars
More sketching
c4e2ebf
@rogpeppe
make it compile
19f7e5b
@alesstimec
Environs config 
Added identity url and identity public key to the environs config.
9192a26
@rogpeppe
Merge pull request #3161 from alesstimec/environs-config-identity 
Environs config
f7ca261
@alesstimec
Environs config IdentityPublicKey change.
8343a82
@jujubot
Merge pull request #3169 from alesstimec/environs-config-identity-02 
Environs config IdentityPublicKey change.



(Review request: http://reviews.vapour.ws/r/2547/)
b26871e
@mhilton @mattyw
apiserver/authentication: Added macaroon authenticator
fef289e
@frankban @alesstimec
Migrating to charm.v6-unstable and macaroon-bakery.v1.
e91b28e
@jujubot
Merge pull request #3173 from alesstimec/dependency-update 
Migrating to charm.v6-unstable and macaroon-bakery.v1.



(Review request: http://reviews.vapour.ws/r/2550/)
f5fbd48
@rogpeppe
apiserver: create macaroon and bakery service
740d2eb
@mattyw
apiserver/authentication: Added comment about macaroon serialisation
f3126d6
@jujubot
Merge pull request #3174 from rogpeppe/037-apiserver-macaroon 
apiserver: create macaroon and bakery service



(Review request: http://reviews.vapour.ws/r/2551/)
30a9491
@jujubot
Merge pull request #3176 from mattyw/serialisation-comment 
apiserver/authentication: Added comment about macaroon serialisation



(Review request: http://reviews.vapour.ws/r/2553/)
03043be
@mattyw
apiserver/authentication: Comments from review
9ce56e7
@jujubot
Merge pull request #3172 from mattyw/macaroon-login-flow 
apiserver/authentication: Added macaroon authenticator



(Review request: http://reviews.vapour.ws/r/2549/)
6e68008
@frankban
Update code to use new bundlechanges.
76632be
@mattyw
apiserver: Changes for macaroon workflow.
988a617
@alesstimec @mattyw
API macaroon-based login flow
b14e240
@mattyw
apiserver: Added tests for the macaroon login flow
8b19ba3
@mattyw
api: trivial changes before landing 
dependencies.tsv: Updated to latest juju/utils
44dc2b7
@tasdomas
Merge branch 'master' into master-update 
Conflicts:
	apiserver/addresser/addresser_test.go
	apiserver/server_test.go
	dependencies.tsv
	state/upgrades.go
	upgrades/steps125.go
1e4d31f
@mattyw
api,apiserver: Macaroon login workflow review comment fixes
66f0f01
@mattyw
api,apiserver: Macaroon login flow from second round of reviews
ca19baf
@jujubot
Merge pull request #3201 from tasdomas/master-update 
Master update



(Review request: http://reviews.vapour.ws/r/2579/)
2f83268
@tasdomas
Use updated charm.v6-unstable in chicago-cubs.
a6e9b96
@jujubot
Merge pull request #3198 from mattyw/003-macaroon-login-flow 
api,apiserver: Macaroon based login workflow



(Review request: http://reviews.vapour.ws/r/2576/)
0773110
@jujubot
Merge pull request #3204 from tasdomas/cubs-use-updated-charm.v6-unst… 
…able

Use updated charm.v6-unstable in chicago-cubs.



(Review request: http://reviews.vapour.ws/r/2582/)
5b1a1cb
@frankban
Update dependencies.
668108d
@mattyw
apiserver: AuthenticatorForTag now exists as a method on the apiserver
5309101
@alesstimec
cmd/juju macaroon based login
2561a34
@jujubot
Merge pull request #3206 from mattyw/authenticator-for-tag-in-server 
apiserver: AuthenticatorForTag now exists as a method on the apiserver



(Review request: http://reviews.vapour.ws/r/2584/)
d01749a
@jujubot
Merge pull request #3205 from alesstimec/cmd-juju-macaroon-login 
cmd/juju macaroon based login



(Review request: http://reviews.vapour.ws/r/2583/)
92b3c7c
@jujubot
Merge pull request #3207 from frankban/new-bundlechanges 
Use new bundlechanges API.

The new bundlechanges API returns changes as interfaces rather than concrete structs. This branch reflects this new behavior.

(Review request: http://reviews.vapour.ws/r/2585/)
cec5e0a
@tasdomas
Merge branch 'master' into chicago-cubs
8b0476d
@jujubot
Merge pull request #3229 from tasdomas/update-cubs-with-master 
Update cubs with master



(Review request: http://reviews.vapour.ws/r/2607/)
4cb123d
@frankban
Bundle deployment base structure. 
For now, the system just ads the charms, but local and cs bundles are
supported, and also bundles including non-public charms.
e7cb82e
@frankban
Provide a verify constraints function to the bundle deployer. Use an … 
…interface for notifying bundle deployment progress.
1a94533
@frankban
Use resolveCharmStoreEntityURL to disambiguate the concept of entity.
11bed3c
@jujubot
Merge pull request #3249 from frankban/deploy-bundle-base 
Bundle deployment base structure.

For now, the system just ads the charms, but local and cs bundles are
supported, and also bundles including non-public charms.

(Review request: http://reviews.vapour.ws/r/2628/)
25596c1
@frankban
Bundle deployment: handle deploying/upgrading services.
a4fd24e
@frankban
bundle deployment: use a separate setServiceOptions and cover charm u… 
…pgrade failures when a different user is provided.
2d734a5
@frankban
Bundle deployment: use errors.Errorf rather than fmt.Sprintf.
686bee7
@frankban
Include service messages in bundle deployment authorization tests.
ec831f7
@jujubot
Merge pull request #3251 from frankban/bundle-add-services 
Bundle deployment: handle deploying/upgrading services.



(Review request: http://reviews.vapour.ws/r/2630/)
2a4bdca
@frankban
Bundle deployment: handle adding service relations. 
Add relations between services if not already established.
cf28936
@rogpeppe
Merge remote-tracking branch 'origin/master' into HEAD 
The only conflicts were file deletions in master and import
path conflicts.

Conflicts:
	apiserver/adminv2_test.go
	apiserver/storage/package_test.go
	cmd/juju/commands/helptool.go
	cmd/juju/commands/publish.go
	dependencies.tsv
	juju/series/series_windows_test.go
	provider/maas/environ.go
	worker/uniter/filter/filter.go
	worker/uniter/filter/filter_test.go
	worker/uniter/filter/interface.go
	worker/uniter/metrics.go
	worker/uniter/modes.go
	worker/uniter/op_plumbing.go
	worker/uniter/operation/metrics_test.go
	worker/uniter/relation/dyingsource.go
	worker/uniter/relation/hookqueue_test.go
	worker/uniter/relation/livesource.go
	worker/uniter/relations.go
	worker/uniter/runner/context/contextfactory.go
	worker/uniter/runner/contextfactory_test.go
	worker/uniter/runner/export_test.go
	worker/uniter/runner/util_test.go
	worker/uniter/storage/source.go
	worker/uniter/storage/source_test.go
987b0fa
@jujubot
Merge pull request #3250 from rogpeppe/039-cubs-merge-master 
merge master into chicago cubs feature branch

The only conflicts were import path changes and file deletions, both trivially resolved.

(Review request: http://reviews.vapour.ws/r/2629/)
e47a74b
@frankban
Change bundle deployment output messages.
89f986e

fwereade and others added some commits Sep 10, 2015

@fwereade @alesstimec
address reviews
2d43257
@perrito666 @alesstimec
Adressed fwereade comments.
9bad041
@perrito666 @alesstimec
Simplified worker.
88a14dd
@perrito666 @alesstimec
Address fwereade comments.
0bebc6e
@tasdomas @alesstimec
Meter status hook worker manifold.
e8924b4
@perrito666 @alesstimec
Corrected import
0dad058
@perrito666 @alesstimec
Register facade.
33273e9
@mattyw @alesstimec
state/watcher: Improving the documentation on the docWatcher
c153801
@bogdanteleaga @alesstimec
Added os specific disk size constraints to the gce provider
120c1cc
@bogdanteleaga @alesstimec
Fix vsphere to comply with changes in the common provider
b1af80f
@bogdanteleaga @alesstimec
Added os specific disk size constraints to the ec2 provider
c3ccbeb
@tasdomas @alesstimec
Added NewWorkerPaths constructor that creates paths with worker-speci… 
…fic socket locations.
f9bd31e
@alesstimec
State builtin metrics tests.
bfa2cc0
@alesstimec
Added metric validation to the metric recorder.
391e508
@alesstimec
Forward port of fix for issue 1499571
3229676
@perrito666 @alesstimec
Fixed test for statushistorypruner worker.
c417f2d
@mjs @alesstimec
network: make HostPort sorting stable 
Sorting of HostPorts wasn't stable for instances with the same address
but different port numbers.

The bug is unlikely to be causing any problems within Juju at the
moment but is good to have correct for future uses of this
functionality.
d357160
@bogdanteleaga @alesstimec
Addressed reviews
24d2468
@alesstimec
lp1373516 - Change default ec2 instance to m3.medium 
This PR changes the selection of ec2 instance types to follow
the following rules:

- m1 instances will only be selected if explicitly specified using
the instance-type=m1.* constraint

- if no constraints are specified, the default instance type will
be m3.medium

- if constraints are specified, m1 instance types which satisfy
the constraints will not be selected for deployment
0a75303
@mjs @alesstimec
network: added SelectInternalHostPorts 
This is like SelectInternalHostPort but selects all the best candidate
addresses, instead of just one of them. The logic for selecting the
best matching addresses has been cleaned up and reworked in terms of
selecting multiple addresses.
a51cc3f
@davecheney @alesstimec
cmd/jujud/agent: refactor primeAgent helper 
Refactor primeAgent helper to reduce the size of a following diff.
6dd9ce1
@mjs @alesstimec
agent: SetAPIHostPorts now uses all candidate addresses for each server 
SetAPIHostPorts used to pick just one address for each API server. It
now writes all best-matched addresses (using the new
SelectInternalHostPorts).

Fixes LP #1497094.
92060df
@cmars @alesstimec
Log rpc Conn.loop deserialization errors.
d1a9a2e
@alesstimec
Delete unneeded json and yaml spaces cmd methods 
The spaces yaml and json marshalling methods do not seem to
actually be required. Deleting them causes no spaces cmd
tests to fail.
4da7d62
@tasdomas @alesstimec
Meter status worker ignore missing hook error.
e04650f
@tasdomas @alesstimec
Slight test refactoring. Log unknown hook errors.
2761406
@davecheney @alesstimec
cmd/juju/commands: refactor bootstrap_test 
Clean up bootstrap test a little to make it easier to merge my mega
change later.
9b89a7a
@mattyw @alesstimec
all: migration from gopkg.in/yaml.v1 to gopkg.in/yaml.v2
9a1e68e
@davecheney @alesstimec
cmd/juju/commands: refactor version.Current patching 
This PR will make it easier to land my followup that changes the defintion of
version.Current from a version.Binary to a version.Number.
6d50132
@howbazaar @alesstimec
Retry unlocking if it fails, longer acquire timeout, and explicit bre… 
…ak lock if time over expectations.
2b77d46
@jujubot
Merge pull request #3477 from rogpeppe/058-juju-names-changes 
update to use latest juju/names changes

all: update for latest names changes

The names.UserTag.Username method gets renamed to Canonical.
The names.UserTag.Provider method gets renamed to Domain.
The names.LocalProvider constant gets renamed to names.LocalUserDomain.

(Review request: http://reviews.vapour.ws/r/2866/)
56ad42b
@mattyw @alesstimec
constraints: errors.Trace the unmarshalling errors
356f1f7
@jujubot
Merge pull request #3513 from alesstimec/chicago-cubs-guibundles-mast… 
…er-updates-01

Updates from the latest blessed master to chicago cubs



(Review request: http://reviews.vapour.ws/r/2910/)
e3d29f6
@rogpeppe
apiserver: allow external users to validate using macaroon authentica… 
…tion
60af5db
@jujubot
Merge pull request #3512 from rogpeppe/061-external-users 
apiserver: allow external users to validate using macaroon authentication

We don't allow an external macaroon discharger to speak for any
locally created user. Since existing implementations do return
usernames with no domains, we automatically append a "@external"
suffix in that case so we can continue to work with them.


(Review request: http://reviews.vapour.ws/r/2909/)
44ecda4
@alesstimec
Fixes for cmd/plugins
05f48d6
@alesstimec
Housekeeping in the cmd/juju/commands package.
0a47a1a
@jujubot
Merge pull request #3522 from alesstimec/chicago-cubs-cmd-juju-comman… 
…ds-cleanup-01

Housekeeping in the cmd/juju/commands package.



(Review request: http://reviews.vapour.ws/r/2919/)
c71ce35
@jujubot
Merge pull request #3523 from alesstimec/chicago-cubs-cmd-plugins-fix… 
…es-01

Fixes for cmd/plugins



(Review request: http://reviews.vapour.ws/r/2920/)
4c0f1b7
@rogpeppe
cmd/juju/commands: fix deploy tests under windows
8baf5a6
@jujubot
Merge pull request #3524 from rogpeppe/062-fix-deploy-bundle-windows 
cmd/juju/commands: fix deploy tests under windows

Under Windows, if call os.Open("local:foo"), it fails not with an os.IsNotExist
error but with another error ("The filename, directory name, or volume label syntax is incorrect").

We change the heuristic to ignore any error caused by opening the local file,
and always fall back to loading from a charm repository in this case.

Also fix a regression of the error printed by juju status.

(Review request: http://reviews.vapour.ws/r/2921/)
750d682
@alesstimec
Charmstore uses api context's cookie jar.
72401b0
@jujubot
Merge pull request #3526 from alesstimec/chicago-cubs-csclient-fix-01 
Charmstore uses api context's cookie jar.



(Review request: http://reviews.vapour.ws/r/2923/)
c439fe0
@rogpeppe
cmd/envcmd: only make apiContext when we need one
f17d30a
@jujubot
Merge pull request #3548 from rogpeppe/063-lazy-apicontext 
cmd/envcmd: only make apiContext when we need one

This means that commands that don't make an API context will not
load and save the cookie jar, which was a problem in particular
for the juju plugins because any cookies a plugin was creating
were being overwritten by the juju wrapper.


(Review request: http://reviews.vapour.ws/r/2948/)
ed597ae
@rogpeppe
Merge remote-tracking branch 'origin/chicago-cubs' into HEAD 
Conflicts:
	apiserver/apiserver.go
	cmd/juju/commands/bootstrap_test.go
	dependencies.tsv
	environs/config/config.go
	worker/fortress/util_test.go
	worker/logsender/worker.go
	worker/logsender/worker_test.go

No obviously controversial conflicts.
d81276b

@rogpeppe rogpeppe changed the title from merge chicago cubs into master to Add macaroon authentication and bundle deployment (chicago-cubs merge) Oct 23, 2015

@mhilton
Member

mhilton commented Oct 23, 2015

LGTM
@rogpeppe
Owner

rogpeppe commented Oct 23, 2015

All commits have already been reviewed, and the feature branch is blessed, so merging without further ado.
@rogpeppe
Owner

rogpeppe commented Oct 23, 2015

$$merge$$
@jujubot
Contributor

jujubot commented Oct 23, 2015

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju
@jujubot
Contributor

jujubot commented Oct 23, 2015

Build failed: Generating tarball failed
build url: http://juju-ci.vapour.ws:8080/job/github-merge-juju/5206
@rogpeppe
Owner

rogpeppe commented Oct 23, 2015

$$merge$$
@jujubot
Contributor

jujubot commented Oct 23, 2015

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju
@jujubot
Contributor

jujubot commented Oct 23, 2015

Build failed:
build url: http://juju-ci.vapour.ws:8080/job/github-merge-juju/5208
@frankban
Member

frankban commented Oct 23, 2015

Spurious?
$$merge$$
@jujubot
Contributor

jujubot commented Oct 23, 2015

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju

jujubot added a commit that referenced this pull request Oct 23, 2015

@jujubot
Merge pull request #3590 from rogpeppe/064-merge-chicago-cubs-into-ma… 
…ster

Add macaroon authentication and bundle deployment (chicago-cubs merge)

This merges the chicago-cubs feature branch into trunk.
This adds macaroon authentication support (along
with much API server cleanup) and support
for bundle deployment.


(Review request: http://reviews.vapour.ws/r/2991/)
7cfd394

@jujubot jujubot merged commit 7cfd394 into juju:master Oct 23, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment