/juju

Merge develop into 2.0 #6510

Merged
merged 329 commits into from Oct 27, 2016

Conversation

Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
@kat-co
Contributor

kat-co commented Oct 27, 2016

No description provided.

anastasiamac and others added some commits Nov 12, 2015

@anastasiamac
Changed filter to a simple collection.
ba4b123
@anastasiamac
show-saas cli
602ba10
@anastasiamac
show-endpoints cli
42c1539
@anastasiamac
Added truncation for long desc.
5d02590
@anastasiamac
Wrap description in tabular format in addition to truncating.
c3ab597
@anastasiamac
Plugged in back-end
5bd8d84
@anastasiamac
Sorted conflicts, service r passed by name.
d00c35a
@anastasiamac
Removed redundant structures.
e07c56b
@anastasiamac
fixed imports.
7b76329
@anastasiamac
API func as property.
5b9c609
@anastasiamac
Removed false copy.
1d8ec79
@anastasiamac
RPC entry point. why me? :(
1ea82c8
@anastasiamac
Review comments.
4a495f9
@anastasiamac
Review comments.
9741bf9
@anastasiamac
Review comments.
639a7a9
@anastasiamac
Stubbed backend and fixed endpoint relation retrieval.
b7952dd
@anastasiamac
Review comments.
1933e4a
@anastasiamac
Review comments.
ad83c75
@anastasiamac
formatting.
b30ccc4
@anastasiamac
formatting.
643c74b
@anastasiamac
Removed no-longer needed mock.
1887c31
@jujubot
Merge pull request #3685 from anastasiamac/x-model-show-cli 
Cross model relations CLI - show

This proposal presents "show-endpoints" command that displays extended information about service's endpoints that have been offered.

Also, changed interactions with services layer to use interfaces rather than struct.



(Review request: http://reviews.vapour.ws/r/3090/)
389569d
@wallyworld
Code review fixes
af40913
@wallyworld
Make endpoints a map of internal->published, add update(), and improv… 
…e tests
26c852a
@wallyworld
Cleanup
12fe749
@jujubot
Merge pull request #3750 from wallyworld/service-offer-state 
Add state model for service offers

Add a collection, doc structs and APIs for persisting service offer intentions to state. A subsequent worker will process these and update the relevant service directory.

We rename a previous collection also.

(Review request: http://reviews.vapour.ws/r/3157/)
d9c61e7
@axw
state: add watchers for remote services 
Added two new methods for watching remote
services and relation unit endpoints:
 - State.WatchRemoteServices
 - Relation.WatchCounterpartEndpointUnits

The first one is necessary just so we can
watch remote services. Nothing special there.

WatchCounterpartEndpointUnits will be used
to watch the local units of a relation
involving a remote service. Because we won't
be tracking units of a remote service, we
cannot use the usual Unit -> RelationUnit ->
RelationUnit.Watch procedure. Instead, we
pass in the name of the remote service to
watch the counterpart units -- the units of
the local service involved in the relation.
4ae27b2
@wallyworld
Add offered services facade and offered service listener
1a20696
@jujubot
Merge pull request #3767 from axw/state-watchremote 
state: add watchers for remote services

Added two new methods for watching remote
services and relation unit endpoints:
 - State.WatchRemoteServices
 - Relation.WatchCounterpartEndpointUnits

The first one is necessary just so we can
watch remote services. Nothing special there.

WatchCounterpartEndpointUnits will be used
to watch the local units of a relation
involving a remote service. Because we won't
be tracking units of a remote service, we
cannot use the usual Unit -> RelationUnit ->
RelationUnit.Watch procedure. Instead, we
pass in the name of the remote service to
watch the counterpart units -- the units of
the local service involved in the relation.

(Review request: http://reviews.vapour.ws/r/3175/)
56d349e
@wallyworld
Code review cleanup
f12cc4b
@wallyworld
Better error handling
5c47776
@jujubot
Merge pull request #3769 from wallyworld/offered-service-listener 
Add offered services facade and offered service listener

We add a state watcher for offered services. Then, a new OfferedServices facade is added to expose this watcher plus provide a method to retrieve offered services which the listener informs about.

(Review request: http://reviews.vapour.ws/r/3177/)
b8594c7
@wallyworld
Merge branch 'master' into merge-master-2011
14fd861
@wallyworld
Fix tests
66bf79f
@jujubot
Merge pull request #3784 from wallyworld/merge-master-2011 
Merge master 2011

Merge latest master.

(Review request: http://reviews.vapour.ws/r/3193/)
40210ee
@axw
apiserver/remoterelations: add facade 
This is part of a new facade that will provide
access to state for an upcoming "remote relations"
worker. The worker will be responsible for
transferring settings between environments involved
in cross-model relations.

The facade is incomplete, but implements a
complex watcher that deserves isolated review.
There are two watchers: one that returns the
IDs of remote services, and one that watches
for any changes to a remote service, or any
of its counterpart relation units.

Tests have been added to featuretests that exercise
the watcher with a real State. Because the client-
side API has not been implemented yet, it is
using the facade methods directly. This will be
changed to call the client methods over the
API server when available, and then later
changed to be exercised via the worker when it
is implemented.
7aef7c1
@axw
apiserver/remoterelations: add comments
f573444
@axw
Add new facade versions
28e4ebf
@jujubot
Merge pull request #3779 from axw/apiserver-remoterelations 
apiserver/remoterelations: add facade

This is part of a new facade that will provide
access to state for an upcoming "remote relations"
worker. The worker will be responsible for
transferring settings between environments involved
in cross-model relations.

The facade is incomplete, but implements a
complex watcher that deserves isolated review.
There are two watchers: one that returns the
IDs of remote services, and one that watches
for any changes to a remote service, or any
of its counterpart relation units.

Tests have been added to featuretests that exercise
the watcher with a real State. Because the client-
side API has not been implemented yet, it is
using the facade methods directly. This will be
changed to call the client methods over the
API server when available, and then later
changed to be exercised via the worker when it
is implemented.

(Review request: http://reviews.vapour.ws/r/3187/)
82b9a5f
@axw
api/remoterelations: add API client 
Add the API client, and fix various bugs in the
apiserver code found with end-to-end testing.
Change the featuretests to call through the
API.

params.ServiceRelationsChange had to be changed
because JSON cannot represent a map with integer
keys.
7ba21a4
@axw
Fix copy/paste
71c4d64
@jujubot
Merge pull request #3796 from axw/api-remoterelations 
api/remoterelations: add API client

Add the API client, and fix various bugs in the
apiserver code found with end-to-end testing.
Change the featuretests to call through the
API.

params.ServiceRelationsChange had to be changed
because JSON cannot represent a map with integer
keys.

(Review request: http://reviews.vapour.ws/r/3205/)
9a0f234
@anastasiamac
CLI output.
8e0ce57
@anastasiamac
whitespace.
2ede4c0
@anastasiamac
added comment.
dd7f4b8
@anastasiamac
added comment.
cadd877
@anastasiamac
Review comments.
00e5c4f
@anastasiamac
Added api layer and tests.
3899156
@anastasiamac
Fixed test.
d14c5c3
@anastasiamac
Added apiserver layer and tests.
201cc7d
@anastasiamac
Type aliases - wow :D
16d7eb0
@anastasiamac
Stub usage.
136bb37
@anastasiamac
Bulk calls catering.
7eee1f9
@anastasiamac
Added missing comments.
5413333
@anastasiamac
doc.
a67701c
@anastasiamac
Fixed fmt usage and removed unused struct.
46b97da
@anastasiamac
comment fix.
9e6938a
@anastasiamac
Renamed filter and related.
b0aabf2
@anastasiamac
changing alias
63866f1
@anastasiamac
tags corrections.
190449b

wallyworld and others added some commits Oct 20, 2016

@wallyworld
Merge pull request #6478 from wallyworld/more-crossmodel-featureflag 
Add cross-model feature flag checks to more places to avoid trying to invoke missing functionality when the flag is not set.
c211c22
@wallyworld
Fix issue when merging develop branch
7766b8a
@wallyworld
Merge pull request #6479 from wallyworld/fix-develop-merge-issue 
Fix issue when merging develop branch
ce10bd3
@wallyworld
Fix develop merge issue
bf24f20
@wallyworld
Merge pull request #6480 from wallyworld/fix-api-network-call 
Fix develop merge issue
71abb0d
network: Reworded NoAddressError() message slightly
c28c927
apiserver: Allow SSHClient API v2 methods during upgrade
928a4b4
juju ssh, scp, and debug-hooks using SSHClient API v2
d25d100
apiserver/client: Fixed NoAddressError-related tests
eda554e
@jujubot
Merge pull request #6473 from juju/cross-model-relations 
Merge cross model relations feature branch

This PR merges the cross model relations feature branch. The vast majority of the work has been reviewed previously for landing into the feature branch.

The main commit to look at is:
dbf2140

The above commit adds the cross model feature flag to hide the cross model relations features - CLI, facades, and database collections.

As a side effect, the work in the feature branch also fixes:
https://bugs.launchpad.net/juju/+bug/1621754

QA:
bootstrap and use a 2.0.0 GA client to deploy and relate services
bootstrap and ensure cross model stuff works with feature flag
6bb1eb5
@frankban
juju gui command: retrieve the API host from the controller connection.
f6559bf
@reedobrien
List models to output cloud/region in leiu of owner
1ae94ff
@jujubot
Merge pull request #6485 from reedobrien/f/show-cloudregion-juju-models 
F/show cloudregion juju models

This changes the juju models output to output cloud/region in place of the Owner column.

QA: 
1. Run unit tests  and the all pass
2. bootstap a controller 
3. add another model in another region, 
4. juju models
```
Controller: reed/aws/us-west-1

Model       Cloud/Region       Status     Machines  Cores  Access  Last connection
controller  awstest/us-west-1  available         1      2  admin   just now
default     awstest/us-west-1  available         0      -  admin   1 hour ago
east1*      awstest/us-east-1  available         0      -  admin   6 minutes ago
```
46edd89
@axw
provider/ec2: regenerate instance types 
Regenerate instance types from the current price list,
bringing in us-east-2 data and some additional instance
types for other regions.
This commit was signed with a verified signature.
@axw axw Andrew Wilkins
GPG key ID: 55314F7628952B05 Learn about signing commits
ffc98ca
@anastasiamac
Fixes lp#1634405: Clarify 'welcome to juju 2' message.
d7d2111
@anastasiamac
Remove controversial "directory" wording to avoid bike-shedding.
28799c3
@anastasiamac
Changed to use template to avoid repeating same value many times in f… 
…ormatter like printf.
dabf85e
@anastasiamac
Fixes lp#1634770: openstack bootstrap panics with identity endpoint w… 
…ithout version.
272b270
@jujubot
Merge pull request #6487 from anastasiamac/fix-welcome-message-lp1634405 
Fixes lp1634405: welcome message juju-1 usage.
e09500c
@anastasiamac
Just worry about collection length to ensure that index is valid \o/
4657166
Clarifications after review
ee8e284
@anastasiamac
Also cater for other version speification such as case-insensitivity;… 
… missing /v specifying; trailing / and everything in between.

Updated tests.
c12a75d
@anastasiamac
Version is optional.
5e1e363
@jujubot
Merge pull request #6481 from dimitern/lp-1616098-ssh+scp-alladdresses 
lp:1616098: ssh,scp,dh pick reachable address among all

Uses the new SSHClient API facade v2 (from #6468), if
available, and network.ReachableHostPort() (#6454) in
code shared between the ssh, scp, and debug-hooks CLI
commands, to select a reachable address from all known
addresses of the target (machine/unit/host).

Fixes http://pad.lv/1616098.

QA steps (live tested on both MAAS 1.9 with multiple-
NIC nodes and on lxd - see below):

  1. Using the tip of `develop`, bootstrap a lxd xenial controller.
  2. juju switch controller
  3. juju status 0    # note of the IP address (IP1)
  4. lxc exec \<juju-instance-id\> -- ip addr add \<IP2\>/24 dev eth0
(add an extra IP on machine-0, from the same subnet)
  5. lxc exec \<juju-instance-id\> -- systemctl restart jujud-machine-0.service
(bounce the controller agent to pick up the new IP)
  6. juju ssh 0             # should work
  7. sudo ip route add blackhole \<IP1\>/32
(make the preferred public IP unreachable from your laptop)
  8. juju --debug ssh 0     # should fail with 'Invalid argument'
  9. git checkout dimitern/lp-1616098-ssh+scp-alladdresses
(switch to this PR's branch and rebuild juju & jujud)
  10. juju ssh 0                    # should work now, using IP2
  11. juju scp 0:/etc/issue .  # as above
  12. juju deploy ubuntu --to 0
  13. juju debug-hooks ubuntu/0 # should work as well
(small delay ~10s observed on lxd, because of 'sudo')
dd11dad
@jujubot
Merge pull request #6486 from axw/ec2-useast2-instancetypes 
provider/ec2: regenerate instance types

Regenerate instance types from the current price list,
bringing in us-east-2 data and some additional instance
types for other regions.

**QA**

1. bootstrap aws/us-east-2
2. bootstrap aws/ap-southeast-2 --constraints instance-type=x1.16xlarge
(Latter fails with "You have requested more instances (1) than your current instance limit of 0 allows for the specified instance type.",
but the fact that Juju passes on the request is proof that we know about the instance type.)
b4298cc
@jujubot
Merge pull request #6488 from anastasiamac/openstack-version-panic-lp… 
…1634770

Fixes lp1634770: openstack bootstrap panics with no version specified.

We panic on bootstrap, when openstack identity URL has no version.
For example, given cloud definition (from the bug):

  nuc03-openstack:
    type: openstack
    auth-types: [userpass]
    regions:
      RegionOne:
        endpoint: http://192.168.20.218:5000/

QA:
1. Setup openstack cloud definition without version in identity URL
2. juju bootstrap nuc03-openstack
3. [no panic]

As a drive-by, we also check if version specified was well-formed.
Any of the variations of these following forms of identity URL should produce user-friendly message helpful to correct endpoint URL:
* "https://keystone.internal/a"
* "https://keystone.internal/v"
* "https://keystone.internal/V
* "https://keystone.internal/V/"
* "https://keystone.internal/100"
* "abc123"
* "https://keystone.internal/vot"
389b797
@anastasiamac
Update welcome message with link to manual page describing how to get… 
… back to Juju 1.
49361d0
@anastasiamac
Clarify "awkward" message.
4a21044
@jujubot
Merge pull request #6489 from anastasiamac/welcome-msg-fix-1634405 
Fixes lp#1634405: Juju 2 welcome message update.

Update welcome message with:
    * link to manual page describing how to get back to Juju 1;
    * use 'juju bootstrap" command as an example instead of "juju switch" which is confusing to users.

*QA*
Welcome message should read:

Welcome to Juju 2.0.1. 
    See https://jujucharms.com/docs/stable/introducing-2 for more details.

If you want to use Juju 1.25.0, run 'juju' commands as 'juju-1'. For example, 'juju-1 bootstrap'.
   See https://jujucharms.com/docs/stable/juju-coexist for installation details.

Since Juju 2 is being run for the first time, downloading latest cloud information.
1479935
@perrito666
Refactor vsphere provider. 
The provider for vsphere was using the wrong aproach to
connect never logging out and it was leaking 45 conn/min
Now all connections login/logout per request.
397d43e
@perrito666
Fixed possible nilptr derefs. 
There where a lot of unsafe pointer dereferences and are
now nil checked.
e162148
@perrito666
Use a govmomi that is go 1.6 compatible
64c19a6
@jujubot
Merge pull request #6490 from perrito666/fix_1629458 
Refactor vsphere provider.

The provider for vsphere was using the wrong aproach to
connect never logging out and it was leaking 45 conn/min
Now all connections login/logout per request.

### QA Steps
Open the VSPhere Management Console.
Click on vCenter Inventory List --> Resources -- vCenter Servers --> your server ip
Go the right and click on Manage --> Sessions
Bootstrap a vsphere provider juju and deploy a few services.
There should NOT be 45 new open/idle processes every minute, instead numbers of 1 or 2 is expected.
805703b
@howbazaar
Add endpoint bindings to the application.
457e121
@howbazaar
Handle endpoint bindings in the state import/export code.
cc04362
Fixed lp:1635464: infinite recursion when validating mongo space set … 
…by the peergrouper
5689c57
@dooferlad
Bootstrap config: inherited overwrites defaults. 
Fixes an error in the bootstrap logic where a default from a cloud would overwrite values read from the users cloud config.

The bootstrap run logic has been split into several functions to aid readability and testing. The logical changes are actually very small and are contained within getBootstrapConfigs, which now builds up configuration by first combining all sources in order of lowest to highest precedence and then splitting the configuration out into bootstrap config, controller config and bootstrap model config. The old logic sort of did this, but it was muddled and had some precedence problems that were causing the fixed bugs.

Fixes http://pad.lv/1614239 and http://pad.lv/1619808
9577e37
@jujubot
Merge pull request #6482 from frankban/gui-addr-from-controller 
juju gui command: retrieve the API host from the controller connection.

This will allow us to properly handle HTTP requests for the GUI from alternative controllers.
125eb3b
@jujubot
Merge pull request #6495 from dimitern/lp-1635464-peergrouper-infrec 
Fixed lp:1635464: infinite recursion in worker/peergroper

When validating the "mongo" space name (common among all controllers),
the way stateShim.Space() was implemented in worker/peergrouper/shim.go
caused infinite recursion when called. This fixes http://pad.lv/1635464.

While I couldn't reproduce this normally, it's easy to trigger the case
when state.ControllerInfo.MongoSpaceState is "valid" by modifying the
mongo controllers collection directly.

QA steps (if you do those same steps without the fix, the bug happens):
 1. juju bootstrap vmaas-21 m21
 2. juju switch controller
 3. juju ssh 0
 4. ubuntu@maas-21-node-0:~$ sudo grep 'fatal error: stack overflow' /var/log/juju/machine-0.log
    # verify no matches are found initially 
 5. ubuntu@maas-21-node-0:~$ export STATE_PASSWORD=$(sudo grep statepassword: /var/lib/juju/agents/machine-0/agent.conf | cut -b16-)
 6. ubuntu@maas-21-node-0:~$ sudo /usr/lib/juju/mongo3.2/bin/mongo 127.0.0.1:37017/admin --ssl --sslAllowInvalidCertificates --sslPEMKeyPassword /var/lib/juju/shared-secret --sslPEMKeyFile /var/lib/juju/server.pem -u machine-0 -p $STATE_PASSWORD
 7. Run the following commands in the mongo shell:
    juju:PRIMARY> use juju;
    juju:PRIMARY> db.controllers.update({_id:"e"}, {"$set": {"mongo-space-name": "space-0", "mongo-space-state": "valid"}})
    # output: WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
    juju:PRIMARY> exit;
 8. ubuntu@maas-21-node-0:~$ sudo grep 'selecting mongo peer hostPort in space space-0 from' /var/log/juju/machine-0.log
    # verify the peergrouper picked up the change (>1 matches expected)
 9. ubuntu@maas-21-node-0:~$ sudo grep 'fatal error: stack overflow' /var/log/juju/machine-0.log
    # verify the bug does not occur after the changes
5983ff2
@alesstimec
Sorted output of 'juju metrics'.
aa69e14
Fixed bash completions
dc7eda5
Changes after review
2c4215a
@dooferlad
Convert some return values intro structs for better clarity. 
Some formatting changes.
5890aa9
@hmlanigan
Update Openstack Provider related to goose endpoint version and tests… 
…ervice changes
cd3c008
@wallyworld
Move some crossmodel facdes to controller
f879362
@jujubot
Merge pull request #6493 from wallyworld/crossmodel-controller-facades 
Move some crossmodel facades to controller

The main purpose here is to move some of the cross model facades to controller facades.
There's a bunch of followup work to be done to move from what was implemented originally to the latest spec. Part of that is done my necessity here. eg remove AllowedUsers from some structs; also start removing code to show offered applications as these will be removed. These changes are only visible with cross-model-relations feature flag.

QA: bootstrap with feature flag, deploy and offer a service, list and show endpoints
aa7b92c
@jujubot
Merge pull request #6497 from alesstimec/fix-juju-metrics-ordering 
Sorted output of 'juju metrics'.

QA instructions:
- deploy charm that reports multiple metrics
- run 'juju metrics <service name>'
- metrics in the output should be sorted by the metric name
64ad967
@axw
Update tests to close all test services
115ac77
@howbazaar
Migrate the cloud credentials.
b8b1ba9
@jujubot
Merge pull request #6499 from dimitern/lp-1622775-bash-completion-tra… 
…ceback

Fixed bash completions

Makefile's install-etc was not installing juju-version
Changed the completions to work for a juju binary, built
from source, as well as the one from the juju-2.0 package.

See bug http://pad.lv/1622775 for more details
(PR was intended to fix it, but simply improves CLI UX)

QA steps:
 1. make install-etc && source /etc/bash_completion
 2. juju boo[tab]tstrap
 3. juju de[tab]p[tab]loy ubuntu
 4. juju status [tab]u[tab]buntu/0
c51929f
@axw
provider/openstack: update goose 
Use new version of goose, to omit the device
field on volume attachment requests when no
device is specified.

Fixes https://bugs.launchpad.net/juju/+bug/1636648
f521854
@jujubot
Merge pull request #6491 from howbazaar/migrate-endpoint-bindings 
Migrate endpoint bindings

Applications can bind endpoints to particular spaces.

This work also fixes an issue with migrated models where there was no endpoint binding doc created for migrated applications, that would cause an assertion failure when upgrading a charm.
2613ad4
@mjs
cmd/juju/command: Fix SSH/SCP test failures 
A recent change updated the unit tests for `juju ssh` and `juju scp` in
such a way that they weren't deterministic any more and frequently
failed. This change limits the fake machine addresses that the tests
will use to make them deterministic and reliable again.
73e27cf
@jujubot
Merge pull request #6502 from mjs/ssh-test-fixes 
cmd/juju/command: Fix SSH/SCP test failures

A recent change updated the unit tests for `juju ssh` and `juju scp` in such a way that they weren't deterministic any more and frequently failed. This change limits the fake machine addresses that the tests
will use to make them deterministic and reliable again.

### QA 

Several hours of race stress test runs. Previously these tests would fail in a handful of iterations.
1d19099
@jujubot
Merge pull request #6501 from axw/lp1636648-openstack-update-goose 
provider/openstack: update goose

Use new version of goose, to omit the device
field on volume attachment requests when no
device is specified.

This branch also uses the new OpenStack API
version discovery code in goose.

Fixes https://bugs.launchpad.net/juju/+bug/1636648
e9ee950
@jujubot
Merge pull request #6496 from dooferlad/staging-config-inherit-fix 
Bootstrap config: inherited overwrites defaults.

Fixes an error in the bootstrap logic where a default from a cloud would overwrite values read from the users cloud config.

The bootstrap run logic has been split into several functions to aid readability and testing. The logical changes are actually very small and are contained within getBootstrapConfigs, which now builds up configuration by first combining all sources in order of lowest to highest precedence and then splitting the configuration out into bootstrap config, controller config and bootstrap model config. The old logic sort of did this, but it was muddled and had some precedence problems that were causing the fixed bugs.

Fixes http://pad.lv/1614239 and http://pad.lv/1619808

# CI Steps
Bootstrap an OpenStack with config: use-floating-ip: true and check that it is honored. This needs to be set in clouds.yaml, not on the CLI to see the bug fix.

# Notes to reviewers
The github diff viewer isn't being helpful, so open bootstrap.go and look at Run (line 314+) that used to be insanely big and make sure that it reads well. The only logical change is in getBootstrapConfigs (line 729+) where the old code was wrong. The other changes in bootstrap.go are just code being moved into functions and, otherwise, being left unaltered.
bebd627
@kat-co
Contributor

kat-co commented Oct 27, 2016

$$merge$$
@jujubot
Contributor

jujubot commented Oct 27, 2016

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju

@jujubot jujubot merged commit 8fcc982 into juju:2.0 Oct 27, 2016
1 check passed

1 check passed

github-check-merge-juju Built PR, ran unit tests, and tested LXD deploy.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment