/juju

Feature resources merge master #4585

Merged
merged 110 commits into from Mar 1, 2016

Conversation

Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
9 participants
@kat-co @jujubot @tych0 @waigani @jameinel @cmars @wallyworld @davecheney @tasdomas
@kat-co
Contributor

kat-co commented Mar 1, 2016

(Review request: http://reviews.vapour.ws/r/4025/)

Tycho Andersen and others added some commits Jan 14, 2016

lxd: update config for latest version 
The LXD API has updated to allow config files to be loaded from arbitrary
locations on disk without mucking around with the environment (sorry about
that), which gets rid of some ugly code.

Additionally, there are some minor symbol signature updates as well.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
2012fe4
lxdclient: fix typo 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
c65e9f1
lxd provider: remove irrelevent documentation 
Since lxd-images is completely independent of images.linuxcontainers.org
(and linuxcontainers.org don't have cloud-init anyway), let's not have
people do this unnecessary step.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
57edf6a
lxd provider: use the right config path in all places 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
8e58356
update lxd client version to 0.27 
There is a breakage with version 0.27 such that older versions of the go
client don't behave correctly when setting configuration. Let's use 0.27.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
7c15515
lxc: get rid of custom container detection code 
The mechanism for detecting whether or not the task is in a container here
(checking if we're in the / cgroup) is very broken, and will not work with
cgroup namespaces (to land in kernel 4.5, but will be backported to
xenial's 4.4 kernel).

Instead, we should use the running-in-container script, as provided by the
init-system-helpers package, which encapsulates all the logic required for
various init systems. (n.b. that this adds a dependency; I'm not sure how
juju handles this.)

Also, I've mode the code to container/ instead of in lxc/lxcutils in
anticipation of the lxd container type which will also need to use this
function.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
24fef1a
lxdclient: get rid of spurious println 
I introduced this while debugging something, but didn't get rid of it.
Happy to squash this commit into the previous ones if necessary.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
b59d585
add a LXD container type 
This patch adds a LXD container type to juju. Note that currently it uses
the same image name schema that the lxd provider uses, i.e. ubuntu-$series.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
1441569
@tych0
switch default local container type to lxd 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
f733783
lxd: move lxdclient from /provider to /tools 
This is in preparation for adding the LXD container type.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
1acefbd
s/insideLXC/insideContainer 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
3a08b62
actually use the verifyLxd function 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
9ec4403
add backports logic for lxd 
This commit needs to be updated when juju/utils#192
is merged.

This assumes that
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1512219 has been fixed
in trusty, which I'm not sure it has. I'm trying to verify that now. If
not, we'll have to do something more complicated.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
8874a83
remove debugging 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
6bad3ff
add lxd container initialization tests 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
4c1ce6e
fix lxdclient test import path 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
fcafc4b
commonize container detection code 
Use the same container detection code for both LXC and LXD. Also add some
tests for this.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
9a74e10
lxd container type: remember to use the namespace 
If we don't do this everything works fine, but if you destroy your
juju environment, it destroys all the containers on the host. Instead,
let's not do that :)

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
12053bc
lxd container type: add some tests 
Here's a basic testcase for the previous commit. It also adds some plumbing
necessary to write more LXD tests.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
868e9b4
lxd: bump version to 2.0.0beta1 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
fc4f4c5
Fix gofmt complaints
3325b2f
Merge branch 'master' into lxd-master-merge 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
1e28322
restrict lxd container type to go1.3 
natefinch | tych0: the landing bot runs go 1.2, which means we have to mark
all LXD code as // +build go1.3  so it only builds with 1.3 or higher

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
7a21c18
add build directive to instance.go too 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
0f0d405
add go 1.2 stubs for LXD 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
9e70b80
use real build tag for <1.3 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
ce674ee
fix compiler error w/ go1.2 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
3b6c155
lxd: blacklist some tests on go1.2 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
21c9765
fix go fmt 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
495a787
@jujubot
Merge pull request #4314 from tych0/lxd-master-merge 
Lxd master merge



(Review request: http://reviews.vapour.ws/r/3756/)
2be2e05
lxd: bump version to i18n-less client 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
6c6bdfc
@jujubot
Merge pull request #4355 from tych0/bump-lxd-version 
lxd: bump version to i18n-less client

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>

(Review request: http://reviews.vapour.ws/r/3800/)
9df6015
Merge branch 'master' into lxd-container-type 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
19844cc
un-fuck tools/lxdclient after master merge 
For whatever reason git couldn't figure this out, so this is basically a
cherry-pick of 2a17adb on top of this
branch.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
59082d9
more un-fucking of master merge 
This is a cherry pick of 70c8b25 and
f8e3c5e.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
c42a58f
remove unused import 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
790522a
@jujubot
Merge pull request #4396 from tych0/lxd-container-type 
merge master to lxd-container-type



(Review request: http://reviews.vapour.ws/r/3838/)
7694111
@waigani
MADE storageprovisioner worker 
Move the storageprovisioner worker from the machine agent runner to the dependency engine.
1f19d36
@waigani
Merge branch 'MADE-workers' into MADE-storageprovisioner-machine
e5966ea
@jameinel
Merge remote-tracking branch 'upstream/master' into lxd-container-type
7616031
@jameinel
Update the lxdclient code to be compatible with 2.0.0beta3. 
There is one glaring ommision, namely that Ips is no longer accessible from ContainerInfo,
which was returned by ListContainers. So we'll have to figure out how to properly build
our Instances. But I'm submitting a request for more information. Everything else is
available.
2373852
@jujubot
Merge pull request #4495 from jameinel/lxd-container-type 
Lxd container type

Merge master and lxd-container-type. This updates the branch without changing anything in lxd-container-type.
I probably messed something up, cause it looks like I accidentally auto-pushed to juju/juju instead of jameinel/juju. But this should resolve it.


(Review request: http://reviews.vapour.ws/r/3927/)
8a6f739
@jameinel
Start removing the addresses code since we have to do it differently.
d494c07
@jameinel
Merge remote-tracking branch 'upstream/lxd-container-type' into lxd-c… 
…ontainer-type-2.0.0beta3
5ab2691
@jameinel
and now we're on to the broken stuff.
5b5917f
@waigani
Merge branch 'master' into MADE-storageprovisioner-machine
afb3d5f
@waigani
Merge branch 'master' into MADE-storageprovisioner-machine
bf80135
@waigani
MADE Identity File Writer 
Move the Identity File Writer worker from the machine agent runner to the dependancy engine.
c0439f8
@jameinel
It compiles against the proposed lxd branch. 
It doesn't pass any test suites. In fact, it segfaults. But it does compile.

This rips out all the code that dealt with writing to disk in order to get
certificates and configuration working with the LXD code.
The nice part is that it is removing a bunch of code. Bad part is it doesn't work yet.
fa3c8a1
lxd: use real upstream hash for new client API 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
22dd01d
lxd: really fix compilation :) 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
98c547a
lxd: fix segfault in tests 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
afd32d9
lxd: implement Addresses() call for lxd provider 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
517f93b
lxd: s/limits.cpus/limits.cpu 
This key was renamed a while ago.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
3d54800
lxd: fix parsing of limits.memory 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
640b8a0
lxd: get rid of lxdNewClient 
This is dead code.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
0317890
lxd: include default port in hostname 
This code is all kind of a clusterfuck, but when we generate the remote
address for a LXD, we need to include the default port as well, since we
set https_address to ::, which binds to 8443 by default.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
03ff20b
lxd: plumb the server cert through 
This adds a config option for the server cert and the plumbing to get it
through.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
e1de2e4
@jameinel
go fmt
7fb524d
@jameinel
Fix the lxdclient test suite. 
Rather than adding a port to 'Host' we'll add it late when trying to connect, and use
shared.DefaultPort instead of assuming we know it ahead of time.
b911cb9
@jameinel
Make client-key a secret attribute and fix server-cert to be present … 
…in the testing set.
64b503a
@jameinel
Fix the provider/lxd test suite. 
We now have passing lxdclient and provider/lxd test suites.
ServerCert needed to be properly set up in all the configuration,
and tests added that it can be empty, etc.
f39fce2
@jameinel
Move ServerPEMCert to an attribute of Remote. 
The Certificate for the Server is an attribute of the Remote host
that we are connecting to. Not an attribute of the Client itself.
0c1b988
@jameinel
Merge remote-tracking branch 'upstream/master' into lxd-container-typ… 
…e-no-disk
7031c85
@jameinel
Handle review feedback. 
Fixed a few things detected in review. Still need to handle the mem conversion bug.
29f0816
@jameinel
Respond to Eric Snow's review comments.
a9489f7
@jameinel
Merge remote-tracking branch 'upstream/master' into lxd-container-typ… 
…e-master
f5141b7
@jameinel
Don't run TestNotAllContainersAreDeleted right now. 
It connects to a real LXD server and creates real containers.
This should become a CI test.
c25804c
@jujubot
Merge pull request #4544 from jameinel/lxd-container-type-master 
merge juju master into lxd-container-type



(Review request: http://reviews.vapour.ws/r/3980/)
b1eff25
@jameinel
Fix newInstanceSummary to properly handle memory. 
Add a few tests so that we actually know it is doing the right thing.
affd09d
@jameinel
Merge remote-tracking branch 'upstream/lxd-container-type' into lxd-c… 
…ontainer-type-no-disk
2b8fec0
@jameinel
Actually really do add the tests this time.
8888629
@jameinel
go fmt
222cab6
@jujubot
Merge pull request #4537 from jameinel/lxd-container-type-no-disk 
Update lxd-container-type to support lxd-2.0.0~beta4

This should be all the steps we need to make Juju compatible with LXD 2.0.0~beta4.

This was done in collaboration with Tycho. I've tested it doing a local "juju bootstrap lxd lxd" on Trusty using the 2.0.0~beta4 packages from ppa:ubuntu-lxc/lxd-stable as well as bootstrapping an AWS controller and installing the ppa there as well before doing "juju deploy --to lxd". 

There may still need to be some tests fixed. containers/lxd still expects you to have installed an image into your local lxd server. But I think this is really close to being good enough to land in master.


(Review request: http://reviews.vapour.ws/r/3973/)
0d93a71
@cmars
Message listing prior agreements to terms for the deployment.
f03642a
@wallyworld
destroy model doesn't use legacy config store
d39d2ae
@jujubot
Merge pull request #4548 from wallyworld/destroy-model-regression 
destroy model doesn't use legacy config store

Fixes: https://bugs.launchpad.net/juju-core/+bug/1550580

juju destroy-model was broken because it still tried to use the legacy config store which we no longer use when creating models


(Review request: http://reviews.vapour.ws/r/3987/)
8d8648d
fix int overflows in lxdclient test 
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
b37733c
@jujubot
Merge pull request #4549 from tych0/fix-int-overflows 
fix int overflows in lxdclient test

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>

(Review request: http://reviews.vapour.ws/r/3988/)
f125ff2
@jameinel
Merge remote-tracking branch 'upstream/master' into lxd-container-typ… 
…e-master
f9e5e00
@jujubot
Merge pull request #4550 from jameinel/lxd-container-type-master 
merge master into lxd-container-type



(Review request: http://reviews.vapour.ws/r/3989/)
222bc21
@wallyworld
Fix typo in test
43b6f2c
@jujubot
Merge pull request #4551 from wallyworld/fix-detect-credentials-windows 
Fix typo in windows test

Wrong env var was patched, should have been USERPROFILE

(Review request: http://reviews.vapour.ws/r/3990/)
b5ee435
@jujubot
Merge pull request #4552 from juju/lxd-container-type 
Lxd container type

This branch is good to merge into master. It is currentish with master. The failures seen on
    http://reports.vapour.ws/releases/3676
are all from master. This branch fixes some of the errors in master, so it is desirable to merge now.
The one bug shown on
     https://bugs.launchpad.net/juju-core/lxd-container-type
is still valid and will affect master when merged. Both the gccgo and golang ppc64el unittests are already broken and need a fix.

(Review request: http://reviews.vapour.ws/r/3991/)
af8aef9
@davecheney
rpc: remove Conn.Go method 
Conn.Go was never called directly, but was exported in the package's
API. As `go` is a reserved word, this probably explains the method's
public name. Because the method was public, a lot of preconditions, some
panicing had to be set to ensure the method was not called incorrectly.

Remove all of this and collapse `Go` into its single caller, reducing
the API surface, and ensuring it is always called correctly.
3d0a910
@jujubot
Merge pull request #4553 from davecheney/rpc-cleanup 
rpc: remove Conn.Go method

Conn.Go was never called directly, but was exported in the package's
API. As `go` is a reserved word, this probably explains the method's
public name. Because the method was public, a lot of preconditions, some
panicing had to be set to ensure the method was not called incorrectly.

Remove all of this and collapse `Go` into its single caller, reducing
the API surface, and ensuring it is always called correctly.

(Review request: http://reviews.vapour.ws/r/3992/)
a70da22
@jujubot
Merge pull request #4391 from cmars/callout-agreements 
Message listing prior agreements to terms for the deployment.



(Review request: http://reviews.vapour.ws/r/3833/)
82f6e84
@waigani
review comments
1eaf596
@waigani
Merge branch 'master' into MADE-identity-file-writer
42087ea
@jujubot
Merge pull request #4516 from waigani/MADE-identity-file-writer 
MADE Identity File Writer

Move the Identity File Writer worker from the machine agent runner to the dependancy engine.

(Review request: http://reviews.vapour.ws/r/3951/)
310a863
@waigani
Merge branch 'master' into MADE-storageprovisioner-machine
ca45887
@jujubot
Merge pull request #4486 from waigani/MADE-storageprovisioner-machine 
Made storageprovisioner machine

Move the storageprovisioner worker from the machine agent runner to the dependency engine.

(Review request: http://reviews.vapour.ws/r/3919/)
3b9389d
@tasdomas
Make registration/allocation steps isolated in tests.
44e1ef3
@jujubot
Merge pull request #4566 from tasdomas/d005-registration-clients 
Make registration/allocation steps isolated in tests.

This addresses https://bugs.launchpad.net/juju-core/+bug/1550593
ecd9b0c
@davecheney
rpc: refactor error handling to use juju/errors 
Use the juju/errors package more consistantly in the rpc package to
provide better error stack traces.
11ead12
@wallyworld
Fix typo in Joyent regions
afe28ec
@jujubot
Merge pull request #4572 from wallyworld/fix-joyent-regions 
Fix typo in Joyent regions

Joyent regions us-east-1 and us-east-2 were misconfigured.

(Review request: http://reviews.vapour.ws/r/4010/)
96bab7a
@jujubot
Merge pull request #4571 from davecheney/rpc-refactor-error 
rpc: refactor error handling to use juju/errors

Use the juju/errors package more consistantly in the rpc package to
provide better error stack traces.

(Review request: http://reviews.vapour.ws/r/4009/)
896b265
@davecheney
apiserver/common: remove unused second argument from common.RestoreError 
None of the callers of this method used the value.
57c5c53
@davecheney
apiserver/params: remove dependency on rpc package 
There was only one constant that tied apiserver/params to the rpc
package, so copy the constant and add a test to make sure it doesn't
change in the future.

This means the millions of packages that import apiserver/params no
longer depend on the rpc package.
44987d3
1550770: Add DefaultSeries tools for current arch 
TestContainerStartedAndStopped starts a container with
the default series, but does not upload tools for non
amd64 arches.  As a result the tests fail because
tools cannot be found.

This patch uploads tools for the default series and
the current arch if the arch is not amd64 (as amd64
tools with the default series are already uploaded
in the base suite)
029e449
@jujubot
Merge pull request #4576 from davecheney/apiserver-common-remove-rest… 
…oreerror-bool-param

apiserver/common: remove unused second argument from common.RestoreError

None of the callers of this method used the value.

(Review request: http://reviews.vapour.ws/r/4014/)
de4f71c
@jujubot
Merge pull request #4575 from davecheney/apiserver-params-no-longer-d… 
…epends-on-rpc

apiserver/params: remove dependency on rpc package

There was only one constant that tied apiserver/params to the rpc
package, so copy the constant and add a test to make sure it doesn't
change in the future.

This means the millions of packages that import apiserver/params no
longer depend on the rpc package.

(Review request: http://reviews.vapour.ws/r/4013/)
01a0ddd
add-juju-bridge.py: python3.5 fixes 
In python 3.5 we get:

Traceback (most recent call last):
  File "./add-juju-bridge.py", line 372, in <module>
    main(arg_parser().parse_args())
  File "./add-juju-bridge.py", line 351, in main
    print_shell_cmd("echo cat {}".format(args.filename))
  File "./add-juju-bridge.py", line 279, in print_shell_cmd
    print(out.rstrip('\n'))

  TypeError: a bytes-like object is required, not 'str'

We fix this by first decoding the stream before calling rstrip().
6d33e03
add-juju-bridge.py: use long form arguments to ifquery 
This make the script compatible with precise; we were using '-l' for
list, but only '--list' is available on precise.
9728784
provider/maas: ensure bridge script works on xenial 
Xenial only has Python 3.5 and there is no /usr/bin/python only
/usr/bin/python3, however the bridge script was reliant on just python.

We now detect what python versions are available before invoking the
script, preferring python 2 over 3 as we don't want to invalidate lots
of existing testing with known cloud-image contents for series older
than xenial.

A summary of Ubuntu releases and the python version included in the
default install of Ubuntu Server is as follows:

 12.04 precise:  python 2 only (2.7.3)
 14.04 trusty:   python 2 only (2.7.5) and python3 (3.4.0)
 14.10 utopic:   python 2 (2.7.8) and python3 (3.4.2)
 15.04 vivid:    python 2 (2.7.9) and python3 (3.4.3)
 15.10 wily:     python 2 (2.7.9) and python3 (3.4.3)
 16.04 xenial:   python 3 only (3.5.1)
 going forward:  python 3 only

This has been tested by bootstrapping with precise (where some extra
fixes where required), trusty, wily and xenial.

Fixes [LP:#1550306](https://bugs.launchpad.net/juju-core/+bug/1550306)
6fc2013
Ensure bridge script tests use Python 2 and Python 3 
Explicitly look for /usr/bin/python2 and /usr/bin/python3 and ensure
unit tests are executed with each version of Python found on the testing
system. If no version of Python is found then the TestSetupSuite will
fail, failing all the tests.
f82e52a
@jujubot
Merge pull request #4582 from frobware/master-lp1550306 
provider/maas: ensure bridge script works on xenial

Xenial only has Python 3.5 and there is no /usr/bin/python only
/usr/bin/python3, however the bridge script was reliant on just python.

We now detect what python versions are available before invoking the
script, preferring python 2 over 3 as we don't want to invalidate lots
of existing testing with known cloud-image contents for series older
than xenial.

A summary of Ubuntu releases and the python version included in the
default install of Ubuntu Server is as follows:

 12.04 precise:  python 2 only (2.7.3)
 14.04 trusty:   python 2 only (2.7.5) and python3 (3.4.0)
 14.10 utopic:   python 2 (2.7.8) and python3 (3.4.2)
 15.04 vivid:    python 2 (2.7.9) and python3 (3.4.3)
 15.10 wily:     python 2 (2.7.9) and python3 (3.4.3)
 16.04 xenial:   python 3 only (3.5.1)
 going forward:  python 3 only

This has been tested by bootstrapping with precise (where some extra
fixes where required), trusty, wily and xenial.

Fixes [LP:#1550306](https://bugs.launchpad.net/juju-core/+bug/1550306)


(Review request: http://reviews.vapour.ws/r/4021/)
e530e94
@jujubot
Merge pull request #4579 from cherylj/create-trusty-tools-lxc-master 
1550770: Add DefaultSeries tools for current arch

TestContainerStartedAndStopped starts a container with
the default series, but does not upload tools for non
amd64 arches.  As a result the tests fail because
tools cannot be found.

This patch uploads tools for the default series and
the current arch if the arch is not amd64 (as amd64
tools with the default series are already uploaded
in the base suite)

(Review request: http://reviews.vapour.ws/r/4017/)
27e271c
@kat-co
Contributor

kat-co commented Mar 1, 2016

$$merge$$
@jujubot
Contributor

jujubot commented Mar 1, 2016

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju
@jujubot
Contributor

jujubot commented Mar 1, 2016

Build failed: Generating tarball failed
build url: http://juju-ci.vapour.ws:8080/job/github-merge-juju/6623
@kat-co
Contributor

kat-co commented Mar 1, 2016

$$merge$$
@jujubot
Contributor

jujubot commented Mar 1, 2016

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju

jujubot added a commit that referenced this pull request Mar 1, 2016

@jujubot
Merge pull request #4585 from kat-co/feature-resources-merge-master 
Feature resources merge master



(Review request: http://reviews.vapour.ws/r/4025/)
976c7c4

@jujubot jujubot merged commit 976c7c4 into juju:feature-resources Mar 1, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment