/juju

Merge branch 'master' into db-log (again) #1857

Merged
merged 98 commits into from Mar 17, 2015

Conversation

Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
10 participants
@mjs @ericsnowcurrently @martinpitt @xnox @howbazaar @anastasiamac @waigani @wallyworld @axw @jujubot
@mjs
Contributor

mjs commented Mar 17, 2015

To deal with conflicts with master.

(Review request: http://reviews.vapour.ws/r/1183/)

anastasiamac and others added some commits Feb 17, 2015

@anastasiamac
Storage list and show commands stack.
a9977f9
@anastasiamac
Resolved merge conflicts.
e63560a
@anastasiamac
Merge remote-tracking branch 'origin/cmd-storage' into cmd-storage 
Conflicts:
	apiserver/storage/storage_test.go
	featuretests/storage_test.go
9dc84e0
@anastasiamac
Changed to suit instance/attachments model and other review comments.
4ec3303
@anastasiamac
Created a new struct to return instances and attachments.
7fede6e
@anastasiamac
Resolving merge conflicts.
c980368
@anastasiamac
Merge remote-tracking branch 'origin/cmd-storage' into cmd-storage 
Conflicts:
	state/storage.go
	state/storage_test.go
1bf1689
@anastasiamac
Addressing review comments.
59a7cab
@waigani
State: Environment.Users() 
Add a Users() method on state.Evironment which lists all the users for
that environment. Fix a bug in factory.MakeEnvUser which made it
create two envUsers instead of one.
e6aae1b
@waigani
Merge branch 'master' into cmd-environ-users
c4bdf51
@waigani
Merge branch 'master' of https://github.com/juju/juju into cmd-enviro… 
…n-users
8321774
@waigani
APISERVER: new EnvUserInfo method 
- Add apiserver.client.EnvUserInfo() which returns information on all
users for the environment.

- Move environment related params to params/envionment.go

- Make state.AddEnvironmentUser take a displayName so that it can be set
for a remote user (envUsers with a local counterpart just inherit
their displayName).

- Improve/tidy up comments and fix bugs with envUser.DisplayName() and
envUser.LastConnection()
f654f62
@waigani
Merge branch 'master' of https://github.com/juju/juju into cmd-enviro… 
…n-users
17b7338
@waigani
CMD: Environment Share 
Add Environment Share sub-command and corresponding tests, including
feature test ensuring CMD > API > APIServer > State is hooked up.
Allow adding a user to an environ twice, logging the APIServer error
that a user already exists as a warning. Change feature flag
feature.MESS to feature.JES.

(Review request: http://reviews.vapour.ws/r/1014/)
bf1ce95
@waigani
Merge branch 'jes-cli' of https://github.com/juju/juju into cli-share… 
…-environment

Conflicts:
	cmd/juju/environment/environment.go
	cmd/juju/environment/export_test.go
e115ad1
@howbazaar
Merge pull request #1747 from waigani/cli-share-environment 
CMD: Environment Share
f40864a
@waigani
CMD: environment unshare 
Add "juju environment unshare" sub-command and corresponding tests in
cmd, api and featuretests. When unshare is called on a user that has
not been added to an environment, the APIServer errors but the CLI
only logs a warning - as the desired outcome has still been reached:
the user does not have access to the environment.

(Review request: http://reviews.vapour.ws/r/1037/)
f6deacb
@howbazaar
Merge pull request #1748 from waigani/cli-unshare-environment 
CMD: environment unshare
388d07f
@waigani
Merge branch 'jes-cli' of https://github.com/juju/juju into apiserver… 
…-environ-users
c4b68c2
@howbazaar
Merge branch 'master' into jes-cli
28d520c
@waigani
Merge branch 'jes-cli' of https://github.com/juju/juju into cmd-envir… 
…on-users
63f1553
@howbazaar
Merge pull request #1749 from waigani/cmd-environ-users 
State: Environment.Users()
9e806a4
@waigani
Merge branch 'jes-cli' of https://github.com/juju/juju into apiserver… 
…-environ-users
4c35b4c
@howbazaar
Merge pull request #1750 from waigani/apiserver-environ-users 
APISERVER: new EnvUserInfo method
f43d3d4
@anastasiamac
Merge branch 'master' into cmd-storage 
Conflicts:
	state/storage_test.go
1ffcb43
@anastasiamac
Merge branch 'cmd-storage' of https://github.com/anastasiamac/juju in… 
…to cmd-storage

Conflicts:
	state/storage_test.go
aafe478
@anastasiamac
Updated to match current storage implementation in state.
80ffa96
@anastasiamac
go fmt
7ba568b
@waigani
API: New api.client.EnvironmentUserInfo 
Add a new method on the API client: EnvironmentUserInfo(). Test the
facade call. Add an end to end feature test to make sure API >
APISERVER > State are hooked up.
2007955
@howbazaar
Merge pull request #1774 from waigani/api-environ-users 
API: New api.client.EnvironmentUserInfo
4bf1f50
@anastasiamac
Fixing tests.
cb65703
@anastasiamac
go fmt
bc89b4c
@anastasiamac
Rephrased comment.
4b23f09
@anastasiamac
Changed json optionality for some properties.
490069f
@anastasiamac
Made text value consistent.
925cd9f
@anastasiamac
Sorted out mess with param names and return types.
87ecd36
@anastasiamac
Clarified comment.
01e59be
@anastasiamac
Changed var to const.
223bfad
@wallyworld
Update lxc config to allow mounting loop devices
095188c
@wallyworld
Change lxc storage config so that we ony enable mounting loop devices… 
… if the container is required to do so according to the volumes specified at creation time
9373cdd
@wallyworld
Explicitly specify loop devices to mount in lxc
7c436e6
@wallyworld
Add env config setting to force user to explicitly allow lxc loop mounts
ccac8a1
@wallyworld
Revert explicit block device enumeration
d4771d4
@anastasiamac
Changed attached and provisioned to be string status rather than bool.
5eb18fa
@anastasiamac
Changed attached and provisioned to be string status rather than bool.
11b11d6
@wallyworld
Fix typo
0a7500b
@anastasiamac
Changed table content
ac8af48
@anastasiamac
Simplified sort implementation.
325852a
@anastasiamac
Simplified sort implementation.
870bd34
@anastasiamac
Merge branch 'master' into cmd-storage 
Conflicts:
	apiserver/params/storage.go
b576491
@anastasiamac
Merge branch 'cmd-storage' of https://github.com/anastasiamac/juju in… 
…to cmd-storage

Conflicts:
	apiserver/params/storage.go
6ff3cb0
@wallyworld
Add code comment
8dff45f
@anastasiamac
Added comments and changed to use status.
38b2642
@anastasiamac
Added comments.
275dd81
@anastasiamac
Added comments.
b52245f
@anastasiamac
Corrected vars to consts.
bcf95ab
@anastasiamac
Removed owner from output, added status to list table, added natural … 
…suffix sort.
ea80e84
@anastasiamac
Cleaned up comment.
4408b92
@anastasiamac
Changed status to be string.
4da5e93
@ericsnowcurrently
Use a script that identifies the init system.
2ae93fe
@ericsnowcurrently
Fix the environs/manual tests.
cf1c500
@wallyworld
A couple of clean ups
cc860e6
@anastasiamac
Merge branch 'master' into cmd-storage 
Conflicts:
	apiserver/params/storage.go
69f20f7
@anastasiamac
Merge branch 'cmd-storage' of https://github.com/anastasiamac/juju in… 
…to cmd-storage

Conflicts:
	apiserver/params/storage.go
a72f5ea
@ericsnowcurrently
Return a string from ListServicesScript.
ba95e07
@ericsnowcurrently
Fix environs/manual.
83bcce7
@ericsnowcurrently
Use a bash function to cehck the init system.
ee28c53
@ericsnowcurrently
Clean up newShellSelectCommad.
30956ce
@axw
state: copy Pool from params to info on SetXInfo 
Set{Volume,Filesystem}Info will now copy the Pool
from Params to Info the first time it's called for
a volume/filesystem.
6cece04
@axw
Misc. fixes/changes to storage
0cbc336
@axw
worker/uniter/storage: fix source/watcher usage
b82e0a1
@axw
{api,apiserver}/storageprovisioner: attachments
c5b73a6
@axw
apiserver/storageprovisioner: VolumeAttachments
e72e12c
@axw
storageprovisioner: VolumeAttachmentParams
0c572d0
@axw
storageprovisioner: SetVolumeAttachmentInfo
5da73ba
@axw
storageprovisioner: AttachmentLife
6ffc6f0
@axw
worker/storageprovisioner: adjust
ef359d1
@axw
apiserver/diskformatter: adjust
eb5c3e6
@axw
Fixes
929d42b
@jujubot
Merge pull request #1611 from anastasiamac/cmd-storage 
Storage list and show commands stack.

This PR has:
    * updates to storage show command as well as supporting api client;
    * new storage list command as well as supporting api client and state implementations.




(Review request: http://reviews.vapour.ws/r/941/)
13914ab
@ericsnowcurrently
Use a const.
38afb9b
@ericsnowcurrently
Drop unnecessary comments.
a3ffa7b
@wallyworld
Fix test
8ddaa37
@mjs
api: extract connection logic out of Open 
The code that establishes a websocket connection to one of the API
servers as been extracted out of Open into a separate Connect
func. Most of the tests for Open were actually exercising this part of
Open so they have been adjusted to match.

This will soon be used by the database logging worker.
1c11b43
@howbazaar
Merge master.
cb070cf
@jujubot
Merge pull request #1846 from ericsnowcurrently/fix-1432421-discover-… 
…shell-select

Use a better approach to shell-based discovery of the init system.

(fixes https://bugs.launchpad.net/juju-core/+bug/1432421)

The existing approach simply hard-coded a mapping of PID 1 executable to init system.  However, this doesn't work when PID 1 is running via a symlink (e.g. /sbin/init).  This patch addresses that by using a separate script based on the discovery mechanism in discoverLocalInitSystem.

(Review request: http://reviews.vapour.ws/r/1173/)
8ed7cb6
@howbazaar
Merge branch 'jes-cli' of github.com:juju/juju into jes-cli
7df4510
@mjs
api: use errors package more
75ff768
@mjs
api: rearrange and document Connect to make it easier to follow
569f25c
@wallyworld
Fix whitespace in tests
aec8123
@axw
Address review comments
ec44fe4
@jujubot
Merge pull request #1853 from mjs/api-open-refactor 
Refactoring of api.Open()

The code that establishes a websocket connection to one of the API servers as been extracted out of Open into a separate Connect func. Most of the tests for Open were actually exercising this part of
the function so they have been adjusted to match. Connect will soon be used by the database logging worker.

Also: use the errors package more

Also: Rearrange and document API connection logic to make it easier to follow

(Review request: http://reviews.vapour.ws/r/1179/)
8591069
@waigani
Testing: fix failing test 
Make TestMakeEnvUserParams pass by setting the envUser's DisplayName.
7c18399
@howbazaar
Merge pull request #1856 from waigani/fix-factory-test 
Testing: fix failing test
6aa815f
@jujubot
Merge pull request #1826 from wallyworld/lxc-loop-mount 
Update lxc config to allow mounting loop devices

We use the "lxc-container-default-with-mounting" appamor profile to allow loop devices to be mounted.

(Review request: http://reviews.vapour.ws/r/1154/)
d6b2414
@jujubot
Merge pull request #1850 from axw/storageprovisioner-api-attachments 
storageprovisioner: attachments in API

This branch adds support for handling attachments
in the storageprovisioner API. Mostly, only volumes
are supported at the moment. These changes will
enable the storage provisioner worker to watch
attachments and attach/detach volumes/machines.

(Review request: http://reviews.vapour.ws/r/1176/)
89fd719
@jujubot
Merge pull request #1854 from juju/jes-cli 
Merge approved JES-CLI work from feature branch.



(Review request: http://reviews.vapour.ws/r/1180/)
6c3ce9e
@mjs
Merge branch 'master' into db-log
f824ad8

howbazaar added a commit that referenced this pull request Mar 17, 2015

@howbazaar
Merge pull request #1857 from mjs/db-log-merge.2 
Merge branch 'master' into db-log (again)
53a15e8

@howbazaar howbazaar merged commit 53a15e8 into juju:db-log Mar 17, 2015

@mjs mjs deleted the mjs:db-log-merge.2 branch Mar 18, 2015

@xnox
Contributor

xnox commented on service/discovery.go in 2ae93fe Mar 31, 2015

I cringe reading this =)

[ -d /run/systemd/systemd ] -> system is running PID 1 as systemd. You can check that in go code natively, without forking out to a shell.

check that system is running upstart can be done with

  • "initctl version"
  • or check that unix:abstract=/com/ubuntu/upstart socket exists
  • or as a private DBus socket call to Version api over unix:abstract=/com/ubuntu/upstart

martinpitt replied Mar 31, 2015

This is also wrong -- /sbin/init is different depending on whether you have systemd-sysv or upstart-sysv installed, so checking /proc/cmdline for /sbin/init doesn't tell you anything. Checking for /run/systemd/system is the correct, reliable, and fast thing to do here indeed.
Contributor

ericsnowcurrently replied Mar 31, 2015

@xnox As noted in my last comment, we have to be able to run this on arbitrary remote systems, both via ssh and a cloud-init script. So using a bash script was our best option. We also needed to be confident about the running init system, which is what checking PID 1 accomplishes perfectly.

@martinpitt We no longer assume /sbin/init implies upstart.

martinpitt replied Mar 31, 2015

You can run ssh remote.server test -d /run/systemd/system through ssh just fine, though.
@xnox
Contributor

xnox commented on service/service.go in 2ae93fe Mar 31, 2015

This is a security vulnerability, no? symlink race.
Contributor

ericsnowcurrently replied Mar 31, 2015

If it's a security issue then we should fix it. Could you elaborate? What would you suggest to fix it?

martinpitt replied Mar 31, 2015

It's a predictable file name in a world-writable directory. So any user can destroy any file in the system (also from other users, or from root) by e. g. ln -s /etc/shadow /tmp/discover_init_system.sh and then wait for juju to run. (google for /tmp overwrite race, there have been plenty of CVEs in the past). So if you create/call a shell script, please use mktemp.

But I'm still convinced that this is overkill. You can check what's running on a remote system with a single shell command (testing for /run/systemd/system and/or for initctl --system list).
Contributor

ericsnowcurrently replied Mar 31, 2015

Thanks for the explanation. I'll open a bug for this.
@xnox
Contributor

xnox commented on 2ae93fe Mar 31, 2015

Please drop all of this crazy shell scripts forking with simply:

if _, err := os.Stat("/run/systemd/system"); err == nil {
    // pid 1 is systemd
} else {
    // pid 1 is upstart
}
Contributor

ericsnowcurrently replied Mar 31, 2015

While that works for the local host, for interacting with remote hosts it isn't an option. We went with a bash script in that case for the confidence that it will work effectively universally. If you know of a better approach that can be used over ssh on arbitrary hosts (as well as in a cloud-init script) then please let us know.

As to the commands for deciding the init system, we went with using the executable running in PID 1 because the approach gives better information in the case that we don't recognize the init system. Furthermore, the only way to be absolutely sure about the running init system is by checking PID 1. Beyond that, we wanted to bake into juju as little knowledge of each init system as possible. The directory that the init system uses is an example of that.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment