Removes the need to know OS series in many cases #6469

Closed
wants to merge 5 commits into
from

Conversation

Projects
None yet
6 participants
Contributor

kat-co commented Oct 18, 2016

This patch does a few things:

  1. Removes hidden panics from almost all the files in this commit.
  2. In many files, removes the need to know series.
  3. Simplifies Juju's understanding of paths. Many of the conditionals can be decided at compile-time.

This will allow new versions of OSs that fall outside of our release cycle (namely MacOS) to run the juju binary without panics or errors.

I HAVE NOT ADDRESSED TESTS UNTIL I GET FEEDBACK ON THIS APPROACH. TESTS WILL FAIL

state/unit.go
@@ -1780,7 +1780,7 @@ func unitMachineStorageParams(u *Unit) (*machineStorageParams, error) {
return nil, errors.Annotatef(err, "getting storage instance")
}
machineParams, err := machineStorageParamsForStorageInstance(
- u.st, chMeta, u.UnitTag(), u.Series(), allCons, storage,
+ u.st, chMeta, u.UnitTag(), allCons, storage,
@kat-co

kat-co Oct 19, 2016

Contributor

This is referring to a path for another machine; thus we need to use series/OS to determine the correct set of paths.

Owner

jameinel commented Oct 20, 2016

Just to make the comment, I think having the paths aggregated into an appropriate area is good. But because at runtime we can easily want to talk about paths on another machine (the above being one of those cases), then we need to have the groups of paths available, and be able to select which one we want.
If we then also have a helper that makes it easy to talk about the "paths-on-this-machine" that should be really good.

Although I've raised a few concerns, I think the overall approach here is good. There's lots of great macro and micro improvements.

Obviously needs test updates and TODOs handled before it can be landed.

agent/agent.go
- p.ConfDir = newPaths.ConfDir
- }
-}
+// type AgentPaths struct {
@mjs

mjs Oct 25, 2016

Contributor

I suspect all this commented out code isn't supposed to be here.

@kat-co

kat-co Oct 27, 2016

Contributor

Fixed

- DataDir() string
+ DataPath() string
+
+ StoragePath() string
@mjs

mjs Oct 25, 2016

Contributor

Doc string?

+ instancePaths.Temp,
+ instancePaths.Data,
+ instancePaths.Log,
+ instancePaths.MetricsSpool,
@mjs

mjs Oct 25, 2016

Contributor

Why not pass instancePaths (i.e. a paths.Collection) instead of passing all the paths individually?

@kat-co

kat-co Oct 27, 2016

Contributor

Because the type doesn't need all the values in the collection.

+ machineId,
+ nonce,
+ modelConfig.ImageStream(),
+ osType,
@mjs

mjs Oct 25, 2016

Contributor

osType isn't used by NewInstanceConfig so I don't think there's a reason to add this arg.

Sorting this out allows simplification in other parts of this PR.

@kat-co

kat-co Oct 27, 2016

Contributor

NewInstanceConfig now uses osType, specifically to store it so that downstream consumers of InstanceConfig can use it instead of re-deriving os.OSType. This was the goal of bringing this into InstanceConfig as it's something pertinent to starting instances.

apiserver/client/instanceconfig.go
if dataDir != "" {
- icfg.DataDir = dataDir
+ icfg.DataPath = dataDir
@mjs

mjs Oct 25, 2016

Contributor

rename dataDir to dataPath for consistency?

@kat-co

kat-co Oct 27, 2016

Contributor

Fixed

@@ -32,6 +33,8 @@ type CloudConfig interface {
// GetSeries returns the series this CloudConfig was made for.
GetSeries() string
+ DataPath() string
@mjs

mjs Oct 25, 2016

Contributor

doc string

+ tempPath string,
+ dataPath string,
+ logPath string,
+ metricsSpoolPath string,
@mjs

mjs Oct 25, 2016

Contributor

Again, suggest passing a paths.Collection for these.

cloudconfig/instancecfg/instancecfg.go
config controller.Config,
cons, modelCons constraints.Value,
+ os os.OSType,
@mjs

mjs Oct 25, 2016

Contributor

Again, given that NewInstanceConfig doesn't use os, this doesn't need it either.

cmd/juju/commands/main.go
+ // This is a non-critical error. The inability to determine
+ // the series of the machine running the Juju command does not
+ // preclude people from actually using Juju.
+ logger.Warningf("%v", errors.Annotatef(err, "cannot determine whether to warn about Juju 1.x"))
@mjs

mjs Oct 25, 2016

Contributor

I agree with your reasoning but I wonder if this should even be Debugf.

@kat-co

kat-co Oct 27, 2016

Contributor

Went ahead and made this Debugf.

@@ -78,6 +86,10 @@ func (c *agentConf) DataDir() string {
return c.dataDir
}
+func (c *agentConf) StoragePath() string {
@mjs

mjs Oct 25, 2016

Contributor

doc string

@@ -83,14 +84,17 @@ func UnregisterImageDataSourceFunc(id string) {
func ImageMetadataSources(env Environ) ([]simplestreams.DataSource, error) {
config := env.Config()
+ // TODO(katco): Pass this in (does it belong in Environ?)
@mjs

mjs Oct 25, 2016

Contributor

probably not?

environs/testing/tools.go
+ return errors.Trace(err)
+ }
+
+ toolsSeries.Add(series)
@mjs

mjs Oct 25, 2016

Contributor

Just use MustHostSeries here? (test code)

@kat-co

kat-co Oct 27, 2016

Contributor

+1

juju/paths/paths.go
-)
-
-type osVarType int
+// collection couples together all the paths Juju knows about into a
@mjs

mjs Oct 25, 2016

Contributor

Collection

juju/paths/paths.go
+ Storage: "C:/Juju/lib/juju/storage",
+ Temp: "C:/Juju/tmp",
+ }
+)
@mjs

mjs Oct 25, 2016

Contributor

A helper which takes an OS type and returns a Collection could be nice.

+
+package paths
+
+var Defaults = Nix
@mjs

mjs Oct 25, 2016

Contributor

Defaults seems dangerous to me. It could easily end up being used inappropriately.

Would you mind seeing how hard it is to do without the existence of Defaults. I'd be a lot happier if we had to think about which variant to use rather than automatically (and sometimes incorrectly) using Defaults.

@@ -184,6 +184,8 @@ type ModelArgs struct {
// MigrationMode is the initial migration mode of the model.
MigrationMode MigrationMode
+
+ StoragePath string
@mjs

mjs Oct 25, 2016

Contributor

doc string

- doc: *udoc,
+ // TODO(katco): Pass this in; unsure if I should store this in
+ // Mongo or not.
+ storagePath: paths.Defaults.Storage,
@mjs

mjs Oct 25, 2016

Contributor

This is certainly wrong as we could end up using the local machines OS to determine the storage path of a remote unit. To avoid the possibility of errors shouldn't the storage path always be calculated using the OS of the host machine?

I don't think this should be stored in mongo, nor passed in (at this level at least).

kat-co added some commits Oct 18, 2016

Removes the need to know OS series in many cases
This patch does a few things:

1. Removes hidden panics from almost all the files in this commit.
2. In many files, removes the need to know series.
3. Simplifies Juju's understanding of paths. Many of the conditionals can be decided at compile-time.

This will allow new versions of OSs that fall outside of our release cycle (namely MacOS) to run the juju binary without panics or errors.
Migrated away from global variables to instances
- I still have not touched tests.
- I admit to getting some refactoring fatigue in the middle of this.
- There is some dead code in this.
Contributor

kat-co commented Nov 16, 2016

Current status of tests: http://pastebin.ubuntu.com/23485814/

@@ -251,7 +183,7 @@ type Config interface {
// MetricsSpoolDir returns the spool directory where workloads store
@hoenirvili

hoenirvili Dec 9, 2016

Contributor

Doc needs update.

Owner

Veebers commented Jul 24, 2017

Oops, sorry for any added noise. I hadn't realised my debugging of the jenkins merge check job was actually commenting on the PRs.
Please ignore the comments added.

Owner

howbazaar commented Aug 29, 2017

With no one driving this, the massive change, and work needed, I'm closing this.

@howbazaar howbazaar closed this Aug 29, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment