Dependency processing

[pjf]

We need to be able to do dependency processing. Right now, we just check if your dependencies are installed, and fail if they're not. That's not ideal.

Real Solar System is a good example of something with "hard" dependencies here. It depends upon its texture files, but there are multiple texture files available, and they in turn depend upon having the RSS base installed.

I think the following algorithm should work:

• Compose a list of everything we already have installed.
• Add to that list everything that we're installing.
• Walk through what we're installing, and see if their dependencies are satisfied by the (installed+installing) list we've created.
• Anything which isn't satisfied gets added to the list of things to install, and we repeat.
• Make sure all of the above uses both the module names and the provides field, as mentioned in the spec.

This means installing RSS with a texture pack works fine. Installing just an RSS texture pack will pull in the dependency on RSS, which should work fine.

It's important that we do allow circular dependencies, because they already exist (X + Y must be installed together type things).

In order to add dependencies, we need features to allow the client to be aware of all mods it has available.

[pjf]

As @NathanKell pointed out on IRC, this isn't safe if one mod needs to overwrite part of another mod (which also relates to #15 ). We may need some sort of signal to indicate that dependencies need to be installed first. If we do go down that route, using it should be discouraged, as well-packaged mods should be install-order independent.

[Wizarth]

Discouraged but still present - for example, mods for mods (how meta) will definitely need this.

[pjf]

@Wizarth : Most mods of mods use ModuleManager to do their changes. This means they don't need to overwrite any files, and install order is truly unimportant. So while RealismOverhaul must be installed after TACLS, it can be installed before before or after any of the other mods it depends upon, just as long as they're all present before one starts the actual game.

So while dependencies themselves are super common (and cyclical dependencies are fine), dependency ordering should be quite rare (and cyclical dependencies are disallowed).

[pjf]

Stealing from the debian policy manual, I'm going to suggest we have depends and pre-depends.

I'm also rather partial to their entire relationship model, which includes:

• recommends - A strong, but not absolute dependency.
• suggests - A suggestion another mod be installed.
• enhances - An indication this mod improves the functionality of another mod.

In #11 we also suggest having a supports field, which is a clear indication of compatibility.

@NathanKell, in the example we had of RealismOverhaul, we might say that it depends on ModuleManager, recommends EngineIgnitor, and suggests TextureReplacer.

A typical install (ckan install RealismOverhaul) would install both dependencies and recommends. However a user may do a ckan install --no-recommends RealismOverhaul to get just the bare minimum, or a ckan install --all-suggested RealismOverhaul to get everything it suggests.

(Actual switch options are indicative only, and may change.)

[NathanKell]

Sounds good to me.

[pjf]

Okay, this is my work unit for the day, and also quite possibly the biggest remaining bit of needed functionality. We'll be abstracting a bunch of things into a generalised relationship class. We'll generate a list of install rounds; nominally just one, but if something has a pre-depends that will create an install round before it (and if there's a pre-depends in that, then another install round before that and so on).

To start with, I'll just be doing dependency resolution; anything with a 'pre-depends' will just throw an exception for now.

[pjf]

I'm following dependency chains, but they're raising a few issues.

When we install a module, we must install its dependencies, and we may install its recommends list (unless specifically asked not to), and its suggests list (if explicitly requested).

But what do we do in the case of chained recommendations? If A depends on B, and B recommends C, then do we install C when asked to install A?

I'm inclined to go with the following rules:

• If we're running in --no-recommends mode, we only install dependencies, no matter what.
• If we're running in --with-recommends mode (the default), we install all recommended modules, all the way down the chain.
• If we're running --with-suggests we only install the suggested modules for the top-level module, everything else gets the default treatment.
• If we're running --with-all-suggests, then we install all suggestions, all the way down.

This meas that the default experience (with recommends) gives both user and module authors the configuration they'd expect. Recommends is a strong relationship, so those things should be installed.

Users running --with-suggests get the suggestions for the module they asked for, but not for something six layers deep that was required, but which the user was unaware of. This means we don't surprise the user with unexpected cruft.

Users running --with-all-suggests (suggested alias: --with-kitchen-sink) get what they asked for. ;)

Thoughts and feedback welcome.

[Wizarth]

Do we know what apt-get does in this circumstance, for reference?

On Sun, Oct 12, 2014 at 6:33 PM, Paul Fenwick notifications@github.com
wrote:

I'm following dependency chains, but they're raising a few issues.

When we install a module, we must install its dependencies, and we may
install its recommends list (unless specifically asked not to), and its
suggests list (if explicitly requested).

But what do we do in the case of chained recommendations? If A depends
on B, and B recommends C, then do we install C when asked to install A?

I'm inclined to go with the following rules:

• If we're running in --no-recommends mode, we only install
  dependencies, no matter what.
• If we're running in --with-recommends mode (the default), we install
  all recommended modules, all the way down the chain.
• If we're running --with-suggests we only install the suggested
  modules for the top-level module, everything else gets the default
  treatment.
• If we're running --with-all-suggests, then we install all
  suggestions, all the way down.

This meas that the default experience (with recommends) gives both user
and module authors the configuration they'd expect. Recommends is a
strong relationship, so those things should be installed.

Users running --with-suggests get the suggestions for the module they
asked for, but not for something six layers deep that was required, but
which the user was unaware of. This means we don't surprise the user with
unexpected cruft.

Users running --with-all-suggests (suggested alias: --with-kitchen-sink)
get what they asked for. ;)

Thoughts and feedback welcome.

—
Reply to this email directly or view it on GitHub
#32 (comment).

[pjf]

Aaah, WWAGD? I don't know, but that's a good thing to check. :)

[pjf]

It looks like apt-get switches things on and off the whole way down. So by default, you get all the recommended packages for everything. With suggests turned on, you get a kitchen sink.

So I'm essentially doing the same there, but tweaking --with-suggests to not include all the suggestions all the way down. Given how many times I've seen the question asked about how one installs the suggests for the requested module but not everything else¹, I think my deviation from the apt-get way of doing things has merit.

But thanks for mentioning that, WWAGD was indeed the obvious way to see if my plan had potential for sanity. :)

¹ Debian lore has it that installing gparted with suggestions will also give you apache, and a full dev environment for three different languages.

[pjf]

More progress on this in 306cc03 with FindReverseDependencies() by @AlexanderDzhoganov .

[pjf]

It sure looks like we're doing the core of this now, so closing.