Add dependency installation to ContentDB

[rubenwardy]

Automatically install dependencies for a mod when installing it.

Alternative to #8661

How it works

1. User decides to install Technic

2. A request is made to a ContentDB API to determine the dependencies.
   The API returns a list of dependencies and the packages which fulfil it.

   Example: https://content.minetest.net/api/packages/RealBadAngel/technic/dependencies/

3. Minetest works out what should be installed, based on the currently installed mods. Minetest will check in both the global mod location and the mod location of the currently selected game. It will prefer to use local content, and it will prefer to not install full games from ContentDB.

4. A simple dependency information dialog is shown, to tell the user what will be installed.

5. (optional) The user may click "Edit" to change which mods are to be installed

6. For each mod to be installed, start again from step 1.

7. Add all mods to the download manager

8. Download

Milestones

• Add download manager to queue and dispatch installs.
• Read /api/packages/<author>/<name>/dependencies/ from ContentDB
• Add simple dependency information dialog, to show what is to be installed.
• Add resolve dependency dialog, to allow the user to choose optional mods and alternatives

Mock Ups

Simple mode

Advanced mode

[AKryukov92]

I think that user should not be able to edit dependencies.

How exactly you will solve problem with following dependency constraints?

• Mod A depends on Mod B and mod C
• Mod B depends on Mod C

In case of 2 level depth of dependencies, your algorithm, informational window in step 4 will be shown multiple times. I think it will be very annoying. I think you should build list of all required dependencies recursively and only show informational window only once.

[AKryukov92]

Why do you want to implement new dependency manager from scratch if there are 2 alternatives?
I think we should learn how to use existing package manager instead of reinviting the wheel.

[rubenwardy]

We have special requirements which the alternatives don't provide. What alternatives are you specifically suggesting?

Users should be able to edit dependency installation so they can opt into installing optional dependencies or alternatives

[AKryukov92]

What requirements do you mean?

[rubenwardy]

1. Packages are installed to different places based on type
2. Packages may share the same name. Packages depend on names, not packages
3. Any solution should not require modifying mods to add meta data, as this makes it hard to add content. People other than the owner should be able to modify package metadata
4. Any solution should work on all platforms and be embedded
5. Packages can fulfill multiple names
6. Packages not installed through contentdb should still be considered (ie: minetest game)

[rubenwardy]

I think that it's a good idea to just not allow installing optional dependencies to begin with, and to just preselect the packages to install with no option to edit. This makes the initial implementation a lot easier

Added two more requirements

[AKryukov92]

Is this UI supposed to be used by advanced users or by some regular players?
I want to note that this complexity with dependencies is ok for mods, but it should be hidden from casual players who download a game, which is supposed a ready-to-play product.

Regarding the requirements you wrote.
What problems are you trying to solve by defining these requirements? I feel that you making your own dependency hell with point 2 and point 5 of your requirements.
What is "package"? So far I heard only about games and mods. Can you define term "package" more specifically?
What do you mean by "packages depend on names, not packages"?
Imagine that there are multiple different packages with name "building_blocks". One package contains clay and stone, other: stone and wood. Point 2 of your requirements allow such situation.
In addition, there is a package named "architect" which depends on "building_blocks". What exactly should be downloaded?

[rubenwardy]

Is this UI supposed to be used by advanced users or by some regular players?

All users.

I want to note that this complexity with dependencies is ok for mods, but it should be hidden from casual players who download a game, which is supposed a ready-to-play product.

Games cannot have dependencies (well, contentdb ignores them) so this form won't be shown

What problems are you trying to solve by defining these requirements? I feel that you making your own dependency hell with point 2 and point 5 of your requirements.

The problems with the existing mod system, and the fact that you can get the same dependency fulfilled in multiple ways - for example, farming is in minetest game and as farming redo

What is "package"? So far I heard only about games and mods. Can you define term "package" more specifically?

A package is installable and updatable content. Content is a game, mod, modpack, or texture pack. Content can contain content - for example, games contains mods - but packages cannot contain packages.

ContentDB deals only in packages, where it flattens the hierarchy into one thing. Ie: with a modpack, you get a list of provided modnames and a list of dependencies, but you don't know where each dependency came from - it's one entity

What do you mean by "packages depend on names, not packages"?

A mod depends on a list of modnames. A package shows this by having a list of modnames which need to be fulfilled

Imagine that there are multiple different packages with name "building_blocks". One package contains clay and stone, other: stone and wood. Point 2 of your requirements allow such situation.
In addition, there is a package named "architect" which depends on "building_blocks". What exactly should be downloaded?

This is a flaw with the current dependency system of mods depending on modnames. ContentDB attempts to get around this by strictly prohibiting mods to share the same name unless they are compatible (ie: farming redo is a fork of farming, so it allowed)

[rubenwardy]

I would like to use an existing package manager, but I haven't found any that are exactly right. Perhaps I could simplify things by requiring all packages to be in a single folder, but heh

[AKryukov92]

Is this UI supposed to be used by advanced users or by some regular players?

All users.

Why do you want to make solution for all users? One size doesn't always fits all. UI usually used to simplify things for new users. Advanced users are skillful enough to manage dependencies with file operations on the file system.

The problems with the existing mod system

Which problems, exactly?

A package is installable and updatable content. Content is a game, mod, modpack, or texture pack. Content can contain content - for example, games contains mods - but packages cannot contain packages.

Can one package depend on other package? Can package depend on mod/modpack/texture pack?
What is difference between modpack and package? Sounds like both of them are directories with mods.

ContentDB attempts to get around this by strictly prohibiting mods to share the same name unless they are compatible (ie: farming redo is a fork of farming, so it allowed)

How exactly you will check that mods with same name are compatible?
Imagine situation: Some modmaker have created mod "architect" with version 1. Some time ago he added a lot of new features to his architect mod, but version 2 is incompatible with version 1. What actions should be done to publish architect2?

[AKryukov92]

I would like to use an existing package manager, but I haven't found any that are exactly right.

You probably won't find any with requirements you previously stated. All modern dependency managers share some basic concepts like:

1. Package has unique name (multiple packages can't share same name)
2. Package has multiple versions
3. Package depend on specific versions of other packages
4. Dependencies are not optional (though I heard that npm has optional dependencies)
5. Packages are being loaded starting from roots of dependency hierarchy

I assume there is some issue with texture packs which should be loaded before other mods and other mods should not depend on texture packs to make them replaceable.