Producing installation package without CI

[tpluscode]

Description

Here's the proposal I submitted on TAG Slack some time last month. It presents my idea on detaching the process of packaging Platform, Evoq or even custom installation bundles from the CI platform.

Current state

To build a package of DNN Platform it is required to build its source code and combine with artifacts produced by Persona Bar and some other repositories.

Until recently, this was done on TeamCity, which created a lock-in and its configuration promoted a pipeline which was hardly replicable on local developer machine.

Intended state

Entire build pipeline should be easily executable from the Dnn.Platform repository:

1. Building solution
2. Downloading all required packages
3. Packaging into the zip files

Getting there

Instead of relying on TeamCity or other 3rd party, which cannot be controlled directly (ie. exists only on CI infrastructure/software), I propose to employ NuGet packages.

Example pipeline

First, the module would be built and published

1. Build module dependency
2. Create zip packages as usual
   • eg. DDRMenu_09.02.01_Install.zip
3. Wrap packages in a nupkg file (new)
   • DDRMenu.9.2.1.nupkg
4. Publish the nupkg to a feed

Now that module can be consumed by Platform build

1. Build Platform solution
2. Get all required packages from the feed
3. Copy the zip installers to the required location
4. Run packaging script

Benefits

This has a number of positive effects on what is possible:

• Repository will become self-contained
• Packaging will be repeatable
• Currently there is no way guarantee that packaging same revision produces identical packages!
• Each module will be allowed to be published and versioned independently
• Module can have dependencies which will actually be resolved
  • Albeit, only at build-time but it's a start
• One would clone Dnn.Platform, add their own modules and package a custom installation package
  • Could use a private feed or use open feed of 3rd party components

Tooling proposal

I mention NuGet, but not the official client. Instead, there is wonderful open source replacement called Paket. It has a number of advantages:

• Can (easily) work outside of a solution/csproj, just downloading the necessary packages
• Comes with a simple text format, both for defining dependencies as well as creating nupkgs
• Guarantees stable folder structure, which is important for easy scripting
• Generates a lock file, like yarn, which guarantees repeatable builds

[tpluscode]

I will try creating a PoC set of cloned repositories to validate this idea and my proposed solution.

[ohine]

I would personally like to see an xml or json manifest of all external components. This file should contain the name of the extension, the version, the url to the install zip, the url to the symbols, and the url to the source zip.

I want to get away from always recompiling every extension regardless of any changes for each release.

Then it would be pretty simple to download those packages and add them into the final platform install/upgrade/source zips.

@dnnsoftware/tag

[bdukes]

@tpluscode's proposal would include a package.json or paket.dependencies file, which would list each component and the means of downloading it (either from NuGet or HTTP, probably). Is that what you're looking for, or is there a reason to do something different?

[ohine]

If it's just consuming an already published package, that's perfectly ok. It sounded like we'd be downloading the other components and compiling / publishing a new package to then bundle into the final installation zip.

[ohine]

Related Issues #2077 #2080 #2064

[valadas]

We just need to update documentation and then this is ready to go.

[valadas]

I have assigned this to 9.4.0 milestone, I would really like to get the process of building locally documented, ping me up if any help is needed to do this.

[valadas]

@ohine any news on the case sensitivity issues, ping me up if I can help anyway regarding this...

[valadas]

Closing since merged