NuGet package creation

[zoq]

mlpack is currently packaged in Fedora, RHEL, Debian, Ubuntu and MacOS (homebrew). But there is no package for windows, one approach is to provide a NuGet package, other approachs are also feasible.

The main goal is to figure out what is necessary to create the NuGet package and maybe there is an easy way to automate and integrate the process into the existing windows setup (AppVeyor). Also since mlpack depends on armadillo and there is no armadillo package in NuGet it might be necessary to provide an armadillo NuGet package first so that it can be defined as a dependency.

References:

• https://docs.microsoft.com/en-us/nuget/schema/nuspec
• https://github.com/mlpack/mlpack/blob/master/.appveyor.yml
• https://github.com/mlpack/jenkins-conf

Note 1: It's probably a good idea to provide the necessary files via the Jenkins config repo instead of the mlpack repo.

Note 2: Please only consider this issue if you are able to build mlpack on windows, also the issue might be interesting for everyone who likes to work on the "Build testing with Docker and VMs" project idea.

[sinjan25]

@zoq I would like to work on this. I have been working on policy gradients and almost done with it. But I didn't find any issue to work on which no one else is already working. So I am glad that you have opened a new issue.

[zoq]

Hello @sinjan25 feel free to take a look at the issue.

[NaxAlpha]

Hi, It is up. At the moment it is not fully integrated but manually built but I will soon integrate it in mlpack build process.
https://www.nuget.org/packages/mlpack-windows/

[zoq]

Looks good, I'm not sure I get the purpose of mlpack-windows.redist, is this package necessary?

[NaxAlpha]

Ok got it. Now it is done!

[zoq]

Nice, do you plan to document the instructions somewhere? Also, I was thinking could we use mlpack as package name instead of mlpack windows?

[NaxAlpha]

I would like to write an article or a tutorial in mlpack official website. Since I was experimenting so, I didn't want to use exact identifier because in Nuget one can't delete package or re-upload same version. In my next PR, I will integrate it in AppVeyor Build.

[zoq]

So, in your PR you are going to rename the package? Not sure if that is of interest to create an extra package for the lib and another one for the executables like we have for ubuntu.

Regarding the tutorial maybe the blog is a good place for that, we planed to use it more often, and that could be an opportunity.

[NaxAlpha]

It is possible to create package for executable using Nuget but it is not a good idea. So I would simple rename package. Also we can provide mlpack chocolaty package for executables.
Blog post looks fine to me!

[zoq]

Can you elaborate on why shipping the command line executables via NuGet is not a good idea? I haven't really thought about it, so I'm just curious to hear your opinion on that.

[NaxAlpha]

As far as, I know NuGet is more specifically for packages which are installed locally just like npm. To provide better command-line experience (in windows), it would be better to install globally.

[rcurtin]

I don't see why this makes a huge difference for executables; even if the executable is still only installed locally, that's ok, because the user can still run it---it should still be in their $PATH (or whatever the equivalent is on Windows).

[NaxAlpha]

Ok, That is also good option I guess. I will experiment with it and make it ready ASAP.

[rcurtin]

Just to clarify on the discussion we had in IRC today, I have learned several things:

• NuGet appears to be used specifically by developers who are including dependencies in their Visual Studio project, and it installs these dependencies specifically to the directory where the project is being built. So from that angle I can agree that it does not make sense to include the command-line executables in the NuGet package. (Either I remembered what NuGet was incorrectly, or I never fully knew; not sure.)

• Chocolatey appears to be a package manager for the entire system, much more like apt or dnf. It appears to be able to make a package out of a given MSI. Therefore, this appears that it will be the more preferable option for users who want to use the mlpack command-line programs.

So it seems to me that a library-only install is the right thing for NuGet (this can be configured with CMake options), and then an MSI installer can be used to create a Chocolatey package. We can also distribute the MSI installer on its own.

@zoq: any thoughts? It's possible I am still misunderstanding here; I know the Windows environment has changed significantly since the last time I used it full-time (2005).

[zoq]

Sounds good, great to have a windows user who is able to provide some valuable insight.

[Kavignon]

What is the current state to having mlpack as a NuGet package ?
More to this, when the automatic bindings to C# is over, do we still feel the need to have a NuGet package?

[zoq]

There is a NuGet package, but it's not integrated into the current build process. I see the NuGet package as a simple way to build against mlpack, it's definitely not the only solution but if someone is already familiar with NuGet, I can't see a reason for not providing a package.

[mlpack-bot]

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions! 👍

[gmanlan]

Hi, the mlpack-windows NuGet package is quite old (mlpack v2.2, 2017). Also, the package is pulling old versions of the dependencies (Boost 1.60, armadillo 7.800...). It would be nice if we can update it and in the process add the automatic scripts to build and publish from AppVeyor. The beauty about NuGet is that you can publish several versions and users can select the one they want to add into their project.

@NaxAlpha and @rcurtin are right, CLI executables should not be included in the NuGet package. The package should only contain DLLs to be used by the VS project the user is developing. You can check the OpenBLAS NuGet package for a great example of this.
Finally, NuGet packages are often the way to go with continuous integration systems building VS projects. That means, if a build system needs to pull a very heavy package each time the build runs (imagine containing all the executables), it would not be a great experience, and also it's not what the developer needs in this context.

[rcurtin]

@NaxAlpha I know it has been a long time since this issue has been discussed, but is there any chance you would be willing to look into updating the mlpack NuGet package? Like @gmanlan pointed out the one that's there is quite old. 👍

[shrit]

@rcurtin if you would like I am happy to handle this one in addition to the ensmallen package.

[rcurtin]

Sure, if you like---I guess I had forgotten about this one. 👍