Make a PreRelease Nuget package version available

[theluk]

1. edit file "test.txt"
2. make a commit (1st commit)
3. edit file "test.txt" again
   3 make a commit (1st commit)

if you now use "git checkout 1stcommit" you will be in detached head mode but no files are dirty(if all were previously commited)

in libgit2sharp if you do checkout(1stcommit) (sharef) you will be also in detached head mode BUT the files did not change and there are now dirty / did not change to the previous commit

[theluk]

oh I see, the latest public version even doesnt make any changes to the working dir...

Couldnt you make a Nuget package which can be used through the -IncludePrereleases ???

I dont want to download the whole project just to have the newest version (even if its buggy...)

[nulltoken]

Couldnt you make a Nuget package which can be used through the -IncludePrereleases ???

At this very moment, no. Once libgit2 is released in version 1.0.0, things should be a bit more stable from the LibGit2Sharp standpoint, which should allow to go back of a much more frequent release schedule of NuGet packages.

Note: I've updated the title to reflect the latest intent of this issue

[nulltoken]

I dont want to download the whole project just to have the newest version (even if its buggy...)

@theluk v0.10.0 has been recently released.

From now on, we're going to (try and) release a new NuGet package every month. This should fulfill your request without having to issue a PreRelease version while we're still in version 0.x.x.

[grenade]

I would like to help with producing a regular NuGet pre-release if there is appetite for doing so.

I think it is just a case of creating a copy of the existing CI build at CodeBetter and adding a step to append an -alpha suffix to the AssemblyInformationalVersion before compile, then adding a NuGet Pack step to the build configuration to produce the NuGet pre-release artifact at each successful build (pre-release versioning: http://docs.nuget.org/docs/reference/versioning#Creating_Prerelease_Packages, TC NuGet Pack: http://confluence.jetbrains.com/display/TCD7/NuGet+Pack).

You could then either allow early adopters to grab the pre-release package directly from the codebetter NuGet repo (http://teamcity.codebetter.com/guestAuth/app/nuget/v1/FeedService.svc/) or a NuGet Publish step could be added to push the .nupkg to nuget.org

I'm happy to do the work/implementation if you want to give me access to your codebetter build configuration or I'm happy to just assist or make tea for whoever normally handles your CI stuff.

Apologies if this came across as showing granny how to suck eggs. Presumably you guys already do this for the regular release.

[nulltoken]

I would like to help with producing a regular NuGet pre-release if there is appetite for doing so.

@grenade Thanks a lot for this proposal!

I really like the idea to automatically publish a Prerelease package on the NuGet gallery for each successful build.

Currently, the TC server is set to monitor the vNext branch of my personal nulltoken/libgit2sharp repo. This is more or less an integration branch. My current process is to first push to this branch, ensure that the amd64, x86 and Mono build pass before pushing to libgit2/libgit2sharp vNext. This guarantees that everything in the libgit2/libgit2sharp vNext branch has successfully passed on all the supported platforms.

Maybe should we create a new build configuration targeting the libgit2/libgit2sharp vNext branch? Once this branch is updated, it should be safe to publish something. If that feel a bit clumsy to you, please chime in. I'm very open to a more streamlined process 😉

Below some additional questions:

• How should we deal with the versioning scheme? (I kind of like what is being done for RavenDB where the patch number represents the build number)
• How to you foresee the mechanism to publish a stable NuGet package?
• Should we create a dedicated NuGet API key? Where should it be securely stored?

I've granted you with some rights that should allow you to peek (and poke 😉) at the TeamCity project settings. I trust you'll be cautious before applying any change as this CI server is one of the most important part of this project.

[nulltoken]

@grenade Another point that might also be taken into account.

With each nuget release, we also push the symbols toward symbolsource.org. This process is described in an excellent post by @davidebbo.

However, since v0.14.1, this process requires now some manual tweaking.

Indeed, #518 changed the location of the native binaries which are now located under the lib NuGet tree hierarchy. @TripleEmcoder explained me that the Symbolsource endpoint tries to parse them as managed assemblies. This results in errors as the following one:

D:\LibGit2Sharp\LibGit2Sharp\nuget.package>..\Lib\NuGet\NuGet.exe push LibGit2Sharp.0.14.1.0.nupkg
Pushing LibGit2Sharp 0.14.1.0 to the NuGet gallery (https://www.nuget.org)...
Your package was pushed.
Pushing LibGit2Sharp 0.14.1.0 to the symbol server (http://nuget.gw.symbolsource.org/Public/NuGet)...
Failed to process request. 'Package submission failed: Format of the executable (.exe) or library (.dll) is invalid. Exception: System.Exception: Error while processing project: 'LibGit2Sharp'. ---> System.Exception: Error while processing version: '0.14.1.0'. ---> System.Exception: Error while processing compilation: 'Release' '.NETFramework,Version=v3.5'. ---> Sys (...). See http://www.symbolsource.org/Public/Home/Help for possible reasons. Fiddler may help diagnosing this error if your client discards attached detailed information.'.
The remote server returned an error: (506) Package submission failed: Format of the executable (.exe) or library (.dll) is invalid. Exception: System.Exception:
 Error while processing project: 'LibGit2Sharp'. ---> System.Exception: Error while processing version: '0.14.1.0'. ---> System.Exception: Error while processing compilation: 'Release' '.NETFramework,Version=v3.5'. ---> Sys (...). See http://www.symbolsource.org/Public/Home/Help for possible reasons. Fiddler may help diagnosing this error if your client discards attached detailed information...

The workaround is to edit the symbol package and remove the native libs from it. I haven't investigated if there's a way to automate this as well.

[grenade]

Maybe should we create a new build configuration targeting the libgit2/libgit2sharp vNext branch?

In light of your current checks and balances, that makes good sense.

How should we deal with the versioning scheme?

Since there is already implicit convention in the current scheme (assembly version: 0.x.0.0, product version: 0.x.0), there are important constraints so as not to confuse users or other automation systems. My first inclination is to use build chaining (snapshot/artifact dependencies) to:

• create an automatically triggered (every change to libgit2/libgit2sharp#vNext) build which produces assembly version: 0.x.{patch-number}.{teamcity-build-number} and product version: 0.x.{patch-number}-alpha. This will likely produce more churn than is suitable for pushing to nuget.org so I propose only hosting these on codebetter nuget. These will be purged on the regular codebetter cleanup runs so should be treated as disposable, but very useful to anyone who wants to code against bleeding edge LibGit2Sharp.
• create a manually (click run) or schedule (daily/weekly - you guys understand the frequency better than me) triggered job that produces assembly version: 0.x.{patch-number}.0 and product version: 0.x.{patch-number}-alpha which it also pushes to nuget.org as pre-release.

How do you foresee the mechanism to publish a stable NuGet package?

A manually triggered (click run) job that produces assembly version: 0.x.{patch-number}.{teamcity-build-number} and product version: 0.x.{patch-number} and pushes to nuget.org.

Should we create a dedicated NuGet API key? Where should it be securely stored?

Probably not. You want to treat the key as disposable in case it is compromised. If you want schedule or VCS triggered pushes to NuGet, you have to store the key as a configuration parameter in your build configuration. If you set it to password/hidden, it is encrypted and stored in the filesystem on the codebetter server. If you go for manually triggered pushes, TeamCity can prompt for the key at the start of the build. This offers the best security since it isn't stored on the teamcity server (it's obfuscated in the logs). We should probably start out in this way and make a call later about whether or not you want to risk storing the key on teamcity. The nupkg files from the -alpha build will still be automatic and always available from codebetter. No key required there.

I've never worked with SymbolSource so will have to read up on that. It would be trivial to produce a SymbolSource specific artifact at the end of the build that doesn't contain the bits SymbolSource doesn't like.

[nulltoken]

Since there is already implicit convention in the current scheme (assembly version: 0.x.0.0, product version: 0.x.0), there are important constraints so as not to confuse users or other automation systems.

@grenade Sadly I'm not a build wizard, so you'll have to bear with me if I'm a little slow here. We're following the semver principles (MAJOR.MINOR.PATCH). How bad would it be if the PATCH was the build number? What kind of impacts/risks would we be facing?

This will likely produce more churn than is suitable for pushing to nuget.org so I propose only hosting these on codebetter nuget.

"more churn than is suitable" -> What would be the downside of publishing a new prerelease build for each push to libgit2 vNext?

[grenade]

How bad would it be if the PATCH was the build number? What kind of impacts/risks would we be facing?

This could work with your main releases (eg: not pre-releases) setup if you only push to libgit2/libgit2sharp#master when you have everything you want in the patch.

For libgit2/libgit2sharp#vNext it's kind of problematic, if you want to release sequential patches, because vNext builds by design are more frequent than master. If you don't care about releasing sequential patches (eg: 0.15.23 following 0.15.1), then its no big deal and you can have your PATCH as the build number. In the actual implementation (because of the branches), the build numbers are going to have to be references to snapshot dependency build numbers rather than the actual sequentially incremented build number of the specific build.

What would be the downside of publishing a new prerelease build for each push to libgit2 vNext?

Just the volume. Sometimes you're simply fixing a broken build, so you change something in configuration and you push to vNext which triggers the rebuild and release. Then you have an actual (pre)release to nuget.org that contains no tangible benefit to your end users. There's nothing wrong with that. It's just churn. My preference is to shield users from that, but considering that consumers of pre-release packages accept a certain level of inconvenience in exchange for bleeding edge functionality, its probably acceptable. Your package, your call.

[nulltoken]

Thanks a lot for this very detailed feedback!

Regarding the potentially huge amount of pre-release packages, I (currently) don't see this as a blocking point. Even if vNext is the development branch, we tend to avoid pushing 'Oops' commits to it (they're usually squashed/rewritten in the pull request they come from). Build issues would be spotted and fixed as part of the pushes to nulltoken/vNext.

One thing I'm unsure about is how should we deal with releasing a new version to master.

Considering the following:

• vNext - commit a - Add feature -> publish a prerelease nuget package v0.15.125-Unstable
• vNext - commit b - Publish LibGit2Sharp v0.16 (updates the release notes and the assembly version properties) -> publish a prerelease nuget package v0.16.126-Unstable
• vNext - commit c - Remove obsolete members -> publish a prerelease nuget package v0.16.127-Unstable

Later, when pushing commit b to master, we should publish a stable nuget package. Is there a way we could version it as v0.16.126 as well?

[grenade]

Is there a way we could version it as v0.16.126 as well?

Yes. You can set the build number to whatever you want in the first build step by printing to stdout. Eg with a command line runner: echo "##teamcity[buildNumber '<new build number>']" Subsequent build steps can access the build number with: %build.number%.

So using any scripting language, you could parse an Assembly Info file to get the build number.

http://confluence.jetbrains.com/display/TCD7/Build+Script+Interaction+with+TeamCity#BuildScriptInteractionwithTeamCity-ReportingBuildNumber

[nulltoken]

So using any scripting language, you could parse an Assembly Info file to get the build number.

Bam! I'm lost. I told you I wasn't a build wizard 😉

From which Assembly Info file should the build number be extracted? I understand that prior to publishing any nuget package, we would update the Assembly Info file to append the build number (and maybe the "Unstable"/"stable" label?) , but I'm not sure to follow how a different build would access this (temporary?) file later.

[grenade]

I probably misunderstood you the first time. I made assumptions about an assembly info file being committed to vNext and merged to master later. If that's not the case, then perhaps a snapshot or artifact dependency on the vNext build could be made to work. I've never done this across branches before though so i'll have to try it on a local test and see what happens. If that doesn't work, it may make sense to have the vNext build actually commit either an updated assembly info file(s) or some new text file (a changelog?) that contains the version number.