Support running docfx on .NET Core

[danroth27]

Today docfx runs on the full .NET Framework (and Mono). We should update docfx to run on .NET Core, as this will be the support xplat story for .NET console applications. Obviously this means working with the authors of the many third party dependencies that docfx has to get them onto .NET Core as well.

@blackdwarf

[agjohnson]

What is the current status for runtime support of docfx? Any ETA on coreclr support? I'm trying to run locally on OSX, a 1.0.0-rc1-update2 on a mono runtime works for the most part, though I hit exceptions with pieces that look to be Mono related.

[danroth27]

It's going to take a while to get docfx off of Mono and on to .NET Core. The problem is that docfx has a bunch of 3rd party dependencies that haven't been updated yet to work on .NET Core.

However, docfx did recently release a preview that can analyze .NET Core based projects: https://github.com/dotnet/docfx/releases/tag/1.9.0-alpha-0091-g1c1b291. I haven't had a chance to test it out yet though. Note that we are moving everything (including docfx) off of DNX.

[agjohnson]

Oh interesting, I didn't realize about the move from dnx -> dotnet cli tool. I should give a try with the latest mono and latest supporting toolchain. I don't know if that will clear up any of the bugs that I'm attributing to mono, but worth a shot.

[danroth27]

Yeah, moving off of DNX is a pretty big change for the product. The good news is that you no longer need to install DNX - docfx is now just a native executable.

[agc93]

Is there any update on this? DocFx seems to have almost ditched the xplat story:

• Commit 562f74b and ef0632d progressively removed most of the documentation references to the xplat version of DocFX
• This issue hasn't had an update in over a year
• Shipping as a Chocolatey package only works for Windows users
• The zip package bundled with each GitHub release includes only the exe
• .NET Core support has been RTM'ed for some time and 2.0 is just around the corner
• Many (looks like not all) of the dependencies are starting to support .NET Standard

Is there still plans to make this tool cross-platform? It was a big reason I chose it and at this point I'm looking at alternatives as support for .NET Core/xplat seems to still be a long way off.

[vicancy]

Hi @agc93 Migrating to net core is in our backlog however not yet with the highest priority. Currently mono docfx is working as an x-platform solution. Could you share your concerns about this solution? We can re-prioritize the issue if it is a common requirement from community.

[agc93]

@vicancy I'm currently building a .NET Core app on Linux and DocFX is the only thing that needs Mono: the app, build scripts, dependencies and server all run just fine on .NET Core natively, no Mono needed. Additionally, I can't CI build most of my projects as my CI environment doesn't include Mono (again .NET Core projects don't need it).

While I think using mono is an acceptable workaround I think .NET Core support still seems like a pretty important step given it's now been stable for some time and 2.0 is around the corner.

Realistically, I was using docfx to avoid extra dependencies, not add them

[vicancy]

@agc93 thanks for your feedback. Yes, I totally agree with you that .NET Core is a meaningful approach. We will investigate the effort migrating to .NET core. We have some 3rd party dependencies not supporting that, e.g. Microsoft.Owin, so it may take some effort to get rid of those dependencies.

[fritz-build]

@vicancy looks like owin is only used for docfx serve -- could just as easily use asp.net core with the static files middleware -- are there any other dependencies off the top of your head that need to be migrated? I'd be happy to contribute where needed -- this would make xplat much, much easier.

[vicancy]

@fritz-build One big change is our plugin system which uses another AppDomain. https://github.com/dotnet/docfx/blob/dev/src/docfx/SubCommands/BuildCommand.cs#L517

Honestly speaking, our team does not have enough resource currently to look into xplatform issues, It would be fantastic if you could contribute. We are eager for contributions and just let me know if I can do anything to help. From owin to asp.net core can be the first step out moving to netcore.

[dmccaffery]

Hey @vicancy i was logged in as @fritz-build by accident when I posted last. I’ll create an issue to convert to asp.net core and create an associated PR sometime this week. I appreciate that resources are limited; just thought that baby steps could go a long way.

[tugberkugurlu]

I imagine that effort to move this to .NET Core 2.0 will require less work considering that you can use most of the full .NET Framework dependencies and API surface are of .NET Standard is much larger.

[tugberkugurlu]

One big change is our plugin system which uses another AppDomain. https://github.com/dotnet/docfx/blob/dev/src/docfx/SubCommands/BuildCommand.cs#L517

@vicancy for example, AppDomain is part of .NET Standard 2.0 now: https://docs.microsoft.com/en-gb/dotnet/api/?view=netstandard-2.0&term=appdomain

[davidfowl]

You can't create app domains though.

[tugberkugurlu]

@davidfowl I am confused 😕 It says you can here: https://docs.microsoft.com/en-gb/dotnet/api/system.appdomain?view=netstandard-2.0 Am I looking at the wrong place?

[tugberkugurlu]

I am confused 😕 It says you can here: https://docs.microsoft.com/en-gb/dotnet/api/system.appdomain?view=netstandard-2.0 Am I looking at the wrong place?

meh, https://github.com/dotnet/corefx/blob/7a673547563dc92f1f4cad802d7b57483cdf7500/src/System.Runtime.Extensions/src/System/AppDomain.cs#L94

It would be off topic to ask why that API is on .NET Standard at the first place. So, I will find a better place to do it.

[davidfowl]

https://github.com/dotnet/standard/

[tugberkugurlu]

@davidfowl thx! Created dotnet/standard#456

[tintoy]

I know I'm a little late to this party, but I'd like to add my vote for moving to .NET Core (or at least .NET Standard) at some point. I've been looking into redoing my DocFX extension to use an LSP language service for DocFX written in managed code (so I can reuse the DocFX engine libraries), but have been kinda blocked for a while now by the fact that some of those libraries have dependencies that aren't fully compatible with .NET Standard (not even 2.0). I could make a Windows-only language service but that sounds like no fun at all :-)

As for AppDomain, I've found there are plenty of more-cross-platform-friendly workarounds available (e.g. launching a separate process and communicating via NamedPipeClientStream / AnonymousPipeClientStream / STDIN+STDOUT).

I'm willing to put my hand up to spend some time / effort on helping migrate DocFX to .NET Standard / .NET Core (or work with anyone else if there are other volunteers).

[vicancy]

#1963 to track the work items

[vicancy]

Thank you @tintoy It would be great if you can give a hand on this issue.

[dmccaffery]

Im 100% down. Let me know how I can help. Any guidance on how I can do so would be greatly appreciated.

[tintoy]

Wow, OK, yak-shaving time. There is no Mustache implementation currently compatible with .NET Standard (any version!). Looks like I'll have to create a .NET Standard port of Nustache (or similar).

Anyone who'd like to help - now's the time to pitch in :)

[tintoy]

Actually, let me see if it'll still run on .NET Standard 2.0 first (I get the usual NU1701 but it may still work correctly).

[agc93]

@tintoy Not strictly Mustache, but Handlebars.Net supports .NET Standard as a starting point

[jskeet]

Any updates on v3? It makes me sad that there's still no docfx release running on .NET Core :(

[superyyrrzz]

@jskeet On the way but not ready for public use now... See https://github.com/dotnet/docfx/blob/v3/docs/roadmap.md

[AndyPook]

@superyyrrzz wrt dotnet Core
I can't see any reference to dotnet Core in the roadmap.md
Is there an issue tracking this?

[superyyrrzz]

@AndyPook It is not mentioned in the roadmap, but v3 starts from .NET Core, and will always support it.

[AndyPook]

@superyyrrzz ok thanks for that. Is there a expected timeframe? I get limited resources etc 😄 not looking for a date, just a guide, weeks, months ?

My current solution is that I've created a docker base image with the mono etc deps installed and a separate CI step with a volume mount ...
https://gist.github.com/AndyPook/a36cfd8708800a4d3fdb06a1eb31eb83

[superyyrrzz]

@AndyPook Not within 2019. We do not determine the exact date, but possible in 2020.

I think there is some existing docfx image on docker hub that can make your work easier.

[roji]

Any timeline update on this? docfx is an important part of the .NET ecosystem and is used by many OSS projects out there - keeping this a modern, functional building block is really important.

[yufeih]

@roji There are two key pieces missing for the initial v3 community release: a community template and exposing the .NET API reference build. We've just started prototyping the community template here and will likely look at .NET API reference after that. Hopefully some early preview releases will come this calendar year.

[KalleOlaviNiemitalo]

I tried DocFX 3.0.0-beta1.246+8b9773b7f0 and https://github.com/docascode/default-template 1e3e3a9b33f2a1dfd5653efd4dac85b7e8125439 on a document tree that contains only Markdown documents, no .NET API. Installed as a .NET local tool on Windows. Found problems:

• dotnet docfx build --template docfx/default-template tries to open docfx\default-template\ContentTemplate\docfx\default-template\ContentTemplate\Conceptual.mta.json.js which does not exist. This path doubled the docfx\default-template\ContentTemplate part. No problem if I specify an absolute path.
• DocFX did not automatically copy the fonts, styles, and svg directories from the template to _site.
• Each Markdown file now translates to a directory with index.html in it, but docfx serve in DocFX 2.45.1 does not find that index.html if the directory name contains spaces or non-ASCII characters. (DocFX v3 doesn't seem to have docfx serve at all.) Filed DefaultFilesMiddleware does not find default file if path contains percent-escaped characters aspnet/AspNetKatana#371 and docfx serve does not find index.html in percent-encoded directory #7369.
• DocFX displayed "Copyright © Microsoft." in the page footer even though I had set build.globalMetadata._appFooter in docfx.json. It seems _includes/footer.html.liquid does not receive the value of _appFooter. I also tried setting _appFooter in a YAML header in a Markdown file, and that had no effect either.

[KalleOlaviNiemitalo]

From #8279 (comment):

NOTE: .NET Framework is still needed to build from cs files or DLLs

Is that only for CompilationUtility.GetNetFrameworkMetadataReferences, which tries to locate mscorlib.dll and System.dll? Perhaps the Microsoft.NETFramework.ReferenceAssemblies NuGet package could be used instead.

[yufeih]

From #8279 (comment):

NOTE: .NET Framework is still needed to build from cs files or DLLs

Is that only for CompilationUtility.GetNetFrameworkMetadataReferences, which tries to locate mscorlib.dll and System.dll? Perhaps the Microsoft.NETFramework.ReferenceAssemblies NuGet package could be used instead.

Right, that is one reason. The other reason is the referenced assembly list to build DLLs is incomplete (only mscorlib and System.dll), we might want to pass all referenced assemblies stated in the DLL to roslyn, which libraries like Cecil or ILSpy may be able to help.

Microsoft.NETFramework.ReferenceAssemblies seems to be a build-time library, we probably don't want to ship it as part of docfx.

[KalleOlaviNiemitalo]

For generating docs from DLL files, I think the caller of docfx metadata should provide the paths of reference assemblies. Preferably in such a way that docfx.json does not have to be modified when (transitive) dependencies change. The user's doc build system could write the paths of reference assemblies (including any that came from NuGet) to a temp file and pass the path of that file to the docfx metadata command line.

Is there a scenario in which docfx metadata should use one set of reference assemblies for one DLL but a different set of reference assemblies for another DLL? Perhaps it can instead be handled by running docfx metadata twice.

[yufeih]

v2.60.0-preview.1 has been released to NuGet.org with the first wave of changes running on .NET Core. See #8305 for details.

[yufeih]

The v2.60.0 release now runs on .NET Core.

[roji]

A huge thanks @yufeih - I've just switched to running the tool with .NET Core, and everything seems to work just fine! Really nice to get rid of .NET Framework!