Generating API documentation for a library that uses 3rd-party dependencies fails

[teo-tsirpanis]

Consider the attached example library. It defines a library with a class with a property that returns a Newtonsoft.Linq.JToken.

dotnet fsdocs building the project fails with

Could not read comments from entity 'FsDocsBug.Inner.MyType': System.InvalidOperationException: The entity or value 'Newtonsoft.Json.Linq.JToken' does not exist or is in an unresolved assembly. You may need to add a reference to ass
embly 'Newtonsoft.Json'
   at FSharp.Compiler.SourceCodeServices.Impl.dflt@113[a](String tupledArg0, String tupledArg1) in /Users/chethusk/oss/FCS/src/fsharp/symbols/Symbols.fs:line 75
   at FSharp.Compiler.SourceCodeServices.Impl.protect[a](FSharpFunc`2 f)
   at FSharp.Compiler.SourceCodeServices.FSharpEntity.get_MembersFunctionsAndValues() in /Users/chethusk/oss/FCS/src/fsharp/symbols/Symbols.fs:line 597
   at FSharp.Formatting.ApiDocs.SymbolReader.getMembers@1886-1.GenerateNext(IEnumerable`1& next)
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl() in E:\A\_work\130\s\src\fsharp\FSharp.Core\seqcore.fs:line 371
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)
   at FSharp.Formatting.ApiDocs.SymbolReader.readType@1882.Invoke(String cat, Int32 catidx, Boolean exclude, IDictionary`2 _cmds, ApiDocComment comment) in C:\GitHub\dsyme\FSharp.Formatting\src\FSharp.Formatting.ApiDocs\GenerateMode
l.fs:line 1899
   at FSharp.Formatting.ApiDocs.SymbolReader.readCommentsInto$cont@1743[c](FSharpSymbol sym, FSharpFunc`2 f, FSharpOption`1 nsdocs, ApiDocComment comment, IDictionary`2 cmds, String cat, Unit unitVar) in C:\GitHub\dsyme\FSharp.Forma
tting\src\FSharp.Formatting.ApiDocs\GenerateModel.fs:line 1753

How to generate documentation for a library that uses 3rd-party types in its API?

It looks like that the command tool has trouble finding the references for the assemblies it processes. One solution would have been through the AssemblyDependencyResolver class.

[dsyme]

Hmm interesting, yes. I guess ideally we'd point everything at a tests folder (i.e. an application) that has all dependencies bundled.

[TheAngryByrd]

I did this recently in MiniScaffold where we do a dotnet publish on the projects and use the publish output directory for the Content docs and the references docs.

---

Aside: I didnt realize how far along you've gotten with fsdocs. I think it's time to for me to consider retiring my docsTool stuff and contribute to this repo.

[teo-tsirpanis]

@TheAngryByrd we can't do that from the command tool though. 😢

[TheAngryByrd]

@TheAngryByrd we can't do that from the command tool though. 😢

Yeah I was more pointing to how this I solved the problem and how this repo could fix it.

[kMutagene]

Same situation, different stack trace (from Dash.NET):

API docs:
  generating model for 1 assemblies in API docs...
  loading 1 assemblies...
  registering entities for assembly Dash.NET...
Error :
FSharp.Compiler.ErrorLogger+UnresolvedPathReferenceNoRange: Assembly: Giraffe, full path: Giraffe.GiraffeViewEngine.XmlNode
   at FSharp.Compiler.TypedTree.CcuThunk.EnsureDerefable(String[] requiringPath) in F:\workspace\_work\1\s\src\fsharp\TypedTree.fs:line 5103
   at FSharp.Compiler.TypedTree.NonLocalEntityRef.TryDeref(Boolean canError) in F:\workspace\_work\1\s\src\fsharp\TypedTree.fs:line 3157
   at FSharp.Compiler.TypedTree.EntityRef.get_Deref() in F:\workspace\_work\1\s\src\fsharp\TypedTree.fs:line 3254
   at FSharp.Compiler.TypedTreeOps.stripTyEqnsA(TcGlobals g, Boolean canShortcut, TType ty) in F:\workspace\_work\1\s\src\fsharp\TypedTreeOps.fs:line 739
   at FSharp.Compiler.TypedTreeOps.isUnitTy(TcGlobals g, TType ty) in F:\workspace\_work\1\s\src\fsharp\TypedTreeOps.fs:line 1666
   at FSharp.Compiler.TypedTreeOps.GetTopValTypeInCompiledForm(TcGlobals g, ValReprInfo topValInfo, Int32 numEnclosingTypars, TType ty, range m) in F:\workspace\_work\1\s\src\fsharp\TypedTreeOps.fs:line 2324
   at FSharp.Compiler.TypedTreeOps.XmlDocSigOfVal(TcGlobals g, Boolean full, String path, Val v) in F:\workspace\_work\1\s\src\fsharp\TypedTreeOps.fs:line 8080
   at FSharp.Compiler.SourceCodeServices.SymbolHelpers.GetXmlDocSigOfScopedValRef(TcGlobals g, EntityRef tcref, ValRef vref) in F:\workspace\_work\1\s\src\fsharp\symbols\SymbolHelpers.fs:line 541
   at FSharp.Compiler.SourceCodeServices.FSharpMemberOrFunctionOrValue.get_XmlDocSig() in F:\workspace\_work\1\s\src\fsharp\symbols\Symbols.fs:line 1845
   at FSharp.Formatting.ApiDocs.CrossReferences.getXmlDocSigForMember(FSharpMemberOrFunctionOrValue memb) in C:\Users\Kevin\source\repos\fsprojects\FSharp.Formatting\src\FSharp.Formatting.ApiDocs\GenerateModel.fs:line 575
   at FSharp.Formatting.ApiDocs.CrossReferenceResolver.registerMember(FSharpMemberOrFunctionOrValue memb) in C:\Users\Kevin\source\repos\fsprojects\FSharp.Formatting\src\FSharp.Formatting.ApiDocs\GenerateModel.fs:line 647
   at FSharp.Formatting.ApiDocs.CrossReferenceResolver.registerEntity(FSharpEntity entity) in C:\Users\Kevin\source\repos\fsprojects\FSharp.Formatting\src\FSharp.Formatting.ApiDocs\GenerateModel.fs:line 669
   at <StartupCode$FSharp-Formatting-ApiDocs>.$GenerateModel.assemblies@2149.Invoke(FSharpEntity arg00) in C:\Users\Kevin\source\repos\fsprojects\FSharp.Formatting\src\FSharp.Formatting.ApiDocs\GenerateModel.fs:line 2149
   at Microsoft.FSharp.Collections.SeqModule.Iterate[T](FSharpFunc`2 action, IEnumerable`1 source) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 497
   at FSharp.Formatting.ApiDocs.ApiDocModel.Generate(FSharpList`1 projects, String collectionName, FSharpOption`1 libDirs, FSharpOption`1 otherFlags, Boolean qualify, FSharpOption`1 urlRangeHighlight, String root, FSharpList`1 substitutions, Boolean strict) in C:\Users\Kevin\source\repos\fsprojects\FSharp.Formatting\src\FSharp.Formatting.ApiDocs\GenerateModel.fs:line 2149
   at FSharp.Formatting.ApiDocs.ApiDocs.GenerateHtmlPhased[a](FSharpList`1 inputs, String output, String collectionName, FSharpList`1 substitutions, FSharpOption`1 template, FSharpOption`1 root, FSharpOption`1 qualify, FSharpOption`1 libDirs, FSharpOption`1 otherFlags, FSharpOption`1 urlRangeHighlight, FSharpOption`1 strict) in C:\Users\Kevin\source\repos\fsprojects\FSharp.Formatting\src\FSharp.Formatting.ApiDocs\ApiDocs.fs:line 54
   at <StartupCode$fsdocs>.$BuildCommand.runGeneratePhase1@577-1.Invoke(Unit unitVar0) in C:\Users\Kevin\source\repos\fsprojects\FSharp.Formatting\src\FSharp.Formatting.CommandTool\BuildCommand.fs:line 601
   at <StartupCode$fsdocs>.$BuildCommand.protect@338(CoreBuildOptions this, FSharpFunc`2 f) in C:\Users\Kevin\source\repos\fsprojects\FSharp.Formatting\src\FSharp.Formatting.CommandTool\BuildCommand.fs:line 340

Would it be valuable to add a command line argument (maybe something like --dependencyResolvePaths or --dependencyPath(s)) to pass folders such as the packages folder when using paket with storage:packages or the nuget cache path (which may be a custom path) or is there already something similar going on?

[dsyme]

Would it be valuable to add a command line argument (maybe something like --dependencyResolvePaths or --dependencyPath(s)) to pass folders such as the packages folder when using paket with storage:packages or the nuget cache path (which may be a custom path) or is there already something similar going on?

We could, perhaps as an interim measure, though I think this is the solution we may need to follow: "ideally we'd point everything at a tests folder (i.e. an application) that has all dependencies bundled."

It may also be possible to set some flag in the project file marking the project as a test DLL (even if it actually isn't), to ensure dependencies get copied to the output directory. That may have the side effect of zero tests being found on test execution though. Not sure if it's a problem

[teo-tsirpanis]

.NET Core has a type that does the entire work for us.

[dsyme]

.NET Core has a type that does the entire work for us.

I see thanks, nice. Maybe that's enough.

[KevinRansom]

@teo-tsirpanis , I will take a look I think I need it for a similar notebook scenario anyway.

[KevinRansom]

@dsyme , @teo-tsirpanis , I think the suggested API, may help us out in FSI. However, I don't think the compiler itself should work off ambient types, the compiler ideally knows all of the assemblies that contribute to a successful build. If this scenario is important for fsdocs then I suppose that fsdocs should implement an AssemblyLoadContext that does this work.

[nojaf]

Hello, we are having the same problem. One project has a NuGet dependency so the cracking fails from the command line.
A workaround is to provide the reference we are missing:

--fscoptions " " -r:C:\Users\nojaf\Projects\fantomas\src\Fantomas\bin\Debug\net6.0\SemanticVersioning.dll"

Notice that this requires an additional space before the -r, see commandlineparser/commandline#58.

Would a PR be accepted that trims all entries of otherflags in

FSharp.Formatting/src/FSharp.Formatting.ApiDocs/GenerateModel.fs

Line 2988 in 9cb0c7d

let otherFlags = defaultArg otherFlags []

?

This would resolve our generation problem when running the tool.

//cc @yisusalanpng

[dsyme]

Yes, sounds good

[joemphilips]

FYI, I encountered a same error when I use the newest SDK version 6.0.400
But it worked fine when I downgraded to 6.0.303.
I couldn't identify what exactly is the problem by reading the release notes but the downgrade was good enough for me.