Throw a consistent error object from makedocs on user errors

[mortenpi]

Inspired by JuliaDocs/DocumenterCitations.jl#59 (comment):

I agree it's not great that Documenter shows a Stacktrace when it fails in strict mode. It makes it look like an internal error.

The doccheck error would now print as:

ERROR: LoadError: DocumenterError (makedocs): Documenter build failed to complete.
Underlying exception:
DocCheckError: `makedocs` encountered errors [:cross_references, :linkcheck] -- terminating build before rendering.
Please check the error logs above as well.
Stacktrace:
 [1] makedocs(; debug::Bool, format::Documenter.HTMLWriter.HTML, kwargs::Base.Pairs{Symbol, Any, NTuple{7, Symbol}, NamedTuple{(:sitename, :build, :remotes, :pagesonly, :pages, :warnonly, :linkcheck), Tuple{String, String, Dict{String, Tuple{Documenter.Remotes.GitHub, String}}, Bool, Vector{String}, Vector{Symbol}, Bool}}})
   @ Documenter ~/juliadocs/Documenter/src/makedocs.jl:263
 [2] makedocs
   @ ~/juliadocs/Documenter/src/makedocs.jl:248 [inlined]
 [3] top-level scope
   @ ./timing.jl:273
in expression starting at /home/mortenpi/juliadocs/testbuild/make.jl:109

caused by: DocCheckError: `makedocs` encountered errors [:cross_references, :linkcheck] -- terminating build before rendering.

Stacktrace:
 [1] runner(#unused#::Type{Documenter.Builder.RenderDocument}, doc::Documenter.Document)
   @ Documenter ~/juliadocs/Documenter/src/builder_pipeline.jl:258
 [2] dispatch(#unused#::Type{Documenter.Builder.DocumentPipeline}, x::Documenter.Document)
   @ Documenter.Selectors ~/juliadocs/Documenter/src/utilities/Selectors.jl:170
 [3] #79
   @ ~/juliadocs/Documenter/src/makedocs.jl:257 [inlined]
 [4] withenv(::Documenter.var"#79#81"{Documenter.Document}, ::Pair{String, Nothing}, ::Vararg{Pair{String, Nothing}})
   @ Base ./env.jl:197
 [5] #78
   @ ~/juliadocs/Documenter/src/makedocs.jl:256 [inlined]
 [6] cd(f::Documenter.var"#78#80"{Documenter.Document}, dir::String)
   @ Base.Filesystem ./file.jl:112
 [7] makedocs(; debug::Bool, format::Documenter.HTMLWriter.HTML, kwargs::Base.Pairs{Symbol, Any, NTuple{7, Symbol}, NamedTuple{(:sitename, :build, :remotes, :pagesonly, :pages, :warnonly, :linkcheck), Tuple{String, String, Dict{String, Tuple{Documenter.Remotes.GitHub, String}}, Bool, Vector{String}, Vector{Symbol}, Bool}}})
   @ Documenter ~/juliadocs/Documenter/src/makedocs.jl:255
 [8] makedocs
   @ ~/juliadocs/Documenter/src/makedocs.jl:248 [inlined]
 [9] top-level scope
   @ ./timing.jl:273

I was hoping it would clean things up a bit (the main error stracktrace just shows makedocs -- best we can do I think). But the caused by might be confusing now.. and it's kinda duplicated in the showerror I added. We could get rid of it by throwing outside of the catch, but it feels like it would be good to somehow retain some information about the inner error. Leaving this as a draft for now -- would appreciate any thoughts on this!

cc @goerz

[goerz]

I vaguely remember there being a way to discard the “caused by” information… let me look into that later tonight.

[goerz]

I must be misremembering… or at least I can't figure out how to directly manipulate the exception stack or suppress the "caused by" message. Having the "caused by" show up like that definitely defeats the purpose, though. I think throwing outside of the catch should be fine.

If you really want to preserve the full original backtrace, the best I've been able to come up with this the following:

function makedocs(; debug = false, format = HTML(), kwargs...)
    exception = nothing
    try
        document = Documenter.Document(; format=format, kwargs...)
        # Before starting the build pipeline, we empty out the subtype cache used by
        # Selectors.dispatch. This is to make sure that we pick up any new selector stages that
        # may have been added to the selector pipelines between makedocs calls.
        empty!(Selectors.selector_subtypes)
        cd(document.user.root) do
            withenv(NO_KEY_ENV...) do
                Selectors.dispatch(Builder.DocumentPipeline, document)
            end
        end
        return debug ? document : nothing
    catch e
        if isa(e, DocumenterBuildException)
            exc_error_log = IOBuffer()
            showerror(exc_error_log, e, catch_backtrace())
            @error String(take!(exc_error_log))  # XXX: better: @debug
            exception = DocumenterError(:makedocs, (e, catch_backtrace()))
        else
            rethrow(e)
        end
    end
    isnothing(exception) || throw(exception)
end

For the test case I was playing with (an intentional error in the citations), it produces the following

output

● goerz@ophelia master:~/Documents/Programming/DocumenterCitations.jl> make docs
julia --project=test docs/make.jl
Starting makedocs
[ Info: SetupBuildDirectory: setting up build directory.
[ Info: Doctest: running doctests.
[ Info: ExpandTemplates: expanding markdown templates.
[ Info: CollectCitations
┌ Error: Key "xxxGoerzQ2022" not found in entries from /Users/goerz/Documents/Programming/DocumenterCitations.jl/docs/src/refs.bib
└ @ DocumenterCitations ~/Documents/Programming/DocumenterCitations.jl/src/collect_citations.jl:92
[ Info: ExpandBibliography: expanding `@bibliography` blocks.
[ Info: ExpandCitations
┌ Error: expand_citation: No destination for key="xxxGoerzQ2022" → unlinked text "GoerzQ2022"
└ @ DocumenterCitations ~/Documents/Programming/DocumenterCitations.jl/src/expand_citations.jl:139
[ Info: CrossReferences: building cross-references.
[ Info: CheckDocument: running document checks.
[ Info: Populate: populating indices.
┌ Error: DocCheckError: `makedocs` encountered an error [:citations] -- terminating build before rendering.
│
│ Stacktrace:
│   [1] runner(#unused#::Type{Documenter.Builder.RenderDocument}, doc::Documenter.Document)
│     @ Documenter ~/Documents/Programming/Documenter.jl/src/builder_pipeline.jl:258
│   [2] dispatch(#unused#::Type{Documenter.Builder.DocumentPipeline}, x::Documenter.Document)
│     @ Documenter.Selectors ~/Documents/Programming/Documenter.jl/src/utilities/Selectors.jl:170
│   [3] #79
│     @ ~/Documents/Programming/Documenter.jl/src/makedocs.jl:258 [inlined]
│   [4] withenv(::Documenter.var"#79#81"{Documenter.Document}, ::Pair{String, Nothing}, ::Vararg{Pair{String, Nothing}})
│     @ Base ./env.jl:197
│   [5] #78
│     @ ~/Documents/Programming/Documenter.jl/src/makedocs.jl:257 [inlined]
│   [6] cd(f::Documenter.var"#78#80"{Documenter.Document}, dir::String)
│     @ Base.Filesystem ./file.jl:112
│   [7] makedocs(; debug::Bool, format::Documenter.HTMLWriter.HTML, kwargs::Base.Pairs{Symbol, Any, NTuple{6, Symbol}, NamedTuple{(:authors, :linkcheck, :warnonly, :sitename, :pages, :plugins), Tuple{String, Bool, Vector{Symbol}, String, Vector{Pair{String, String}}, Vector{CitationBibliography}}}})
│     @ Documenter ~/Documents/Programming/Documenter.jl/src/makedocs.jl:256
│   [8] top-level scope
│     @ ~/Documents/Programming/DocumenterCitations.jl/docs/make.jl:21
│   [9] include(mod::Module, _path::String)
│     @ Base ./Base.jl:457
│  [10] exec_options(opts::Base.JLOptions)
│     @ Base ./client.jl:307
│  [11] _start()
│     @ Base ./client.jl:522
└ @ Documenter ~/Documents/Programming/Documenter.jl/src/makedocs.jl:265
ERROR: LoadError: DocumenterError (makedocs): Documenter build failed to complete.
Underlying exception:
DocCheckError: `makedocs` encountered an error [:citations] -- terminating build before rendering.
Please check the error logs above as well.
Stacktrace:
 [1] makedocs(; debug::Bool, format::Documenter.HTMLWriter.HTML, kwargs::Base.Pairs{Symbol, Any, NTuple{6, Symbol}, NamedTuple{(:authors, :linkcheck, :warnonly, :sitename, :pages, :plugins), Tuple{String, Bool, Vector{Symbol}, String, Vector{Pair{String, String}}, Vector{CitationBibliography}}}})
   @ Documenter ~/Documents/Programming/Documenter.jl/src/makedocs.jl:272
 [2] top-level scope
   @ ~/Documents/Programming/DocumenterCitations.jl/docs/make.jl:21
in expression starting at /Users/goerz/Documents/Programming/DocumenterCitations.jl/docs/make.jl:21
make: *** [Makefile:36: docs] Error 1
```

Having the "caused by" info earlier in an @error-log (from the line marked with XXX) makes it much more readable, albeit still verbose. Personally, I think it would be okay to use @debug instead of @error in the XXX line. That way, the backtrace is pretty concise for normal usage, but people can get the more verbose version by activating debug-logging for Documenter.