Make CI builds fail when building docs fails

[nalimilan]

The current recommended Travis CI configuration does not mark a package's build as failed when the docs fail to build. This is problematic since it means the documentation can silently fail to be updated. In particular, doctests can fail due to changes done in a PR without the CI checks making this apparent.

Wouldn't it make sense to make builds fail by default if docs failed to build? Or is that difficult to implement?

[mortenpi]

Build stages is the way to go here: https://docs.travis-ci.com/user/build-stages

But you can also just run Documenter in your tests (i.e. have the include() in test/runtests.jl) if you'd like them to fail when deployment fails. Not much to implement here, although it's not a very pretty solution. Hence, I was extremely excited when I heard about build stages. Needs to be tested and documented though.

[nalimilan]

Interesting. Any idea what a .travis.yml file using stages could look like?

[mortenpi]

I played around with them a bit when they came out: .travis.yml and the corresponding Travis build.

Edit: I think for a package the it would look like:

jobs:
  include:
    - stage: "Deploy docs"
      julia: nightly
      os: linux
      script: julia --color=yes -e 'Pkg.add("Documenter")'
        && julia --color=yes docs/make.jl deploy

Edit2: Note that I have deploydocs wrapped in an ARGS check there:

"deploy" in ARGS && deploydocs(
    repo = "github.com/JuliaDocs/Documenter.jl.git",
    target = "build",
    deps = nothing,
    make = nothing,
)

[nalimilan]

Looks great. We should probably wait for Build Stages to be out of beta status, and then recommend it?

[mortenpi]

Yup, that was partially why I didn't go much further than just playing with it at the time.

[mforets]

Hello,

Thanks for this great documentation generator that we extensively use in our projects.

But you can also just run Documenter in your tests (i.e. have the include() in test/runtests.jl) if you'd like them to fail when deployment fails.

Would you mind extending a tiny bit on this approach? Which include() should I use here?

[mortenpi]

Would you mind extending a tiny bit on this approach? Which include() should I use here?

Just include("../docs/make.jl) in runtests.jl (Documenter itself does it, actually). I would suggest looking into build stages though.. they are still officially beta, but my guess is that they are here to stay and the basic interface won't change.

[mforets]

Thanks for your reply. I tried with your example and it understands the "Deploy Docs job", but there are many fails (on purpose) and it still gives the green tick. Details here

[mortenpi]

This is now the recommended way and has been documented.