Support partial builds with serve

[chanified]

Is there any way to speed up the rendering when 'mkdocs serve' is running?

Currently I have approximately 750 markdown documents in the docs folder and when I make a small change to a file it takes approximately 3-5 minutes to update.

It's as if everything that sits in the docs file needs to be refactored, not the single document itself.

Anyone know how to overcome this?

[facelessuser]

It's not slow rendering that is your problem, it is that you want incremental builds. I imagine all 750 files are getting rebuilt every edit. I'm not sure if Mkdocs is targeting users with massive document archives. While I think it could be possible to add incremental builds, someone from the Mkdocs team would need to decide if that is in a direction they are willing to go.

[chanified]

It would make sense for all the files to be cached or compiled somewhere and individually updated as you change them or until the next time you run 'mkdocs serve' where the compiling could take place.

If it's not for scalable document sites, can you imagine all the sites that when they do get sized up, they're going to have to look for alternatives and jump ship, so one would think that it's not a good solution for running your docs in the first place!

Would be good to make it a scalable solution, or at least indicate that it's for small-end documentation if that's the way they want it.

[waylan]

This was discussed in some detail in #971. The conclusion there is that we just need to take the time to do the work and make sure we do it right. As there is no open issue for this, I'll leave this open and rename the title.

[waylan]

Following up on the conversation started in #971:

the fact that --clean isn't the default is a bug and we should change that for 1.0 and perhaps add a new flag that means --dont-clean, but with a better name.

This would actually be pretty easy. Boolean Flags can support two flags to enable/disable a feature. We can just add the second flag and change the default to be --clean. We don't even need to deprecate anything. Existing users who have --clean hard-coded into scripts will not get any errors. And they can feel good that they will always get clean builds regardless of what the default is. Those leaving --clean out of their scripts will benefit from actually getting clean builds for production. True, those who actually want "dirty" builds will need to explicitly add it, but given the fact that there is currently no real benefits, the overall effect should be minimal.

A few proposed names were --develop or --quick. Either would work, but I like --dirty. It is a clear opposite of "clean" and it explicitly makes clear that the build will not be clean. Regardless, whoever builds the shed can paint it...

I would like to see a smarter cache be explored, perhaps there is a way we can cover enough of the edge cases to make it okay.

Ideally something like this would be best done as a plugin, then we can have as many opinionated caches as people like, but that is obviously tricky at the moment.

As we work on #206 we'll try to keep this in mind. That said, I don't see any reason why the clean/dirty switch could be made now. As it stands now, --clean only deletes the build_dir prior to starting a new build. That shouldn't need to change, so any plugin could implement whatever cache system it wants with no additional changes to the core (aside from implementing the Plugin API).

As the serve command always uses a tmp dir as the build_dir (the build_dir setting is ignored/overriden), we always get a clean build on starting the serve command. The issue there is when rebuilding under the livereload server. Having serve accept the --clean/--dirty flags and passing them on to the subsequent rebuilds should be sufficient to allow a plugin to take over from there. A --dirty flag would simply cause the build_dir to not be wiped on a rebuild, and the cache system implemented by the plugin could be used to determine which pages need to be rebuilt or skipped.