Have one clear API for calling Sphinx without unintended side-effects (deprecate SphinxTestApp?)

[chrisjsewell]

Currenty, there is no clearly documented way to "properly" use Sphinx programatically.
By "properly", I mean in a way that could be called multiple times within the same Python interpretor, without any unexpected polluting of the global state, and thus act exactly the same as if each call had been made in a separate Python interpretor

At present, there are diverging ways to do this, for the main CLI and for test cases:

with patch_docutils(confdir), docutils_namespace():
	app = Sphinx(...)
    app.build(...)

app = SphinxTestApp(...)
app.build(...)
app.cleanup()

Its not really clear from first appearance, but these actually almost achieve the same thing:

• docutils_namespace, on exit, resets the directives/roles lookups and additional node visitors
• SphinxTestApp.cleanup resets the directives/roles lookups and additional node visitors

However, there are also clear differences:

• patch_docutils temporarily patches docutils in a number of ways, that do not appear to be addressed by SphinxTestApp, in essence meaning you are not strictly testing the behaviour of Sphinx in production when using SphinxTestApp
• SphinxTestApp.cleanup does addition global state cleaning, restoring: sys.path, sphinx.locale.translators, and sphinx.pycode.ModuleAnalyzer.cache

I feel this divergence is unecessary, and ideally there would be one clear way to call sphinx as an API, perhaps something like:

with Sphinx.restore_global_state(confdir):
	app = Sphinx(...)
	app.build(...)

cc @picnixz and @jayaddison for discussion, as it relates to #12093 etc

---

An additional note (as I just mentioned in #12089 (comment))
is that it is not currently possible to temporarily turn-off colored logging output.
This is annoying for if you want to assert the content of the warning stream

In the CLI it can be turned off, but is then never reset to its previous state: https://github.com/chrisjsewell/sphinx/blob/fa290049515c38e68edda7e8c17be69b8793bb84/sphinx/cmd/build.py#L327