Show proper error to user when conf.py is not found #3326
I added a couple of tests for old behaviour and also to catch this PR behaviour. Besides, I added the exception message as a class attribute.
Please, re-review this PR when you have some time.
(the last commit is styling, so you can skip it from the logic review)
This PR should be ready to merge.
The linting error is unrelated.
I didn't find why it's failing yet.