Documentation for New Developers

[dufferzafar]

I think we should have some doc pages targetting new contributors.

I was once talking with @s4chin and he had a very simple question "How do I understand mitmproxy's codebase? It's huge and I have no idea where to start."

Understanding new codebases isn't an easy task, but we can surely tell people where to start.

Some contents (off the top of my head):

• The Hacking & Testing section from the Readme
• Describe the basic folder structure
  • what is where
  • what are netlib/pathoc etc.
  • how they fit into mitmproxy
• 'Suggested/Further Readings' for topics like HTTP/2 etc.
  • There are a lot of people who just know python and want to contribute to open source projects but may not know all the advanced networking topics involved.
  • I didn't understand 70% of what is on the How mitmproxy works when I read it for the first time.

Some of the files already have a top-level docstring. Extending this to every file would be great.

Thoughts?

---

Seeing Anaconda's Developer Docs finally made me open this issue.

[cortesi]

I'd be all in favour of better developer documentation. One caveat I'd add is that we should try to keep documentation that belongs in the code in the code. Every bit of external documentation we add is a maintenance burden, and we already have quite a lot of things in the docs that are gently sliding out of date.

What this means concretely is that in addition to a broad guide to developers, we should encourage people to explore the specific interfaces in the project with tools like pydoc or perhaps rendered documentation. There's a lot we can improve there. Each module should have a brief statement of purpose, for instance. Each class should have a class-level explanation if it's not immediately obvious from the name. It's instructive to just fire up pydoc on various bits of the project and notice just how under-documented things are at the moment.

[Sauron754]

As a newomer myself I am experiencing a big difficulty in understanding mimtproxy. For me the biggest problem is to understand the module structures. I think it would really help if there were a module-tree documentation (Maybe there is already one somewhere but I am just too incompetent to find it). There is a very big posibility that a lot of new contributers would come if you explain the code structures of mitmproxy and therefore make the entry for newcomers easier.

[cortesi]

@Sauron754 Yes, my subjective impression is that the structure of the project has become quite a bit harder to understand in the last year or so, partly just due to natural growth of the codebase. Some of this we can tackle with documentation, and some of it is refactoring to make things easier to understand by improving modularity (the recent addons changes have been a big win there, for instance).

Is there any part of the project you remember finding particularly hard to understand?

[Sauron754]

@cortesi Not really. As I've said the biggest problem was/is the module structure. But I'll inform you if I find something.

[dufferzafar]

Something similar: Guido and others are working on a guided tour to CPython's source code. (see discussion)