[docs] Documentation restructure

[bmorelli25]

For #616. Demo available here.

I'd like to backport this to 4.x. For 5.x, I'm guessing the best way is to just wait for the next release?

• Top level heading and doc sync for log correlation: [docs] Agents should have a top level heading for Log correlation apm#149
• Use shared version attributes: [docs] Agents should use shared version/attribute files apm#150
• Add a section on how the Agent does instrumentation: Add how agents do instrumentation section for each agent apm#146
• Rename the APM UI to "APM app": https://github.com/elastic/apm-dev/issues/567
• Use apm-app-ref: Add APM app attribute docs#1276
• Updates to the new documentation structure outlined here
• Sync Central config docs: Central configuration implementation apm-agent-dotnet#497 (comment)

[bmorelli25]

One of the things this PR aims to do is add in a section on how the agent does instrumentation. I've added some boilerplate text here to hopefully assist with this. There are two {CONTENT_HERE} sections above that I'll need one of you to fill out. Feel free to write something different or amend this as necessary.

I've also updated elastic/apm#146 with links to examples in other Agent's docs.

[bmorelli25]

@beniwohli, think you'll have time for a review on this one and #619 at some point next week?

[beniwohli]

@bmorelli25 I'll take a look today, and try to carve out some time tomorrow to write the "how we instrument" section.

[beniwohli]

LalmostGTM, awesome!

[basepi]

Once bmorelli25#1 is merged this looks great.

[bmorelli25]

Moving the conversation to this PR:

@beniwohli, I moved the majority of your content to the Advanced topics section. I kept just few high-level paragraphs in the getting started section. Here's what I kept (feel free to edit)

[[how-it-works]]
=== How does the Agent work?

The Python Agent instruments your application to collect APM events in a few different ways:

To collect data about incoming requests and background tasks, the Agent integrates with <<supported-technologies,supported technologies>> to make use of hooks and signals provided by the framework.
These framework integrations require limited code changes in your application.

To collect data from database drivers, HTTP libraries etc.,
we instrument certain functions and methods in these libraries.
Instrumentations are set up automatically and do not require any code changes.

In addition to APM and error data,
the Python agent also collects system and application metrics in regular intervals.
This collection happens in a background thread that is started by the agent.

More detailed information on how the Agent works can be found in <<link-here>>.

cc @basepi

[basepi]

@bmorelli25 I think your changes are 👍 👍 -- it was definitely too technical for a Getting Started page, but valuable for a click-through, so this is perfect.

Instrumentations are set up automatically and do not require any code changes.

This is the only piece I'm unsure on -- technically you do need code changes, you have to import our library or add us to the INSTALLED_APPS -- maybe something like this?

Instrumentations are set up automatically and do not require any code changes, beyond the initial top-level setup.

^ I don't love that but I'm trying to stay succinct. Or maybe I'm being too picky on the details?

[bmorelli25]

That's a good point @basepi. I had originally copied the "Instrumentations are set up automatically and do not require any code changes." sentece directly from Beni's PR. I like the change here though. I'll make it tomorrow unless @beniwohli has any objections.

[bmorelli25]

Follow up question -- anything I can do to get the continuous-integration/appveyor/pr — AppVeyor build failed check to pass?

[beniwohli]

Looks great! Regarding windows tests, those are unfortunately somewhat flakey. I gave them another go, it's all green now.