Document Event Logs

[strawgate]

Problem Description

Event logs are very useful for determining when and why Crawler decides to skip a url that is on a page it has been asked to crawl.

Right now we have some instructions in the config.yml for enabling event logging and a sample filebeat config for sending them to elasticsearch, but there is very little indicating to a user that the event logs would be useful to enable or to review. And no information on how to review them.

Proposed Solution

We should document what event logs are, when to use them, how to enable them, what they are for, and how to interpret them. We should point to them as the way to troubleshoot issues with unexpected pages appearing in crawl results or expected pages not appearing in crawl results, etc.

[mattnowzari]

Closing this, as documentation for Event logs was added in this PR