Deflect 'Cruncher' Awstats Log Analysis
This is the method Deflect does its Awstats setup for all edges running Apache Traffic Server.
Directory Structure and Files
- Installed in /var/deflect/
- ... /
- ... /
The Scripts ( bin/ )
- This script just does a copy of the config and awstats data files before running anything. For easier recovery if anything goes wrong.
- This script reads a list of edges and rsync the ATS log files '-.gz' to a archive
- Gets date and looks for zipped file in archive that match the right date. Unzips to a temp location in a separated YYYYMM directory If file was already unzipped in this directory before it does a 'cat /dev/null > old_file'
- Uses some of awstats tools to merge the unzipped log files together into one file YYYMM/current.log
- Read all the Awstats configs setup and forks the to all crunch using YYYYMM/current.log
- Runs all the files above
- A simple config creator script to add a client to Awstats
- you need (domain, config name, password)
- Also gives a option to manually add REGEX for advanced REGEX
Configuration ( etc/ )
- List of all edges we are collecting log files from.
- Private ssl key so scripts can access (rsync) logs
Locking ( lock/ )
Scripts write files in here to make sure things finish. If a problem occurs, like a edges could not get collected from, you can later just delete the file inside this directory and do a manual run.