日志切割

Logrotate

mkdir /etc/logrotate-eefocus.d
echo "include /etc/logrotate-eefocus.d" > /etc/logrotate-eefocus.conf

# 手动添加cron 
# logrotate
1 0 * * * /usr/sbin/logrotate -fv /etc/logrotate-eefocus.conf  2>&0 >/dev/null

# 直接下载rpm有时候会卡住
cd ~
wget http://rpmfind.net/linux/Mandriva/devel/cooker/x86_64/media/main/release/logrotate-3.8.1-1-mdv2012.0.x86_64.rpm
rpm -ihv --force logrotate-3.8.1-1-mdv2012.0.x86_64.rpm

# 或者从入口scp过去
scp logrotate-3.8.1-1-mdv2012.0.x86_64.rpm xxu@192.168.0.183:

cp /home/xxu/logrotate-3.8.1-1-mdv2012.0.x86_64.rpm .
rpm -ihv --force logrotate-3.8.1-1-mdv2012.0.x86_64.rpm

• 给每个网站新建一个logrotate的配置文件，放在/etc/logrotate-eefocus.d下，设置cron每天0点01分切割日志并发送给日志服务器42。
• 给每个网站都建一个配置，方便后续管理改动。以后肯能会集成进puppet或ansible。
• 把eefocus网站的日志切割和其他的分离了开来，所以可以保证不影响其他日志的切割，centos默认dailly cron是3点，还会有随机延迟，不太适合我们需要统计每天的数据，所以手动加cron，每天0点01分运行。

配置文件模板

# nginx access
/var/log/nginx/www.testxxu.com/access.log {
    copytruncate
    daily
    rotate 7
    compress
    missingok
    dateext
    dateyesterday
    lastaction 
        datetime=`date -d yesterday +%Y%m%d`
        rsync -qa --rsync-path="mkdir -p /data/log-new/www.testxxu.com/192.168.0.163/nginx/access && rsync" /var/log/nginx/www.testxxu.com/access.log-$datetime.gz copylog@192.168.0.42:/data/log-new/www.testxxu.com/192.168.0.163/nginx/access
    endscript
}

# nginx error
/var/log/nginx/www.testxxu.com/error.log {
    copytruncate
    daily
    rotate 7
    compress
    missingok
    dateext
    dateyesterday
    lastaction 
        datetime=`date -d yesterday +%Y%m%d`
        rsync -a --rsync-path="mkdir -p /data/log-new/www.testxxu.com/192.168.0.163/nginx/error && rsync" /var/log/nginx/www.testxxu.com/error.log-$datetime.gz copylog@192.168.0.42:/data/log-new/www.testxxu.com/192.168.0.163/nginx/error
    endscript
}

# php-fpm slow
/var/log/php-fpm/www.testxxu.com/slow.log {
    copytruncate
    daily
    rotate 7
    compress
    missingok
    dateext
    dateyesterday
    lastaction 
        datetime=`date -d yesterday +%Y%m%d`
        rsync -a --rsync-path="mkdir -p /data/log-new/www.testxxu.com/192.168.0.163/php-fpm/slow && rsync" /var/log/php-fpm/www.testxxu.com/slow.log-$datetime.gz copylog@192.168.0.42:/data/log-new/www.testxxu.com/192.168.0.163/php-fpm/slow
    endscript
}

# php-fpm error
/var/log/php-fpm/www.testxxu.com/error.log {
    copytruncate
    daily
    rotate 7
    compress
    missingok
    dateext
    dateyesterday
    lastaction 
        datetime=`date -d yesterday +%Y%m%d`
        rsync -a --rsync-path="mkdir -p /data/log-new/www.testxxu.com/192.168.0.163/php-fpm/error && rsync" /var/log/php-fpm/www.testxxu.com/error.log-$datetime.gz copylog@192.168.0.42:/data/log-new/www.testxxu.com/192.168.0.163/php-fpm/error
    endscript
}

外部网站

网站	类型	服务器IP	logrotate配置文件路径(。。。。。。。。。。。。。。。。。。。。。。。。。)
www.eefocus.com	主站	192.168.0.163 192.168.0.181	/etc/logrotate-eefocus.d/www.eefocus.com
www.eefocus.com/*	文章	192.168.0.158 192.168.0.161 192.168.0.182	/etc/logrotate-eefocus.d/www.eefocus.com.article
www.moore8.com	摩尔吧	192.168.0.162 192.168.0.180	/etc/logrotate-eefocus.d/www.moore8.com
www.eeboard.com	爱板	192.168.0.140 192.168.0.140(Baiduspidedr only)	/etc/logrotate-eefocus.d/www.eeboard.com
www.eeboard.com/bbs	爱板论坛	192.168.0.165 192.168.0.184	/etc/logrotate-eefocus.d/bbs.eefocus.com
jingyan.eeboard.com	爱板经验	192.168.0.165 192.168.0.184	/etc/logrotate-eefocus.d/jingyan.eeboard.com
www.cirmall.com	cirmall	192.168.0.2 (入口)	/etc/logrotate-eefocus.d/www.cirmall.com
bbs.cirmall.com	cirmall_bbs	192.168.0.112)	/etc/logrotate-eefocus.d/bbs.cirmall.com
www.cndzz.com	cndzz	192.168.0.132	/etc/logrotate-eefocus.d/www.cndzz.com
mm.nxpic.org	nxp社区视频	192.168.0.164 192.168.0.183	/etc/logrotate-eefocus.d/mm.nxpic.org
www.nxpic.org	nxp社区	192.168.0.164 192.168.0.183	/etc/logrotate-eefocus.d/www.nxpic.org
www.openhw.org	openhw社区	192.168.0.164 192.168.0.183	/etc/logrotate-eefocus.d/www.openhw.org
www.openhw.org	openhw首页	192.168.0.166	/etc/logrotate-eefocus.d/www.openhw.org
www.infineonic.org	英飞凌社区	192.168.0.164 192.168.0.183	/etc/logrotate-eefocus.d/www.infineonic.org
www.stmcu.org / st.eefocus.com	st社区	192.168.0.164 192.168.0.183	/etc/logrotate-eefocus.d/www.stmcu.org
rf.eefocus.com	rf社区	192.168.0.164 192.168.0.183	/etc/logrotate-eefocus.d/rf.eefocus.com
microchip.eefocus.com	microchip社区	192.168.0.164 192.168.0.183	/etc/logrotate-eefocus.d/microchip.eefocus.com
analog.eefocus.com	analog社区	192.168.0.164 192.168.0.183	/etc/logrotate-eefocus.d/analog.eefocus.com
linear.eefocus.com / adi.eefocus.com	adi社区	192.168.0.164 192.168.0.183	/etc/logrotate-eefocus.d/adi.eefocus.com
ledlight.eefocus.com	led社区	192.168.0.164 192.168.0.183	/etc/logrotate-eefocus.d/ledlight.eefocus.com
rohm.eefocus.com	rohm社区	192.168.0.164 192.168.0.183	/etc/logrotate-eefocus.d/rohm.eefocus.com
search.eefocus.com	搜索	192.168.0.119	/etc/logrotate-eefocus.d/search.eefocus.com
www.datasheet5.com	d5	192.168.0.166 192.168.0.185	/etc/logrotate-eefocus.d/www.datasheet5.com
www.eehub.cn	eehub	192.168.0.2 (入口) node服务(日志未收集):192.168.0.122	/etc/logrotate-eefocus.d/www.eehub.cn
www.yanzhouren.org	兖州人	192.168.0.139	/etc/logrotate-eefocus.d/www.yanzhouren.org
www.ecbcamp.com	ecbcamp	192.168.0.122	/etc/logrotate-eefocus.d/www.yanzhouren.org

内部服务

网站	类型(。。。。。。。。。)	服务器IP	logrotate配置文件路径(。。。。。。。。。。。。。。。。。。。。)
account.eefocus.com	account	192.168.0.158 192.168.0.161	/etc/logrotate-eefocus.d/account.eefocus.com
sso.eefocus.com	sso	192.168.0.114 192.168.0.134	/etc/logrotate-eefocus.d/sso.eefocus.com
simplesamlphp	sso	192.168.0.114 192.168.0.134	/etc/logrotate-eefocus.d/simplesamlphp
gg.eefocus.com	gg广告	192.168.0.137 192.168.0.163	/etc/logrotate-eefocus.d/gg.eefocus.com
sendy.eefocus.com	sendy邮件平台	192.168.0.119	/etc/logrotate-eefocus.d/sendy.eefocus.com
center.eefocus.com	数据平台mysql界面	192.168.0.119	/etc/logrotate-eefocus.d/center.eefocus.com
monitor.eefocus.com		192.168.0.119	/etc/logrotate-eefocus.d/monitor.eefocus.com
url.eefocus.com	短链接	192.168.0.119	/etc/logrotate-eefocus.d/url.eefocus.com
select.eefocus.com	线上数据库mysql界面	192.168.0.119	/etc/logrotate-eefocus.d/select.eefocus.com

数据库

端口	类型	内容	服务器IP	logrotate配置文件路径(。。。。。。。。。。。。。。。。)
3306	master	search,老主站	192.168.0.81	/etc/logrotate-eefocus.d/mysql-3306
3307	master	cndzz	192.168.0.81	/etc/logrotate-eefocus.d/mysql-3307
3308	master	eehub,ecbcamp,以及一些关停的站	192.168.0.81	/etc/logrotate-eefocus.d/mysql-3308
3309	master	d5	192.168.0.81	/etc/logrotate-eefocus.d/mysql-3309
3310	master	eeboard	192.168.0.81	/etc/logrotate-eefocus.d/mysql-3310
3311	master	account用户信息	192.168.0.81	/etc/logrotate-eefocus.d/mysql-3311
3312	master	openx	192.168.0.81	/etc/logrotate-eefocus.d/mysql-3312
3313	master	support(包括短链接，sendy等等)	192.168.0.81	/etc/logrotate-eefocus.d/mysql-3313
3314	master	主站，文章，新闻抓取	192.168.0.81	/etc/logrotate-eefocus.d/mysql-3314
3315	master	moore8	192.168.0.81	/etc/logrotate-eefocus.d/mysql-3315
3316	master	社区	192.168.0.81	/etc/logrotate-eefocus.d/mysql-3316
3306	slave	search,老主站	192.168.0.82	/etc/logrotate-eefocus.d/mysql-3306
3307	slave	cndzz	192.168.0.82	/etc/logrotate-eefocus.d/mysql-3307
3308	slave	eehub,ecbcamp,以及一些关停的站	192.168.0.82	/etc/logrotate-eefocus.d/mysql-3308
3309	slave	d5	192.168.0.82	/etc/logrotate-eefocus.d/mysql-3309
3310	slave	eeboard	192.168.0.82	/etc/logrotate-eefocus.d/mysql-3310
3311	slave	account用户信息	192.168.0.82	/etc/logrotate-eefocus.d/mysql-3311
3312	slave	openx	192.168.0.82	/etc/logrotate-eefocus.d/mysql-3312
3313	slave	support(包括短链接，sendy等等)	192.168.0.82	/etc/logrotate-eefocus.d/mysql-3313
3314	slave	主站，文章，新闻抓取	192.168.0.82	/etc/logrotate-eefocus.d/mysql-3314
3315	slave	moore8	192.168.0.82	/etc/logrotate-eefocus.d/mysql-3315
3316	slave	社区	192.168.0.82	/etc/logrotate-eefocus.d/mysql-3316

其他

网站	类型	服务器IP	logrotate配置文件路径
邮件服务器	邮件服务器	61.155.217.120	/etc/logrotate-eefocus.d/maillog

日志服务器

• 负责日志存储

服务器	路径
192.168.0.42	/data/log-new/