awesome-AIOps

• Awesome AIOps
  • White Paper
  • Course and Slides
  • Industry Practice
  • Article
  • Tools and Algorithms
  • Paper
  • Dataset
  • Useful WeChat Official Accounts

White Paper

• 《企业级 AIOps 实施建议》白皮书

Course and Slides

• Tsinghua-Peidan - AIOps course in Tsinghua.
• 基于机器学习的智能运维

Industry Practice

---

• 腾讯运维的AI实践
• AI 时代下腾讯的海量业务智能监控实践
• 织云Metis时间序列异常检测全方位解析
• 腾讯织云Metis智能运维学件平台开源代码

---

• 阿里全链路监控方案
• 阿里开源4000台服务器真实数据集

---

• 百度智能流量监控实战
• 异常检测：百度是这样做的
• Next Generation of DevOps AIOps in Practice @Baidu [video]

---

• 搭建大规模高性能的时间序列大数据平台
• Yahoo大规模时列数据异常检测技术及其高性能可伸缩架构
• Netflix: Robust PCA
• LinkedIn: exponential smoothing
• Uber: multivariate non-linear model

Article

• 智能运维|AIOps中的四大金刚都是谁？
• A Comparison of Mapping Approaches for Distributed Cloud Applications
• AIOps探索：基于VAE模型的周期性KPI异常检测方法

Tools and Algorithms

• Tools to Monitor and Visualize Microservices Architecture
• python-fp-growth，挖掘频繁项集
• Anomaly Detection with Twitter in R
• 百度开源时间序列打标工具：Curve
• Microsoft开源时间序列打标工具: TagAnomaly
• Anomaly Detection Examples
• facebook/prophet, Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.
• google/CausalImpact, An R package for causal inference in time series
• 时间序列分析之ARIMA
• 时间序列特征提取库tsfresh
• Yahoo EGADS : A Java package to automatically detect anomalies in large scale time-series data
• Awesome Time Series Analysis and Data Mining

Paper

• Survey on Models and Techniques for Root-Cause Analysis
• 基于机器学习的智能运维
• HotSpot: Anomaly Localization for Additive KPIs With Multi-Dimensional Attributes
  • Chinese：清华AIOps新作:蒙特卡洛树搜索定位多维指标异常
• Opprentice: Towards Practical and Automatic Anomaly Detection Through Machine Learning
• Robust and Rapid Clustering of KPIs for Large-Scale Anomaly Detection
• KPI-TSAD: A Time-Series Anomaly Detector for KPI Monitoring in Cloud Applications
• Anomaly Detection Based on Mining Six Local Data Features and BP Neural Network
• Generic and Robust Localization of Multi-Dimensional Root Causes
• Papers from Tsinghua NetMan Lab

Dataset

• Alibaba/clusterdata

• Azure/AzurePublicDataset *

• Google/cluster-data

• The Numenta Anomaly Benchmark(NAB)

• Yahoo: A Labeled Anomaly Detection Dataset

• 港中文loghub数据集

• 2018 AIOPS挑战赛预赛测试集 2018 AIOPS挑战赛预赛训练集

  • 链接已经失效

• 微软数据集

  • 有API的延迟,并且有带标签!!!

Useful WeChat Official Accounts

• 智能运维前沿（清华裴丹团队的）
• AIOps智能运维（百度的）

自己的笔记

1. 基础知识

• 梦家

  讲解的真的很不错,基础知识讲的通透.也有介绍基础算法和最新前沿的算法.

  但不是专门介绍运维方向的.

2.算法

• ARIMA

  • ARIMA相关论文

    [8] A. H. Yaacob, I. K. Tan, S. F. Chien, and H. K. Tan, “Arima based network anomaly detection,” in Communication Software and Networks,2010. ICCSN’10. Second International Conference on. IEEE, 2010, pp. 205–209.

  • 变式 SARIMA ,考虑季节性

  • 代码

• 聚类方法- DBSCAN.

  优点:抗干扰

  缺点:不能探测到change radically.

  • 相关论文 [2019] Unsupervised Online Anomaly Detection with Parameter Adaptation for KPI Abrupt Changes

  • DBSCAN的基础知识

  • 视频

  • 代码

python

• python中pandas读取文件用法

  • pandas可以读取xlsx、xls文件,csv文件,txt文件
  • 返回类型是DataFrame

• DataFrame 的Shift

  • 整列往下移动一行

    data['delay_1'] = data["delay"].shift(1)

• DataFrame 运算的效率比较

  整组直接进行运算最快

  DF['eee'] = DF['aaa'].values + DF['bbb'].values

• 散点图教程