VM Placements Datasets

Overview

The VM Placements Datasets is published by Huawei Group. By providing the standard datasets of virtual machine scheduling, it can help researchers better study and train the virtual machine scheduling algorithms. We are open source based on CC-BY-NC. Please follow the open source protocol when using datasets.

So far, one dataset has been released:

• Huawei-East-1 includes about 125429 data, which is formed with 5 attributes:vmid,cpu,memory,time and type. The Huawei-East-1 is the first standard dataset for virtual machine scheduling which can help different scheduling algorithms. To see more about this dataset, see related documents (Huawei-East-1).

We encourage anyone to use the datasets for study or research purposes, and if you had any question when using the datasets, please file an issue on Github. Filing an issue is recommanded as the discussion would help all the community. Note that the more clearly you ask the question, the more likely you would get a clear answer.

We are open source based on CC-BY-NC. When using datasets, please follow the open source protocol, and cite our paper.

@article{sheng2021vmagent,
  title={VMAgent: Scheduling Simulator for Reinforcement Learning},
  author={Sheng, Junjie and Cai, Shengliang and Cui, Haochuan and Li, Wenhao and Hua, Yun and Jin, Bo and Zhou, Wenli and Hu, Yiqiu and Zhu, Lei and Peng, Qian and others},
  journal={arXiv preprint arXiv:2112.04785},
  year={2021}
}

In future, we will try to release new datasets, please stay tuned.

Motivation

The project has created the first standard data set of virtual machine scheduling in China. The project aims to achieve the following three objectives:

1. Increase industry influence and promote industry development. After creating the virtual machine scheduling standard datasets, it can continuously optimize and iterate the virtual machine scheduling algorithms under various scenarios, and increase efficiency increase with lower cost.

2. Help Huawei's cloud development. By promoting the development of virtual machine scheduling algorithms, the understanding of scheduling problems can be continuously deepened. Meanwhile, it can develop the platform for industry-university research cooperation , and increase sales persuasion, which attachs great importance for HUAWEI Cloud.

3. Attract talents and experts to pay attention to scheduling problems. At present, there is no standard datasets for virtual machine scheduling. This project hopes that after creating the standard datasets, it can stimulate academic cooperation, and attract more talents and experts to pay attention to scheduling problems.

Outcomes from the datasets

The original intention of our standard datasets for virtual machine scheduling is to provide more algorithms and projects as standards for training, so as to better promote the development of the industry. It is a huge encouragement to us to see more works using our data. We are open source based on CC-BY-NC. When using datasets, please follow the open source protocol, and cite our paper.

@article{sheng2021vmagent,
  title={VMAgent: Scheduling Simulator for Reinforcement Learning},
  author={Sheng, Junjie and Cai, Shengliang and Cui, Haochuan and Li, Wenhao and Hua, Yun and Jin, Bo and Zhou, Wenli and Hu, Yiqiu and Zhu, Lei and Peng, Qian and others},
  journal={arXiv preprint arXiv:2112.04785},
  year={2021}
}

Projects using VM Placement Datasets

• VMAgent, is a platform for exploiting Reinforcement Learning (RL) on Virtual Machine (VM) scheduling tasks. It is developed by the Multi-Agent Artificial Intelligence Lab (MAIL) in East China Normal University and Algorithm Innovation Lab in HUAWEI Cloud.VMAgent is constructed based on one month real VM scheduling dataset called Huawei-East-1 from HUAWEI Cloud and it contains multiple practicle VM scheduling scenarios (such as Fading, Rcovering, etc).

Papers using VM Placement Datasets

• Junjie Sheng, Shengliang Cai, Haochuan Cui, Wenhao Li, Yun Hua, Bo Jin, Wenli Zhou, Yiqiu Hu, Lei Zhu, Qian Peng, Hongyuan Zha and Xiangfeng Wang, VMAgent: Scheduling Simulator for Reinforcement Learning. arXiv preprint arXiv:2112.04785, 2021.
• Junjie Sheng, Yiqiu Hu, Wenli Zhou, Lei Zhu, Bo Jin, Jun Wang and Xiangfeng Wang, Learning to Schedule Multi-NUMA Virtual Machines via Reinforcement Learning, Pattern Recognition, 121, 2021, pp.108254.

LISCENCE

Licensed under the CC-BY-NC License.

Future works

More datasets will be released in future.