DRL-freshMCS

This is the code accompanying the paper: "Mobile Crowdsensing for Data Freshness: A Deep Reinforcement Learning Approach", published in IEEE INFOCOM 2021.

📄 Description

Data collection by mobile crowdsensing (MCS) is emerging as data sources for smart city applications, however how to ensure data freshness has sparse research exposure but quite important in practice. In this paper, we consider to use a group of mobile agents (MAs) like UAVs and driverless cars which are equipped with multiple antennas to move around in the task area to collect data from deployed sensor nodes (SNs). Our goal is to minimize the age of information (AoI) of all SNs and energy consumption of MAs during movement and data upload. To this end, we propose a centralized deep reinforcement learning (DRL)-based solution called “DRL-freshMCS” for controlling MA trajectory planning and SN scheduling. We further utilize implicit quantile networks to maintain the accurate value estimation and steady policies for MAs. Then, we design an exploration and exploitation mechanism by dynamic distributed prioritized experience replay. We also derive the theoretical lower bound for episodic AoI. Extensive simulation results show that DRLfreshMCS significantly reduces the episodic AoI per remaining energy, compared to five baselines when varying different number of antennas and data upload thresholds, and number of SNs. We also visualize their trajectories and AoI update process for clear illustrations.

🔧 Installation

1. Clone repo

   git clone https://github.com/BIT-MCS/DRL-freshMCS.git
   cd DRL-freshMCS

2. Install dependent packages

   conda create -n mcs python==3.8
   conda activate mcs
   conda install pytorch torchvision torchaudio cudatoolkit=11.3 tensorboard future
   pip install -r requirements.txt

💻 Training

Train our solution

python train.py --config realAoI_iqn_lstm.json --log-dir ./rltime_logs

🏁 Testing

Test with the trained models

python eval.py --path ./rltime_logs/your_model_path

Random test the env

python try_real_aoi.py

👏 Reference

• https://github.com/opherlieber/rltime

📜 Acknowledgement

This paper was supported by National Natural Science Foundation of China (No. 62022017).
Corresponding author: Chi Harold Liu.

📧 Contact

If you have any question, please email 3120215520@bit.edu.cn.

Paper

If you are interested in our work, please cite our paper as

@INPROCEEDINGS{dai2021mobile,
    author={Dai, Zipeng and Wang, Hao and Liu, Chi Harold and Han, Rui and Tang, Jian and Wang, Guoren},
    booktitle={IEEE Conference on Computer Communications (INFOCOM)},
    title={Mobile Crowdsensing for Data Freshness: A Deep Reinforcement Learning Approach},   
    year={2021},  
    pages={1-10}
}