Generative Adversarial User Model

Tensorflow implementation for:

Generative Adversarial User Model for Reinforcement Learning Based Recommendation System [1]

(Currently the ant financial dataset is not authorized to released. Experiments on other public dataset are released.)

Setup

Install

Clone and install the current package.

pip install -e .

Data

The dataset can be obtained via the shared dropbox folder.

After downloading the .txt files in the shared folder, put then under the 'dropbox' folder, so that the default bash script can automatically find them.

Finally the project has the following folder structure:

ganrl
|___ganrl  # source code
|   |___common # common implementations
|   |___experiment_user_model # code for experiments in Sec 6.1 in the paper
|
|___dropbox  # yelp, tb, rsc dataset.
    |___process_data.py
    |___process_data.sh
    |___yelp.txt
    |___tb.txt
    |......
...

Process the data before running the experiments:

cd dropbox
./process_data.sh

Explanation of the original .txt file: The column 'session_new_index' corresponds to user ID. The column 'item_new_index' corresponds to item ID. If several items have the same 'Time' index, then they are displayed at the same time (in the same display set).

Experiments

By modifying the sh scripts, You can tune the hyperparameters like the architecture of the neural networks, learning rate, etc.

GA User Model with Shannon Entropy

Navigate to the experiment folder. You can run the sh script directly or set the hyperparameters by yourself. To try a different split of train, test, validation sets, you can change -resplit False to -resplit True in the sh file.

cd ganrl/experiment_user_model/
./run_gan_user_model.sh

The trained model will be saved in scratch/ folder.

GA User Model with L2 Regularization

First, train the user model using Shannon Entropy by running ./run_gan_user_model.sh. With this saved model as an initilization, you can continue to train the model using other regularizations. For example, L2:

cd ganrl/experiment_user_model/
./run_gan_user_model.sh
./run_gan_L2_regularized_yelp.sh

Citation

If you found it useful in your research, please consider citing

@inproceedings{chen2019generative,
  title={Generative Adversarial User Model for Reinforcement Learning Based Recommendation System},
  author={Chen, Xinshi and Li, Shuang and Li, Hui and Jiang, Shaohua and Qi, Yuan and Song, Le},
  booktitle={International Conference on Machine Learning},
  pages={1052--1061},
  year={2019}
}

References

[1] Xinshi Chen, Shuang Li, Hui Li, Shaohua Jiang, Yuan Qi, Le Song. "Generative Adversarial User Model for Reinforcement Learning Based Recommendation System." In International Conference on Machine Learning. 2019.