LM

Code for the LM project collaboration

It may be helpful to install this git repo for the maze code: https://github.com/MattChanTK/gym-maze.

Required Packages: Quandl, Gym, Pygame, Pandas, Sklearn, Torchvision, Pytorch, Tqdm, Scipy

Optional Packages: Comet_ml

Initial Stock Experiments

• stockSynthesis.py - simple LSTM close price prediction given the corresponding open price
• predictOpen.py - predict the next open price given the previous one
• LSTM_Sequential.py - predict an open price sequence given a sequence of previous open prices
• stockDataset.py - manipulate kaggle stock data into sequences of open prices
• stockAgent.py - reinforcement learning agent to learn a multiplier to produce the next open price
• stockRL.py - runner code for the Stock Agent to predict open prices
• pytorchSequentialPredict.py - LSTM_Sequential.py but implemented in pytorch instead of keras

Q Learning and Feudal RL Maze Experiments

• maze_2d_q_learning.py - solve a maze using Q learning
• maze_env.py - creates the base maze environment for these experiments
• maze_view_2D.py - helper for maze_env.py
• maze_2d_dqn.py - solve a maze using Deep Q learning
• maze_multiscale_qlearning.py - solve a maze using Feudal Q learning
• multi_maze.py - creates the environment for the feudal learning experiments
• maze2d_002.npy - maze file for the manager (2x2 maze)
• maze2d_003.npy - maze file for the worker (4x4 maze)
• maze_qmodels.py - contains code for the worker and manager networks for maze_multiscale_qlearning.py
• maze_multiscale_dqn.py - solve a maze using Feudal Deep Q Networks
• maze_dqn_models.py - contains code for the worker and manager networks for maze_multiscale_dqn.py

Stock Agent Portfolio Experiments

• stock_qlearning.py - portfolio experiment with a Q learning agent
• stock_env.py - stock market environment for stock_qlearning.py
• stock_dqn.py - portfolio experiment with a DQN agent
• stock_env_dqn.py - stock market environment for stock_dqn.py
• multiscale_q_stock.py - portfolio experiment with feudal Q learning
• multi_stock_env.py - stock market environment for multiscale_q_stock.py
• stock_qmodles.py - contains the worker and manager code for multiscale_q_stock.py
• stock_multi_dqn.py - portfolio experiment with feudal DQNs
• multi_stock_env_dqn.py - stock market environment for stock_multi_dqn.py
• stock_dqn_models.py - contains the worker, manager, and other helper classes for multi_stock_env_dqn.py
• multi_transaction_feudal.py - feudal portfolio experiment where the manager chooses from several workers with different levels of temporal abstraction
• multi_trans_stock_env.py - stock market environment for multi_transaction_feudal.py
• hard_coded_qLearning.py - portfolio experiment with a hard coded agent as a baseline
• dynamicM_hardcodeW.py - feudal portfolio experiment where the manager chooses from several hard coded workers
• hard_coded_agents.py - contains the hard coded agents for dynamicM_hardcodeW.py

Steering Angle and TSNE Prediction

• mutAngTSNE.py - use the TSNE embedding as the subroutine ID when predicting steering angles; also capable of predicting multiple steering angles out
• predZKomanda.py - use a modified version of the winning udacity steering angle challenge network to predict steering angles and subroutine IDs
• tsnePredict.py - predict the TSNE centroids using images (poor results)
• udacityData.py - dataset class to process the udacity driving dataset
• komanda.py - predict steering angles using the winning udacity steering angle challenge network
• tsnePrevDataPred.py - predict the next TSNE centroid given the input data for the previous centroid (poor results)