-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to train offline agent on the huge dataset (50 Million) ? #10
Comments
That's correct, it's not possible to load the entire dataset into memory. So, the way the dataset is stored is in 50 files of size 1M each corresponding to replay buffers for every 4 iterations seen during training. Now, to load this huge dataset, I simply created a new FixedReplayBuffer class for loading the offline dataset which is created in the offline agents (e.g., here) which loads 5 random replay buffers of size 1M at each iteration from the set of 50 buffers. That said, experiment with the loading buffers of 5M still has a large RAM requirement, so there are two possible alternatives:
The way to do this is very simple: simply set the size of the replay buffer to be smaller than 1M (let's say 50000, so it'll only load the first 50000 data points from the dataset files, this can done through changing the size in the The following functions for the
Please let me know if you have any more questions! |
Hi, @agarwl, thank you for your detailed reply! I have learned a lot from your reply, now my question has been answered. |
Reopening this for visibility :) |
Hi, I have read your paper which was published on ICML 2020, now I try to do some research on the offline image data. I have noticed that when training the online agent, such as DQN, replay buffer capacity is usually set to be 1 million, when the size of collected data is above 1 million, the new data will cover the oldest data in the replay buffer. But when training DQN on offline data, such as your contributed data, the size of data is 50M, how do I train the agent on this so huge dataset? Since memory of the machine is limited, it's impossible to load 50M data into the memory once. I wonder that how you solved this problem and if you implement your idea in this project, please refer it to me. At last, I really appreciate your great job and your open-source code!
The text was updated successfully, but these errors were encountered: