Add timestep support for CloudGAN #56
Conversation
|
Other time-based GANs include https://arxiv.org/pdf/1801.09710.pdf and https://papers.nips.cc/paper/2019/file/c9efe5f26cd17ba6216bbe2a7d26d490-Paper.pdf, of which there is this PyTorch TimeGAN implementation https://github.com/d9n13lt4n/timegan-pytorch and Tensorflow implementation of TempoGAN here https://github.com/thunil/tempoGAN -> The TempoGAN might not be as useful, but seems like it could have some decent ideas |
|
Relates to #4 |
|
Current work is making CloudGAN work for both predicting all timesteps at once, like Pix2Pix, or conditioning on a future timestep, like MetNet does. The slides originally talk about only prediciting for a single timestep in the future, 15minutes in their case, so conditioning on time makes the model work a bit more like in the slides. In that implementation that now exists, the generator is given the previous timesteps in (T, C, H, W) format, runs a recurrent model on it, and returns a single image in the future, and the discriminator takes a single timestep for each and compares them. This might have the downside of less temporal consistency that the discriminator looks at though, although means the discriminator doesn't have a huge 288 or so channel set of images to choose between. |
Otherwise have to add logic for that elsewhere
|
CloudGAN works, although runs out of memory quite quickly. Getting it more efficient will be in a different PR. |
The previous PR #50 was getting quite large, so this is just to finish some more of it, and flesh out a few of the GANS, especially adding support for predicting individual timesteps instead of just all the outputs being concatenated together as a huge multi-channel image.