Project Proposal For Contribution to GSoC'19

[shreyas-kowshik]

Hi I am a GSoC'19 aspirant looking to contribute to the model-zoo of Flux. I have a few ideas in mind for my work and wanted to check the feasibility of it. I was planning to implement Generative models as in CycleGAN, pix2pix, neural image captioning, and an object detection pipeline as in YOLO-v3. I also see that the object detection modules as in Faster-RCNN and mobilenet-SSD have not been implemented and I can also work with them. I have prior experience with generative models and object detection using Deep Learning. @MikeInnes @ViralBShah

[ViralBShah]

Welcome! In general, it is good to have contributions made to model-zoo or Flux or some related repos. Also cc @staticfloat

[shreyas-kowshik]

@ViralBShah Yeah I would be contributing to model-zoo. However are all/any of the models above relevant? Or do have any of these models been implemented in some other repository/ are not of special interest at the moment?

[ViralBShah]

Let's wait for @MikeInnes to chime in. Also @avik-pal could perhaps chime in.

[avik-pal]

CycleGAN, Pix2Pix, and Neural Image Captioning are interesting additions. As for YOLO, someone recently posted about their implementation on slack. Also, someone is working on Mask RCNN, which is essentially an extension of Faster RCNN, for segmentation tasks. So it might not be ideal to spend time on Faster RCNN (we can simply build it using the layers of MaskRCNN).

[ViralBShah]

@dhairyagandhi96 seems to be looking into Mask RCNNs as well.

[DhairyaLGandhi]

Mask RCNN is simply building on top of Faster RCNN, but Mask RCNN can have different versions within itself. YOLO.jl is already a nice implementation of the same.

Welcome to Flux and Julia!

[DhairyaLGandhi]

There are some interesting multi-modal learning applications in the GAN-verse too, if you're interested

[shreyas-kowshik]

@dhairyagandhi96 Can you help me get started with an issue? I was thinking of adding tutorials for the already done work(As in the GAN models built last year) in a detailed jupyter notebook in a format similar to that in tensorflow
by addressing #69.

Would that be an important issue to pick at the moment or should I first focus on working on the internals as in the issues in Flux/NNlib?

[DhairyaLGandhi]

Do feel free to take a look at issues as you wish.
#69 is in fact something we've been working on. We have notebooks built around for the current models, but a lot of them need explanatory material to go along with them.

I would be interested to see some of those fleshed out as well. Maybe in conjunction with a couple gan implementations?

[shreyas-kowshik]

I'll get to working on the GAN implementation. As for the existing notebooks with explanatory material do you reference this and this repositories?

[DhairyaLGandhi]

Before moving full steam, it'd be useful and important to get Mike's and Elliot's thoughts on this.

[shreyas-kowshik]

@staticfloat I was thinking of adding tutorials for the already done work(As in the GAN models built last year) in a detailed jupyter notebook in a format similar to that in tensorflow and also to add explanatory material to these and these notebooks containing work on tutorials that has already been done. What's your opinion on this?

[sidhantnagpal]

I would like to contribute to CV and RL models as well. It is so nice to train models with Julia/Flux on Google Colab. I have implemented GANs with dense layers and DCGANs in Flux. I was waiting on ConvTranspose which just got merged in FluxML/Flux.jl#311 (thanks to Tejank).

[nikunj-taneja]

Hi @dhairyagandhi96 @staticfloat ! I am interested in the Port ML Tutorials project as well. I was thinking of porting the tutorials from Udacity and Stanford 231n. I could start with simple neural networks and then move on to CNNs, RNNs/LSTMs and then maybe GANs. The notebooks will be theoretical explanation + Flux code, assuming basic knowledge of Julia and Flux. Existing notebooks could be incorporated in the same. What do you think about this?

[shreyas-kowshik]

I have been contributing to model-zoo for sometime now...These are the PR's I've made so far :
#111
#98
#82
I was thinking of starting to work upon the proposal along with solving issues parallelly. For that I wanted to know what all models are effectively required to be implemented in terms of priority for now.

I have the following in mind : pix2pix, cycleGAN, neural-image-captioning, generative-adversarial-text-to-image-synthesis, Proximal Policy Optimization For Policy Gradients, Mobilenet-Single-Shot-Detector, Deep Recurrent Q Networks...Some of these models like Mobilenet-SSD are part of tensorflow and have pretrained weights to be used on the fly. So I think adding them should be highly beneficial.

The image captioning model would utilise the Resnet layers that have been built and I see that even this is a basic module present in many frameworks by default.These ideas were somewhat inspired from : https://github.com/tensorflow/models/tree/master/research

What are your views on priority/variety or on the overall proposition...?
@staticfloat @MikeInnes @ViralBShah @dhairyagandhi96 @avik-pal

[MikeInnes]

It would be nice to get one of these PRs polished and merged. I suggest focusing on one only and addressing feedback so that we can get it in. Do ask if you aren't sure what needs to be done.

[shreyas-kowshik]

Thanks for the reply. I am working on the PRs and have made the changes requested so far. I'll prioritize them one by one to get them polished and merged.

[ToucheSir]

Cleaning up this old issue for the same reason as #114 (comment).