Skip to content

Latest commit

 

History

History
32 lines (25 loc) · 14.8 KB

filtered_read_and_summarised.md

File metadata and controls

32 lines (25 loc) · 14.8 KB
Raw

Read and Summarised Papers

Read

Paper Name Status Topic Category Year Conference Author Summary Link
0 ZF Net (Visualizing and Understanding Convolutional Networks) Read CNNs, CV , Image Visualization 2014 ECCV Matthew D. Zeiler, Rob Fergus Visualize CNN Filters / Kernels using De-Convolutions on CNN filter activations. link
1 Inception-v1 (Going Deeper With Convolutions) Read CNNs, CV , Image Architecture 2015 CVPR Christian Szegedy, Wei Liu Propose the use of 1x1 conv operations to reduce the number of parameters in a deep and wide CNN link
2 ResNet (Deep Residual Learning for Image Recognition) Read CNNs, CV , Image Architecture 2016 CVPR Kaiming He, Xiangyu Zhang Introduces Residual or Skip Connections to allow increase in the depth of a DNN link
3 Evaluation of neural network architectures for embedded systems Read CNNs, CV , Image Comparison 2017 IEEE ISCAS Adam Paszke, Alfredo Canziani, Eugenio Culurciello Compare CNN classification architectures on accuracy, memory footprint, parameters, operations count, inference time and power consumption. link
4 SqueezeNet Read CNNs, CV , Image Architecture, Optimization-No. of params 2016 arXiv Forrest N. Iandola, Song Han Explores model compression by using 1x1 convolutions called fire modules. link
5 Attention is All you Need Read Attention, Text , Transformers Architecture 2017 NIPS Ashish Vaswani, Illia Polosukhin, Noam Shazeer, Łukasz Kaiser Talks about Transformer architecture which brings SOTA performance for different tasks in NLP link
6 BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Read Attention, Text , Transformers Embeddings 2018 NAACL Jacob Devlin, Kenton Lee, Kristina Toutanova, Ming-Wei Chang BERT is an extension to Transformer based architecture which introduces a masked word pretraining and next sentence prediction task to pretrain the model for a wide variety of tasks. link
7 Reformer: The Efficient Transformer Read Attention, Text , Transformers Architecture, Optimization-Memory, Optimization-No. of params 2020 arXiv Anselm Levskaya, Lukasz Kaiser, Nikita Kitaev Overcome time and memory complexity of Transformers by bucketing Query, Keys and using Reversible residual connections. link
8 Bag of Tricks for Image Classification with Convolutional Neural Networks Read CV , Image Optimizations, Tips & Tricks 2018 arXiv Tong He, Zhi Zhang Shows a dozen tricks (mixup, label smoothing, etc.) to improve CNN accuracy and training time. link
9 The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Read NN Initialization, NNs Optimization-No. of params, Tips & Tricks 2019 ICLR Jonathan Frankle, Michael Carbin Lottery ticket hypothesis: dense, randomly-initialized, feed-forward networks contain subnetworks (winning tickets) that—when trained in isolation— reach test accuracy comparable to the original network in a similar number of iterations. link
10 Pix2Pix: Image-to-Image Translation with Conditional Adversarial Nets Read GANs, Image 2017 CVPR Alexei A. Efros, Jun-Yan Zhu, Phillip Isola, Tinghui Zhou Image to image translation using Conditional GANs and dataset of image pairs from one domain to another. link
11 Language-Agnostic BERT Sentence Embedding Read Attention, Siamese Network, Text , Transformers Embeddings 2020 arXiv Fangxiaoyu Feng, Yinfei Yang A BERT model with multilingual sentence embeddings learned over 112 languages and Zero-shot learning over unseen languages. link
12 T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Read Attention, Text , Transformers 2020 JMLR Colin Raffel, Noam Shazeer, Peter J. Liu, Wei Liu, Yanqi Zhou Presents a Text-to-Text transformer model with multi-task learning capabilities, simultaneously solving problems such as machine translation, document summarization, question answering, and classification tasks. link
13 Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask Read NN Initialization, NNs Comparison, Optimization-No. of params, Tips & Tricks 2019 NeurIPS Hattie Zhou, Janice Lan, Jason Yosinski, Rosanne Liu Follow up on Lottery Ticket Hypothesis exploring the effects of different Masking criteria as well as Mask-1 and Mask-0 actions. link
14 SpanBERT: Improving Pre-training by Representing and Predicting Spans Read Question-Answering, Text , Transformers Pre-Training 2020 TACL Danqi Chen, Mandar Joshi A different pre-training strategy for BERT model to improve performance for Question Answering task. link
15 Learning to Extract Attribute Value from Product via Question Answering: A Multi-task Approach Read Question-Answering, Text , Transformers Zero-shot-learning 2020 KDD Li Yang, Qifan Wang Question Answering BERT model used to extract attributes from products. Introduce further No Answer loss and distillation to promote zero shot learning. link
16 VL-T5: Unifying Vision-and-Language Tasks via Text Generation Read CNNs, CV , Generative, Image , Large-Language-Models, Question-Answering, Text , Transformers Architecture, Embeddings, Multimodal, Pre-Training 2021 arXiv Hao Tan, Jaemin Cho, Jie Le, Mohit Bansal Unifying two modalities (image and text) together in a single transformer model to solve multiple tasks in a single architecture using text prefixes similar to T5. link