- I am training smallish (<4B params) language models on heavily curated datasets to investigate emergence of different skillsets like coherence, logical reasoning, coding ability etc.
- I am building a ML powered product analytics tool - Crunch
- Language and speech modelling: At ObserveAI I was training language models (variants of BERT mostly) and experimenting with different pretraining algorithms for speech data to extract the most usable embeddings for downstream models.
- Research: Language Agnostic Speech Embeddings for Emotion Classification -PDF ICML
- Keras documentation examples:
- Generative models: Miniature GPT
- Reinforcement Learning: Actor Critic Method
- NLP: Text Extraction with BERT
- ASR: Sequence to sequence speech recognition with Transformers
- Random minimal implementations: Prototypical Networks, UNet, Masked language model training, Causal dilated convolutions with attention, Training on TPUs (Old)