LLM Assignments - Fall 2023

This Repository for assignments completed during the Large Language Models (LLM) course at Sharif University of Technology, Fall 2023. The assignments cover various aspects of LLMs, including efficient fine-tuning, data preprocessing, multi-modal models, and assessing capabilities of LLMs in tasks like reasoning and knowledge retrieval.

Course Description

The course covered theoretical and practical aspects of large language models within the field of natural language processing. Topics included the development and application of LLMs, model architecture, training techniques, and fine-tuning processes, with an emphasis on ethical issues such as bias and fairness.

Instructors

• Dr. Mahdieh Soleymani Baghshah
• Dr. Ehsaneddin Asgari
• Dr. Mohammad Hossein Rohban

Assignments Overview

Assignments in this repository include:

1. PEFT & Data Preprocessing: Techniques for improving LLM performance and preprocessing data.
2. LLMs Reasoning Capabilities: Analyzing LLMs' ability to reason.
3. Multi-Modal Language Models & RAG: Study on processing and generating multi-modal data, and Retrieval-Augmented Generation.
4. Loss Visualization & GPT-2's Loss Landscape: Techniques for visualizing loss and exploring GPT-2's loss landscape.
5. Sycophancy in LLMs: Examination of sycophancy phenomena in LLM interactions.
6. Sampling Techniques in NLG: Analysis of sampling methods in natural language generation.