Uncertainty in Large Language Models

This repository contains the code and experiments related to the research article "Uncertainty Quantification in Large Language Models Through Convex Hull Analysis" published in Discover Artificial Intelligence (2024). The study focuses on quantifying uncertainty in large language models (LLMs) and provides insights into the methodology, experimental setup, and results.

📄 Published Work

The published article can be accessed at Springer. If you find this repository or the paper useful for your research, please consider citing the work (citation details are provided below).

---

📂 Repository Structure

• llm-uncertainty-experiments.ipynb: Jupyter Notebook containing the code for experiments, data preprocessing, and analysis of uncertainty in LLMs.

---

🧪 Experiments

The notebook includes:

• Dataset Preparation: Methods for preparing input data for LLMs.
• Uncertainty Quantification: Approaches for measuring uncertainty in LLM predictions.
• Visualization: Plots and figures for interpreting results.
• Reproducibility: Steps to replicate the experiments.

---

🚀 Getting Started

Prerequisites

1. Install Python 3.8+ and Jupyter Notebook.
2. Install the required dependencies:

   pip install -r requirements.txt

Running the Notebook

1. Clone the repository:

   git clone https://github.com/ocatak/uncertainty_in_llm.git
   cd uncertainty_in_llm

2. Open the Jupyter Notebook:

   jupyter notebook llm-uncertainty-experiments.ipynb

3. Follow the steps in the notebook to reproduce the experiments.

---

📌 Citation

If you use this repository or the associated publication in your work, please cite:

@article{catak2024uncertainty,
  author    = {Catak, Ferhat Ozgur and Kuzlu, Murat},
  title     = {Uncertainty Quantification in Large Language Models Through Convex Hull Analysis},
  journal   = {Discover Artificial Intelligence},
  volume    = {4},
  pages     = {90},
  year      = {2024},
  doi       = {10.1007/s44163-024-00200-w},
  url       = {https://doi.org/10.1007/s44163-024-00200-w}
}

---

🤝 Contributing

Contributions are welcome! Please feel free to submit a pull request or open an issue for any suggestions or improvements.

---

📝 License

This repository is licensed under the MIT License.

---

✉️ Contact

For any questions or discussions related to the project, feel free to contact Ferhat Ozgur Catak.