Fix/naming and docs

.github/workflows/ci.yml

@@ -22,27 +22,27 @@ jobs:
     steps:
       - name: Checkout code
         uses: actions/checkout@v3
-      
+
       - name: Set up Python and Poetry
         uses: ./.github/actions/python-poetry
         with:
           groups: main,dev,test
-      
+
       - name: Run pre-commit
         uses: pre-commit/action@v3.0.1
-      
+
       - name: Run tests with coverage
         run: |
           poetry run python -m pytest --junitxml=pytest.xml --cov-report=term-missing:skip-covered --cov=. tests/ | tee pytest-coverage.txt
-      
+
       - name: Generate coverage report & comment on PR
         id: coverageComment
         uses: MishaKav/pytest-coverage-comment@main
         with:
           pytest-coverage-path: ./pytest-coverage.txt
           junitxml-path: ./pytest.xml
           hide-report: true
-      
+
       - name: Update README with coverage badge
         run: |
           cat > update_readme.py << 'EOF'
@@ -54,15 +54,15 @@ jobs:
           with open('README.md', 'r') as f:
               content = f.read()
           new_content = re.sub(
-              r'!\[Coverage\]\(https://img\.shields\.io/badge/Coverage-[^)]+\)', 
-              badge_md, 
+              r'!\[Coverage\]\(https://img\.shields\.io/badge/Coverage-[^)]+\)',
+              badge_md,
               content
           )
           with open('README.md', 'w') as f:
               f.write(new_content)
           EOF
           python update_readme.py
-        
+
       - name: Commit coverage updates to README
         uses: stefanzweifel/git-auto-commit-action@v4
         with:
@@ -77,15 +77,15 @@ jobs:
     steps:
       - name: Checkout code
         uses: actions/checkout@v3
-      
+
       - name: Set up Python and Poetry
         uses: ./.github/actions/python-poetry
-      
+
       - name: Build wheel
         run: poetry build --format wheel
-      
+
       - name: Upload wheel
         uses: actions/upload-artifact@v4
         with:
           name: ${{ github.event.repository.name }}
-          path: dist/
+          path: dist/

.pre-commit-config.yaml

@@ -1,5 +1,5 @@
 fail_fast: true
-exclude: '^(?!(promptolution/|tests/)).*$|^promptolution/templates.py'
+exclude: '^.*\.md$'
 repos:
   - repo: https://github.com/gitleaks/gitleaks
     rev: v8.18.2
@@ -46,4 +46,4 @@ repos:
     hooks:
       - id: conventional-pre-commit
         stages: [commit-msg]
-        args: []
+        args: []

README.md

@@ -1,20 +1,16 @@
 ![promptolution](https://github.com/user-attachments/assets/84c050bd-61a1-4f2e-bc4e-874d9b4a69af)
 
-
-![Coverage](https://img.shields.io/badge/Coverage-90%25-brightgreen)
+![Coverage](https://img.shields.io/badge/Coverage-87%25-green)
 [![CI](https://github.com/finitearth/promptolution/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/finitearth/promptolution/actions/workflows/ci.yml)
 [![Docs](https://github.com/finitearth/promptolution/actions/workflows/docs.yml/badge.svg?branch=main)](https://github.com/finitearth/promptolution/actions/workflows/docs.yml)
 ![Code Style](https://img.shields.io/badge/Code%20Style-black-black)
 ![Python Versions](https://img.shields.io/badge/Python%20Versions-≥3.9-blue)
 
-# Promptolution
 
-Promptolution is a library that provides a modular and extensible framework for implementing prompt tuning experiments. It offers a user-friendly interface to assemble the core components for various prompt optimization tasks.
 
-This project was developed by [Timo Heiß](https://www.linkedin.com/in/timo-heiss/), [Moritz Schlager](https://www.linkedin.com/in/moritz-schlager/) and [Tom Zehle](https://www.linkedin.com/in/tom-zehle/).
+Promptolution is a library that provides a modular and extensible framework for implementing prompt tuning for single tasks and larger experiments. It offers a user-friendly interface to assemble the core components for various prompt optimization tasks.
 
-## Getting Started
-Take a look at our getting started notebook: [getting_started.ipynb](https://github.com/finitearth/promptolution/blob/main/notebooks/getting_started.ipynb)
+This project was developed by [Timo Heiß](https://www.linkedin.com/in/timo-heiss/), [Moritz Schlager](https://www.linkedin.com/in/moritz-schlager/) and [Tom Zehle](https://www.linkedin.com/in/tom-zehle/) as part of a study program at LMU Munich.
 
 ## Installation
 
@@ -30,7 +26,6 @@ If you want to run your prompt optimization locally, either via transformers or
 pip install promptolution[vllm,transformers]
 ```
 
-
 Alternatively, clone the repository, run
 
 ```
@@ -39,34 +34,28 @@ poetry install
 
 to install the necessary dependencies. You might need to install [pipx](https://pipx.pypa.io/stable/installation/) and [poetry](https://python-poetry.org/docs/) first.
 
-## Documentation
-
-A comprehensive documentation with API reference is availabe at https://finitearth.github.io/promptolution/.
-
 ## Usage
 
-Create API Keys for the models you want to use:
-- OpenAI: store token in openaitoken.txt
-- Anthropic: store token in anthropictoken.txt
-- DeepInfra (for Llama): store token in deepinfratoken.txt
+To get started right away, take a look at our [getting started notebook](https://github.com/finitearth/promptolution/blob/main/notebooks/getting_started.ipynb).
+For more details, a comprehensive **documentation** with API reference is availabe at https://finitearth.github.io/promptolution/.
 
-## Optimization Algorithms to choose from
-| **Name** | **Paper** | **init prompts** | **Exploration** | **Costs** | **Convergence Speed** | **Parallelizable** | **Utilizes Fewshot Examples** |
-|:--------:|:-------:|:--------------:|:---------------:|:---------:|:---------------------:|:------------------:|:---------------------:|
-| `CAPO` | [Zehle et al.](https://arxiv.org/abs/2504.16005)| *required* | 👍 | 💲 | ⚡⚡ | ✅ | ✅ |
-| `EvoPromptDE` | [Guo et al.](https://arxiv.org/abs/2309.08532) | *required* | 👍 | 💲💲 | ⚡⚡ | ✅ | ❌ |
-| `EvoPromptGA` | [Guo et al.](https://arxiv.org/abs/2309.08532) | *required* | 👍 | 💲💲 | ⚡⚡ | ✅ | ❌ |
-| `OPRO` | [Yang et al.](https://arxiv.org/abs/2309.03409)| *optional* | 👎 | 💲💲 | ⚡ | ❌ | ❌ |
+### Featured Optimizers
 
-## Core Components
+|   **Name**    |                    **Paper**                     | **init prompts** | **Exploration** | **Costs**  | **Parallelizable** | **Utilizes Fewshot Examples** |
+| :-----------: | :----------------------------------------------: | :--------------: | :-------------: | :-------: | :-------------------: | :---------------------------: |
+|    `CAPO`     | [Zehle et al.](https://arxiv.org/abs/2504.16005) |    _required_    |       👍        |    💲     |        ✅         |              ✅               |
+| `EvoPromptDE` |  [Guo et al.](https://arxiv.org/abs/2309.08532)  |    _required_    |       👍        |   💲💲    |               ✅         |              ❌               |
+| `EvoPromptGA` |  [Guo et al.](https://arxiv.org/abs/2309.08532)  |    _required_    |       👍        |   💲💲    |               ✅         |              ❌               |
+|    `OPRO`     | [Yang et al.](https://arxiv.org/abs/2309.03409)  |    _optional_    |       👎        |   💲💲    |                  ❌         |              ❌               |
 
-- Task: Encapsulates initial prompts, dataset features, targets, and evaluation methods.
-- Predictor: Implements the prediction logic, interfacing between the Task and LLM components.
-- LLM: Unifies the process of obtaining responses from language models, whether locally hosted or accessed via API.
-- Optimizer: Implements prompt optimization algorithms, utilizing the other components during the optimization process.
-- Exemplar Selectors: Implements algorithms for the search of few shot examples that are added to the prompt.
+### Core Components
 
-## Key Features
+- `Task`: Encapsulates initial prompts, dataset features, targets, and evaluation methods.
+- `Predictor`: Implements the prediction logic, interfacing between the `Task` and `LLM` components.
+- `LLM`: Unifies the process of obtaining responses from language models, whether locally hosted or accessed via API.
+- `Optimizer`: Implements prompt optimization algorithms, utilizing the other components during the optimization process.
+
+### Key Features
 
 - Modular and object-oriented design
 - Extensible architecture
@@ -75,6 +64,48 @@ Create API Keys for the models you want to use:
 - Integration with langchain for standardized LLM API calls
 - Detailed logging and callback system for optimization analysis
 
+## Changelog
+
+Release notes for each version of the library can be found [here](https://finitearth.github.io/promptolution/release-notes/)
+
+## Contributing
+
+The first step to contributing is to open an issue describing the bug, feature, or enhancements. Ensure the issue is clearly described, assigned, and properly tagged. All work should be linked to an open issue.
+
+### Code Style and Linting
+
+We use Black for code formatting, Flake8 for linting, pydocstyle for docstring conventions (Google format), and isort to sort imports. All these checks are enforced via pre-commit hooks, which automatically run on every commit. Install the pre-commit hooks to ensure that all checks run automatically:
+
+```
+pre-commit install
+```
 
+To run all checks manually:
 
-This project was developed for the "AutoML in the age of large pre-trained models" seminar at LMU Munich.
+```
+pre-commit run --all-files
+```
+
+### Branch Protection and Merging Guidelines
+
+- The main branch is protected. No direct commits are allowed for non-administrators.
+- Rebase your branch on main before opening a pull request.
+- All contributions must be made on dedicated branches linked to specific issues.
+- Name the branch according to {prefix}/{description} with one of the prefixes fix, feature, chore, or refactor.
+- A pull request must have at least one approval from a code owner before it can be merged into main.
+- CI checks must pass before a pull request can be merged.
+- New releases will only be created by code owners.
+
+### Testing
+
+We use pytest to run tests, and coverage to track code coverage. Tests automatically run on pull requests and pushes to the main branch, but please ensure they also pass locally before pushing!
+To run the tests with coverage locally, use the following commands or your IDE's test runner:
+
+```
+poetry run python -m coverage run -m pytest
+```
+
+To see the coverage report run:
+```
+poetry run python -m coverage report
+```

docs/api/exemplar_selectors.md

@@ -0,0 +1,5 @@
+# Exemplar Selectors
+
+::: promptolution.exemplar_selectors
+    options:
+      show_submodules: true

docs/api/callbacks.md → docs/api/helpers.md

@@ -1,5 +1,5 @@
-# Callbacks
+# Helpers
 
-::: promptolution.callbacks
+::: promptolution.helpers
     options:
       show_submodules: true

docs/api/utils.md

@@ -0,0 +1,5 @@
+# Utils
+
+::: promptolution.utils
+    options:
+      show_root_heading: true