Config Composer

A Python library for hierarchical YAML configuration composition.

Overview

Config Composer allows you to create layered configuration systems where configuration files at different levels of a folder hierarchy are automatically merged together. This is useful for applications that need different configuration settings based on the location of files or folders within a project structure.

Features

• Hierarchical Configuration: Automatically merges config files from root to target location
• Deep Merging: Intelligently merges nested dictionaries
• Flexible Setup: Configurable default config location and folder config filenames
• Multiple Instances: Support for multiple composer instances with different settings
• Path Validation: Ensures target paths are within the specified root folder

Installation

pip install -e .

Quick Start

from config_composer import ConfigComposer

# Initialize the composer
composer = ConfigComposer(
    default_config_path="config/default.yaml",
    root_folder="project_root",
    folder_config_filename="folderconfig.yaml"
)

# Get composed config for a specific file
config = composer.get_config("project_root/subfolder/myfile.txt")

# Get config as YAML string
config_yaml = composer.get_config_yaml("project_root/subfolder/myfile.txt")

How It Works

Given a folder structure like:

A/
├── folderconfig.yaml
├── B/
│   └── file.txt
└── C/
    ├── folderconfig.yaml
    └── D/
        └── file.txt

When you query for A/C/D/file.txt, the composer will:

1. Start with the default configuration
2. Overlay any settings from A/folderconfig.yaml
3. Overlay any settings from A/C/folderconfig.yaml
4. Return the final merged configuration

Configuration Examples

Default Configuration

database:
  host: localhost
  port: 5432
  name: default_db
  username: default_user
  password: default_pass

logging:
  level: INFO
  format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
  file: app.log

features:
  feature_a: true
  feature_b: false
  feature_c: false

api:
  timeout: 30
  retries: 3
  base_url: "https://api.example.com"

Root Folder Configuration (project_root/folderconfig.yaml)

database:
  name: production_db
  username: prod_user

environment: production

logging:
  level: WARN

features:
  feature_a: true

api:
  base_url: "https://prod-api.example.com"
  timeout: 60

Subfolder Configuration (project_root/backend/folderconfig.yaml)

database:
  host: backend-db.example.com
  port: 3306

logging:
  level: DEBUG
  file: backend.log

features:
  feature_b: true

api:
  timeout: 45

cache:
  max_size: 5000

File-Specific Metadata (project_root/backend/api/app.py.meta)

database:
  connection_timeout: 60
  pool_size: 20

api:
  rate_limit: 1000
  cors_enabled: true
  cors_origins: ["https://example.com", "https://app.example.com"]

logging:
  level: "DEBUG"

features:
  feature_api_v2: true
  feature_metrics: true

Resulting Configuration

When querying for project_root/backend/api/app.py, the final merged configuration would include:

• All settings from the default config
• Overrides from project_root/folderconfig.yaml
• Overrides from project_root/backend/folderconfig.yaml
• File-specific overrides from project_root/backend/api/app.py.meta

Example

See example_usage.py for a complete working example that demonstrates the library's functionality with the sample project structure.

API Reference

ConfigComposer

__init__(default_config_path, root_folder, folder_config_filename="folderconfig.yaml")

Initialize a new ConfigComposer instance.

• default_config_path: Path to the default YAML configuration file
• root_folder: Root folder to search for folder configs
• folder_config_filename: Name of config files to look for in folders (default: "folderconfig.yaml")

get_config(target_path)

Get the composed configuration for a given file or folder path.

• target_path: Path to file or folder within root_folder
• Returns: Dictionary containing the merged configuration

get_config_yaml(target_path)

Get the composed configuration as a YAML string.

• target_path: Path to file or folder within root_folder
• Returns: String containing the merged configuration in YAML format

clear_cache()

Clear all caches. Useful for testing or when you know files have changed.

get_cache_stats()

Get statistics about the internal cache usage.

• Returns: Dictionary containing cache statistics with keys:
  • file_cache_size: Number of cached YAML files
  • config_cache_size: Number of cached composed configurations
  • hierarchy_cache_size: Number of cached folder hierarchies

Performance Features

The library includes several performance optimizations:

• File Caching: YAML files are cached with modification time tracking
• Configuration Caching: Final merged configurations are cached until source files change
• Hierarchy Caching: Folder hierarchy calculations are cached
• Automatic Cache Invalidation: Caches are automatically invalidated when source files are modified

Running the Example

To see the library in action with the provided sample data:

python example_usage.py

This will demonstrate configuration composition across the sample project structure and show how different files receive different merged configurations based on their location in the hierarchy.

License

MIT License