|
| 1 | +# pycomposefile |
| 2 | + |
| 3 | +pycomposefile is a Python library for structured deserialization of Docker Compose files. It provides a structured way to parse, validate, and manipulate Docker Compose YAML configurations in Python applications. |
| 4 | + |
| 5 | +Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here. |
| 6 | + |
| 7 | +## Working Effectively |
| 8 | + |
| 9 | +### Bootstrap and Setup |
| 10 | +- **Required Python version**: 3.9 or higher (tested with 3.10-3.14) |
| 11 | +- **Network Dependencies**: Most pip commands require PyPI access and may timeout in sandboxed environments |
| 12 | +- **Initial setup** (if pip access works): `python -m pip install -U pip` then `python -m pip install --group dev` |
| 13 | +- **Dependencies**: pyyaml>=6.0, build>=0.7, ruff==0.12.5, timeout-decorator==0.5.0 |
| 14 | +- **If network issues**: Dependencies may already be installed; verify with `python -m ruff --version` |
| 15 | + |
| 16 | +### Development Workflow Commands |
| 17 | +- **Lint code**: `python -m ruff check` (takes <1 second) - **ALWAYS run before committing** |
| 18 | +- **Format code**: `python -m ruff format` (takes <1 second) - Auto-formats Python code |
| 19 | +- **Check formatting**: `python -m ruff format --check` (takes <1 second) - **WARNING: Currently shows 31 files would be reformatted - this is normal for this codebase** |
| 20 | +- **Run tests**: `python -m unittest discover -v -s . -p "test*.py"` (takes <1 second, 85 tests, 1 typically skipped) |
| 21 | +- **Build package**: `python -m build` - **NETWORK DEPENDENT: Will fail with PyPI timeout in sandboxed environments. Use manual testing instead.** |
| 22 | + |
| 23 | +### Key Project Structure |
| 24 | +``` |
| 25 | +/pycomposefile/ # Main library code |
| 26 | + __init__.py # Exports ComposeFile class |
| 27 | + compose_file.py # Core ComposeFile class |
| 28 | + compose_element/ # Compose element parsing utilities |
| 29 | + service/ # Service-specific parsing |
| 30 | + secrets/ # Secret management |
| 31 | +/tests/ # Test suite (unittest framework) |
| 32 | + compose_file/ # Tests for compose file parsing |
| 33 | + service/ # Tests for service parsing |
| 34 | + environment_variable_evaluation/ # Tests for env var handling |
| 35 | + compose_generator/ # Test data generators |
| 36 | +/sample/ # Sample files for testing |
| 37 | + test.env, common.env # Environment files for testing |
| 38 | + my_secret.txt # Sample secret file |
| 39 | + apps/web.env # Application-specific env vars |
| 40 | +``` |
| 41 | + |
| 42 | +## Validation |
| 43 | + |
| 44 | +### Manual Functionality Testing |
| 45 | +Always test the core library functionality after making changes: |
| 46 | + |
| 47 | +```python |
| 48 | +import sys |
| 49 | +sys.path.insert(0, '.') |
| 50 | +from pycomposefile import ComposeFile |
| 51 | +import yaml |
| 52 | + |
| 53 | +# Test basic parsing |
| 54 | +compose_yaml = ''' |
| 55 | +version: '3' |
| 56 | +services: |
| 57 | + web: |
| 58 | + image: nginx:latest |
| 59 | + ports: |
| 60 | + - '80:80' |
| 61 | + db: |
| 62 | + image: postgres:13 |
| 63 | + environment: |
| 64 | + POSTGRES_DB: myapp |
| 65 | +''' |
| 66 | +parsed = yaml.load(compose_yaml, Loader=yaml.Loader) |
| 67 | +compose_file = ComposeFile(parsed) |
| 68 | +print(f'Version: {compose_file.version}') |
| 69 | +print(f'Services: {list(compose_file.services.keys())}') |
| 70 | +``` |
| 71 | + |
| 72 | +### Test Environment Variable Processing |
| 73 | +```python |
| 74 | +from tests.compose_generator import ComposeGenerator |
| 75 | +# Test environment files |
| 76 | +compose = ComposeGenerator.get_compose_with_environment_file_list() |
| 77 | +# Test service dependencies |
| 78 | +compose = ComposeGenerator.get_compose_with_service_dependencies() |
| 79 | +# Test secrets |
| 80 | +compose = ComposeGenerator.get_compose_with_one_secret() |
| 81 | +``` |
| 82 | + |
| 83 | +### CI Validation Steps |
| 84 | +Always run these commands before committing (matches .github/workflows/build.yaml): |
| 85 | +1. `python -m ruff check` - Must pass with no errors |
| 86 | +2. `python -m unittest discover -v -s . -p "test*.py"` - All tests must pass (85 tests, 1 may be skipped) |
| 87 | + |
| 88 | +**Note on formatting**: `python -m ruff format --check` shows 31 files would be reformatted, which is the current state of the codebase. Focus on ensuring your changes don't introduce new linting errors. |
| 89 | + |
| 90 | +**Build command limitations**: The `python -m build` command requires network access to PyPI and will timeout in sandboxed environments. Use manual functionality testing instead. |
| 91 | + |
| 92 | +## Important Development Notes |
| 93 | + |
| 94 | +### Network Dependencies |
| 95 | +- **PyPI Access Required**: Package building and some pip commands require downloading from PyPI |
| 96 | +- **Timeout Issues**: In sandboxed environments, pip and build commands frequently fail with "Read timed out" errors |
| 97 | +- **Workarounds**: |
| 98 | + - Test core functionality directly with Python imports instead of relying on package building |
| 99 | + - Dependencies are often pre-installed in development environments |
| 100 | + - Focus on linting, testing, and manual functionality validation |
| 101 | + - Use `python -m ruff --version` to verify ruff is available |
| 102 | + |
| 103 | +### Code Quality Requirements |
| 104 | +- **Linting**: All code must pass `ruff check` without warnings |
| 105 | +- **Formatting**: Code must be formatted with `ruff format` |
| 106 | +- **Testing**: All new functionality must have corresponding unittest tests |
| 107 | +- **Coverage**: Tests are comprehensive with 85+ test cases covering all major functionality |
| 108 | + |
| 109 | +### Common File Locations |
| 110 | +- **Configuration**: `pyproject.toml` (build config, dependencies, ruff settings) |
| 111 | +- **Main API**: `pycomposefile/__init__.py` (exports ComposeFile) |
| 112 | +- **Core Logic**: `pycomposefile/compose_file.py` (main parsing logic) |
| 113 | +- **Test Data**: `tests/compose_generator/__init__.py` (ComposeGenerator with test YAML) |
| 114 | +- **Sample Files**: `sample/` directory (test.env, common.env, my_secret.txt) |
| 115 | + |
| 116 | +### Testing Framework Details |
| 117 | +- **Framework**: Python unittest (not pytest) |
| 118 | +- **Test Discovery**: Use `python -m unittest discover -v` |
| 119 | +- **Test Organization**: Tests are organized by functionality in separate directories |
| 120 | +- **Test Data**: ComposeGenerator class provides Docker Compose YAML snippets for testing |
| 121 | +- **Expected Results**: 85 tests run, typically 1 skipped, all others should pass |
| 122 | + |
| 123 | +### Environment & Dependencies |
| 124 | +- **Build Backend**: flit_core (defined in pyproject.toml) |
| 125 | +- **Main Dependency**: pyyaml for YAML parsing |
| 126 | +- **Dev Dependencies**: build, ruff, timeout-decorator |
| 127 | +- **Python Support**: 3.9, 3.10, 3.11, 3.12, 3.13, 3.14 (per CI matrix) |
| 128 | + |
| 129 | +### Development Container Support |
| 130 | +- **DevContainer**: `.devcontainer/devcontainer.json` with Python 3 setup |
| 131 | +- **Post-Create Script**: `.devcontainer/postCreate.sh` sets up virtual environment |
| 132 | +- **VSCode Config**: Configured for Python development with unittest integration |
| 133 | + |
| 134 | +Always validate your changes by running the full development workflow: install dependencies, lint, format-check, and run tests. |
0 commit comments