Skip to content

Development

Jump to bottom
Brian Rodriguez edited this page Dec 9, 2025 · 2 revisions

Development Guide

Setting Up Development Environment

Clone and Install

git clone https://github.com/CervezaStallone/XY-MD02_WebApp.git
cd XY-MD02_WebApp
python -m venv .venv
source .venv/bin/activate  # Linux/macOS
.venv\Scripts\activate  # Windows
pip install -r requirements.txt

Enable Debug Mode

Edit .env:

APP_DEBUG=False          # Keep False - uses Waitress, not Flask dev server
DEBUG_LOGGING=True       # Enable detailed console output
APP_PORT=8050

Debug logging enables:

  • Database query details (table counts, timestamps)
  • Downsampling statistics (before/after point counts)
  • Comfort score calculations (first 5 points per update)
  • Modbus communication details
  • Error tracebacks

Note: Application uses Waitress production server, so APP_DEBUG doesn't enable hot reloading. For development iteration, restart the application after code changes.

Project Structure

XY-MD02_WebApp/
├── app.py                 # Main entry point - Waitress WSGI server
├── callbacks.py           # Dash callbacks (8 callbacks for interactivity)
├── layout.py              # UI layout (5-graph dashboard)
├── database.py            # SQLite operations (table-per-day partitioning)
├── modbus_reader.py       # Modbus RTU communication thread
├── psychrometric.py       # Psychrometric chart generation
├── translations.py        # Multi-language support (5 languages)
├── requirements.txt       # Python dependencies
├── pyproject.toml         # Project metadata (version, etc.)
├── startApp.ps1           # PowerShell startup script (Windows)
├── .env.example           # Configuration template
├── .env                   # Your configuration (not in git)
├── .gitignore            # Git ignore rules
├── .github/
│   ├── workflows/         # CI/CD automation
│   ├── ISSUE_TEMPLATE/    # Issue templates
│   └── dependabot.yml     # Dependency updates
├── .wiki/                 # Wiki documentation (markdown)
├── img/                   # Static images
├── src/                   # Data directory (database storage)
│   └── modbus_sensor_data.db  # SQLite database
└── tests/                 # Test files (if any)

Code Architecture

Module Dependencies

app.py
  ├── layout.py
  │   └── translations.py
  ├── callbacks.py
  │   ├── database.py
  │   ├── psychrometric.py
  │   └── translations.py
  └── modbus_reader.py
      └── database.py

Data Flow

  1. modbus_reader.py:

    • Reads sensor every 1 second via Modbus RTU
    • Validates readings (-50 to 100°C, 0-100% RH)
    • Calculates dewpoint, absolute humidity
    • Buffers 30 measurements
    • Batch writes to database every 30 seconds
    • Runs as background daemon thread

  2. database.py:

    • SQLite operations with WAL mode
    • Table-per-day partitioning (measurements_YYYYMMDD)
    • Automatic cleanup based on DATA_RETENTION_DAYS
    • Optimized union queries across tables
    • Indexed timestamp filtering

  3. callbacks.py:

    • 8 Dash callbacks for UI updates
    • Queries database via union queries
    • Applies adaptive downsampling for performance
    • Calculates comfort scores (Humidex)
    • Updates 5 synchronized graphs
    • Handles historical data replay

  4. layout.py:

    • Renders UI components (5 cards, 5 graphs)
    • Psychrometric chart container
    • Historical replay modal and slider
    • Multi-language dropdown
    • Responsive design

  5. psychrometric.py:

    • Generates Mollier (psychrometric) diagrams
    • Plots current or historical conditions
    • Shows RH curves, comfort zone, saturation line

Coding Standards

Style Guide

  • Follow PEP 8
  • Line length: 120 characters
  • Use type hints where practical
  • Docstrings for all functions

Code Formatting

Run Black formatter:

black --line-length 120 *.py

Linting

Run flake8:

flake8 . --max-line-length=120 --exclude=.venv,__pycache__

Pre-commit Checks

Before committing:

# Format code
black --line-length 120 *.py

# Check syntax
python -m py_compile *.py

# Run linter
flake8 . --max-line-length=120

# Verify imports
python -c "import database; import callbacks; import layout"

Testing

Manual Testing

# Run application
python app.py

# Open browser
# Test all features manually

Syntax Validation

python -m py_compile app.py
python -m py_compile database.py
python -m py_compile modbus_reader.py
# ... etc

Translation Testing

Verify all languages display correctly:

  1. Switch to each language
  2. Check all UI elements
  3. Verify graph labels and tooltips
  4. Test with actual data

Adding Features

New Translation Key

  1. Add to all languages in translations.py:
"new_key": {
    "nl": "Nederlandse tekst",
    "en": "English text",
    "de": "Deutscher Text",
    "fr": "Texte français",
    "es": "Texto español"
}
  1. Use in layout or callbacks:
t("new_key", lang)

New Callback

  1. Add to callbacks.py
  2. Use @app.callback decorator
  3. Follow existing patterns
  4. Add error handling

Example:

@app.callback(
    Output('output-id', 'property'),
    Input('input-id', 'property')
)
def callback_function(input_value):
    try:
        # Process input
        result = process(input_value)
        return result
    except Exception as e:
        print(f"Error: {e}")
        return default_value

New Database Table

  1. Add creation in database.py
  2. Update ensure_table_exists()
  3. Add cleanup logic if needed
  4. Test with various data volumes

Continuous Integration

GitHub Actions automatically:

  • Tests Python 3.10, 3.11, 3.12, 3.13
  • Checks code quality (Black, flake8)
  • Validates translations
  • Runs on every push and PR

Key Features Implementation Guide

Adaptive Downsampling

Location: callbacks.py - update_graph() callback

How it works:

if time_range_minutes > 60 or original_points > 5000:
    df = df.set_index('timestamp')
    
    # Select appropriate interval based on time range
    if time_range_minutes <= 360:  # 1-6 hours
        df = df.resample('1min').mean()
    elif time_range_minutes <= 1440:  # 6-24 hours
        df = df.resample('5min').mean()
    # ... more intervals ...
    
    df = df.dropna().reset_index()

Key points:

  • Uses pandas resample() with mean aggregation
  • Automatically triggers for long ranges
  • Preserves timestamp index during resampling
  • Logs reduction statistics when DEBUG_LOGGING=True
  • Maintains visual quality while dramatically improving performance

Comfort Score Calculation

Location: callbacks.py - calculate_comfort_score() function

Formula:

  1. Calculate dewpoint: Td = T - ((100 - RH) / 5.0)
  2. Calculate vapor pressure: e = 6.11 × exp(5417.7530 × (1/273.16 - 1/(273.15+Td)))
  3. Calculate Humidex: H = T + 0.5555 × (e - 10)
  4. Map Humidex to 0-6 score based on ranges

Used in:

  • Real-time comfort card display (via get_comfort_level())
  • Historical comfort score graph (5th graph)
  • Applied to every datapoint after downsampling

Historical Data Replay

Location: callbacks.py - Multiple callbacks

Implementation:

  1. User clicks "Choose range" button → opens modal
  2. User selects preset (5min-1month) or custom date/time range
  3. handle_range_selection() captures choice
  4. load_historical_data() queries database and resamples to 1-minute intervals
  5. Creates slider with appropriate time marks
  6. update_historical_view() responds to slider movement (including drag)
  7. Psychrometric chart updates in real-time showing selected moment

Features:

  • Supports both preset and custom ranges
  • Intelligent slider marks based on time span
  • Live updates during drag
  • Uses create_psychrometric_chart_historical() for styling

Table-Per-Day Partitioning

Location: database.py

Benefits:

  • Faster queries (only relevant tables scanned)
  • Efficient cleanup (drop entire tables vs. delete rows)
  • Better organization
  • Scales to years of data

Implementation:

def get_table_name(date=None):
    if date is None:
        date = datetime.now()
    return f"measurements_{date.strftime('%Y%m%d')}"

def build_union_query(cursor, columns, start_timestamp=None, end_timestamp=None):
    # Determine relevant tables for time range
    tables = get_tables_for_timerange(cursor, start_timestamp, end_timestamp)
    # Build UNION ALL query across tables
    queries = [f'SELECT {columns} FROM {table} WHERE ...' for table in tables]
    return ' UNION ALL '.join(queries)

Multi-Language Support

Location: translations.py, callbacks.py, layout.py

Adding new translation key:

  1. Add to all 5 languages in translations.py:
'new_key': {
    'nl': 'Nederlandse tekst',
    'en': 'English text',
    'de': 'Deutscher Text',
    'fr': 'Texte français',
    'es': 'Texto español'
}
  1. Use in callbacks:
t = TRANSLATIONS[lang]
text = t['new_key']
  1. Use in layout (for static elements):
t = TRANSLATIONS[DEFAULT_LANGUAGE]
html.Div(t['new_key'])

Note: Layout uses DEFAULT_LANGUAGE for initial render, callbacks update dynamically.

Waitress Production Server

Location: app.py

Configuration:

serve(
    server,                    # Flask/Dash server object
    host='0.0.0.0',           # Listen on all interfaces
    port=8050,                # Port number
    threads=8,                # Worker threads for concurrent requests
    channel_timeout=60,       # Idle connection timeout
    cleanup_interval=10,      # Connection cleanup frequency
    asyncore_use_poll=True    # Better Windows performance
)

Why Waitress:

  • Production-ready (not Flask dev server)
  • Cross-platform (Windows/Linux/macOS)
  • Multi-threaded for concurrent users
  • Better error handling and logging
  • No dependencies on C compilers (pure Python)

View workflow runs: https://github.com/CervezaStallone/XY-MD02_WebApp/actions

Dependency Management

Adding Dependency

  1. Install in virtual environment:
pip install new-package
  1. Update requirements:
pip freeze > requirements.txt
  1. Update pyproject.toml:
dependencies = [
    "existing-package==1.0.0",
    "new-package==2.0.0",
]
  1. Test installation:
pip install -r requirements.txt

Updating Dependencies

Dependabot automatically creates PRs for updates.

Manual update:

pip install --upgrade package-name
pip freeze > requirements.txt

Commit Conventions

Use Conventional Commits format:

type(scope): description

[optional body]

[optional footer]

Types:

  • feat: New feature
  • fix: Bug fix
  • docs: Documentation
  • style: Formatting
  • refactor: Code restructuring
  • test: Adding tests
  • chore: Maintenance

Examples:

feat(ui): Add temperature unit selection
fix(modbus): Handle timeout errors gracefully
docs(readme): Update installation instructions
chore(deps): Bump pandas from 2.2.3 to 2.3.0

Release Process

  1. Update version in pyproject.toml
  2. Commit: chore: Bump version to X.Y.Z
  3. Create tag: git tag -a vX.Y.Z -m "Release vX.Y.Z"
  4. Push: git push origin main --tags
  5. Create GitHub release with changelog

Contributing

See CONTRIBUTING.md for:

  • Code of Conduct
  • Pull request process
  • Issue reporting guidelines

Clone this wiki locally