Development Guide

This guide explains how to extend Epilykos with new visual blocks, data sources, and metrics. Intended for developers comfortable with JavaScript, Node.js, SQLite, and basic CSS/HTML.

---

Prerequisites

• Node.js 18+
• Epilykos installed (Docker or local)
• JavaScript ES6+ (ES modules)
• Async/await patterns
• Basic SQL (optional)
• Familiar with HTML/CSS

---

Architecture Overview

Backend (modules/)

Module	Responsibility
database.js	SQLite connection, getConfig(), setConfig()
sessionAuth.js	Session-based auth, CSRF, rate limiting
ha.js	Home Assistant polling, entity fetching
mqtt.js	MQTT broker connections, topic subscriptions
modbus.js	Modbus TCP/Serial polling
external.js	External REST API polling
bms.js	Bluetooth BMS bridge polling
history.js	Populate legacy history table from latest_metrics
grid.js	Grid status polling, timeline, uptime hours
solar.js	Solar forecast (Solcast/Open-Meteo), power integration
savings.js	PV savings (today/week/month/all-time)
metrics.js	Current metrics, metric history
metricsManager.js	User-created metrics CRUD
dashboard-config.js	Dashboard layout (JSON storage)
backup.js	Database backup/restore
logger.js	Winston structured logging
utils.js	Grid state parser

Frontend (public/js/)

All ES modules loaded via <script type="module">.

Module	Purpose
main.js	Entry point — theme, config load, WebSocket, polling fallback
dashboard.js	CSS Grid layout + SortableJS drag-to-reorder
updater.js	updateWithState(state) dispatches to all active block updaters
components/index.js	componentBuilders registry
components/*.js	Individual block builders + updaters
api.js	Centralised fetch calls
charts.js	Chart.js lifecycle, range selectors, configurable datasets
theme.js	Light/dark mode
forecast.js	Solar forecast sparkline + weather rendering
tables.js	Lazy-load data tables
grid.js	Grid card helpers (timeline, formatting)
utils/blockId.js	Block ID generation

Data Flow

Backend Polling (30s interval)
    → buildDashboardState()
    → broadcast via WebSocket to all clients
    → Frontend updateWithState(state)
    → Each active block's updater reads state.metrics

Fallback: WebSocket fails → setInterval polls /api/dashboard-state every 60s

Dashboard Layout

Blocks live in a 12-column CSS Grid (#dashboard-container). Each block has:

• colSpan — grid-column span (1–12, defaults to 12). Controls width.
• rowSpan — optional min-height in px (0 = auto, content-determined).
• config.metrics — per-block metric name overrides (see below).

Drag-to-reorder: SortableJS enables drag-reorder on the dashboard for authenticated users. Order persists on drop.

Block wrappers (.dashboard-block) are transparent flex columns. Each component handles its own card styling internally.

---

Adding a New Block Type

A block is a reusable dashboard widget. Adding one involves frontend changes only — no database modifications.

Step 1: Create the Builder

Create public/js/components/myBlock.js:

export function buildMyBlock(block = {}) {
  const config = block.config || {};
  const metrics = config.metrics || {};
  // Optional: extract configurable metric names with defaults
  const myMetric = metrics.my_value || 'default_metric_name';

  const container = document.createElement('div');
  container.className = 'my-block card';
  container.style.background = 'var(--card-bg)';
  container.style.borderRadius = 'var(--radius)';
  container.style.padding = '1rem';
  container.style.boxShadow = 'var(--shadow)';
  container.style.border = '1px solid var(--border)';
  container.dataset.metricMap = JSON.stringify({ value: myMetric });

  container.innerHTML = `
    <h3 style="margin:0 0 0.5rem 0;">${escapeHtml(config.title || 'My Block')}</h3>
    <div class="my-block-value" data-metric="${escapeHtml(myMetric)}">--</div>
  `;

  return container;
}

function escapeHtml(str) {
  const div = document.createElement('div');
  div.textContent = str;
  return div.innerHTML;
}

Pattern: Store metric names on container.dataset.metricMap as JSON. Use data-metric attributes on value elements. Provide sensible defaults for backward compatibility.

Step 2: Register the Builder

Edit public/js/components/index.js:

import { buildMyBlock } from './myBlock.js';

export const componentBuilders = {
  // ...existing...
  'my-block': buildMyBlock
};

Step 3: Create the Updater

In public/js/components/myBlock.js:

export function updateMyBlock(state) {
  const container = document.querySelector('.my-block');
  if (!container) return;

  let metricsMap;
  try {
    metricsMap = JSON.parse(container.dataset.metricMap);
  } catch (e) { return; }

  const metricName = metricsMap.value;
  if (!metricName) return;

  const val = state.metrics?.[metricName]?.value;
  if (val !== undefined && val !== null) {
    const el = container.querySelector('.my-block-value');
    if (el) el.textContent = val.toFixed(1);
  }
}

The state object contains:

• state.metrics — { metricName: { value, timestamp } } — all latest metrics
• state.current — latest power values in kW
• state.savings — computed savings
• state.gridStatus / state.gridHours / state.gridTimeline — grid data
• state.powerHistory — 24h power time series
• state.dailyEnergyBar — 7d energy time series

Step 4: Register the Updater

Edit public/js/updater.js:

import { updateMyBlock } from './components/myBlock.js';

export function updateWithState(state) {
  const activeLayout = dashboardConfig.dashboards.find(
    db => db.id === dashboardConfig.activeDashboard
  )?.layout || [];

  // ...existing updates...

  if (activeLayout.some(b => b.type === 'my-block')) {
    updateMyBlock(state);
  }
}

Step 5: Register in Settings

Edit public/settings.js, find renderDashboardBlockEditor() and add to the type dropdown:

<option value="my-block">My Block</option>

Also add a config panel. In the config button handler:

} else if (block.type === 'my-block') {
  const currentMetrics = block.config?.metrics || {};
  configPanel.innerHTML = `
    <label>Block Title</label>
    <input type="text" class="config-title" value="${escapeHtml(block.config?.title || '')}" style="width:100%; margin-bottom:0.5rem;">
    <label>Value Metric</label>
    <select class="config-metric" data-role="value" style="width:100%; margin-bottom:0.5rem;">${generateMetricOptionsHtml(currentMetrics.value)}</select>
    <button class="fetch-btn save-config">Save</button>`;
}

And in the save handler:

} else if (block.type === 'my-block') {
  block.config.title = configPanel.querySelector('.config-title')?.value || '';
  block.config.metrics = {};
  configPanel.querySelectorAll('.config-metric').forEach(select => {
    const role = select.dataset.role;
    if (select.value) block.config.metrics[role] = select.value;
  });
}

The generateMetricOptionsHtml() helper is already available in settings.js — it populates <select> elements from the global allMetrics list.

Step 6: Add CSS

Add to public/style.css:

.my-block-value {
  font-size: var(--fs-large);
  font-weight: 600;
  color: var(--text);
}

---

Adding a New Metric / Data Source

Option A: Use an Existing Source

If your data comes from Home Assistant, MQTT, Modbus, or external REST API:

1. Go to Settings → appropriate device section
2. Click + Add Metric Mapping
3. Enter metric name (e.g., grid_frequency)
4. Map to entity ID / MQTT topic / Modbus register / JSON path
5. Click Save All Settings

The backend immediately starts collecting. The metric appears in latest_metrics and is available to all blocks.

Option B: New Data Source Type

Create modules/mySource.js:

const { getConfig, getDb } = require('./database');
const { logger } = require('./logger');

async function pollMySource() {
  const sources = JSON.parse(getConfig('my_sources') || '[]');
  if (!sources.length) return;

  const db = getDb();
  const metricInsert = db.prepare(
    'INSERT OR IGNORE INTO metrics (timestamp, metric, value) VALUES (?, ?, ?)'
  );
  const latestUpsert = db.prepare(
    'INSERT OR REPLACE INTO latest_metrics (metric, value, timestamp) VALUES (?, ?, ?)'
  );

  for (const src of sources) {
    if (!src.enabled) continue;
    try {
      const data = await fetchFromMySource(src);
      const now = Math.floor(Date.now() / 1000);
      for (const [metric, value] of Object.entries(data)) {
        if (value !== undefined && !isNaN(value)) {
          metricInsert.run(now, metric, value);
          latestUpsert.run(metric, value, now);
        }
      }
    } catch (err) {
      logger.error(`MySource ${src.name} error: ${err.message}`);
    }
  }
}

function startMySourcePolling() {
  setInterval(pollMySource, 60000);
  pollMySource();
}

module.exports = { startMySourcePolling };

Register in server.js:

const { startMySourcePolling } = require('./modules/mySource');
// After other polling setup:
startMySourcePolling();

Add the config key to database.js essentials list:

const essentialKeys = [ ..., 'my_sources' ];

Add settings UI in public/settings.html and public/settings.js (follow the external sources pattern).

---

Understanding the Aggregated Endpoint

/api/dashboard-state returns all data in one request. Built by buildDashboardState() in server.js.

To add custom data, extend buildDashboardState():

async function buildDashboardState() {
  // ...existing queries...

  const myExtraData = db.prepare('SELECT ...').all();

  return {
    // ...existing fields...
    myExtraData
  };
}

Then in your updater: state.myExtraData.

Prefer extending the aggregated endpoint over creating new API routes — it keeps WebSocket broadcasts efficient.

---

WebSocket Real-Time Updates

The backend broadcasts new state after every polling cycle (30s). Frontend receives it and calls updateWithState(state).

Client-side (main.js)

ws.onmessage = (event) => {
  const message = JSON.parse(event.data);
  if (message.type === 'dashboard-state') {
    updateWithState(message.data);
  }
};

WebSocket closed → automatic fallback to 60s polling via updateAllComponents().

Custom Broadcast Triggers

If your data source updates asynchronously (e.g., MQTT message received), trigger a broadcast:

const state = await buildDashboardState();
broadcastDashboardState(state);

No additional client-side work needed — blocks benefit from real-time updates automatically.

---

Coding Conventions

Security

// Use textContent for user data (not innerHTML)
element.textContent = userInput;

// Escape HTML when innerHTML is unavoidable
element.innerHTML = `<div>${escapeHtml(userInput)}</div>`;

// Use parameterised SQL queries
db.prepare('SELECT * FROM metrics WHERE metric = ?').all(metricName);

Frontend

// Named exports
export function buildMyBlock(block = {}) { ... }
export function updateMyBlock(state) { ... }

// Lazy imports for heavy modules
const module = await import('./charts.js');

// Store config on DOM for updater access
container.dataset.metricMap = JSON.stringify(metrics);

// Check for element existence before updating
const el = document.getElementById('my-element');
if (!el) return;

// Check value exists before displaying
const val = state.metrics?.[name]?.value;
if (val === undefined || val === null) return;

Backend

// Lazy DB access
const { getDb } = require('./database');
const db = getDb();

// Cache prepared statements
let cachedStmt = null;
function getMyStmt() {
  if (!cachedStmt) cachedStmt = getDb().prepare('SELECT ...');
  return cachedStmt;
}

// Polling functions never throw — log and continue
async function pollMySource() {
  try { /* ... */ }
  catch (err) { logger.error('Poll error:', err.message); }
}

---

Complete Example: Frequency Card

1. Create public/js/components/frequencyCard.js

export function buildFrequencyCard(block = {}) {
  const config = block.config || {};
  const metrics = config.metrics || {};
  const freqMetric = metrics.frequency || 'grid_frequency';

  const container = document.createElement('div');
  container.className = 'stat-card';
  container.dataset.metricMap = JSON.stringify({ frequency: freqMetric });

  container.innerHTML = `
    <div class="stat-label">${escapeHtml(config.title || 'Grid Frequency')}</div>
    <div class="stat-value" data-metric="${escapeHtml(freqMetric)}">-- Hz</div>
  `;
  return container;
}

export function updateFrequencyCard(state) {
  const container = document.querySelector('.stat-card');
  if (!container) return;

  let metricsMap;
  try { metricsMap = JSON.parse(container.dataset.metricMap); }
  catch (e) { return; }

  const freq = state.metrics?.[metricsMap.frequency]?.value;
  if (freq === undefined) return;

  const el = container.querySelector('.stat-value');
  if (el) {
    el.textContent = `${freq.toFixed(2)} Hz`;
    el.style.color = (freq >= 47.5 && freq <= 52.5) ? '#51cf66' : '#ff6b6b';
  }
}

function escapeHtml(str) {
  const div = document.createElement('div');
  div.textContent = str;
  return div.innerHTML;
}

2. Register in components/index.js

import { buildFrequencyCard } from './frequencyCard.js';
// ...
'frequency-card': buildFrequencyCard,

3. Register in updater.js

import { updateFrequencyCard } from './components/frequencyCard.js';
// ...
if (activeLayout.some(b => b.type === 'frequency-card')) updateFrequencyCard(state);

4. Add to settings type dropdown + config panel + save handler

Follow the pattern in Step 5 of "Adding a New Block Type" above.

5. Map the metric

Go to Settings → data source → + Add Metric Mapping → metric name grid_frequency → map to entity/topic/register.

6. Add the block, save, refresh dashboard

---

FAQ

My block isn't showing

1. Builder registered in components/index.js?
2. <option> added in settings.js type dropdown?
3. Hard refresh browser (Ctrl+Shift+R)?
4. No console errors?
5. Block enabled !== false in config?

WebSocket not connecting

• Reverse proxy may not support WebSocket upgrade.
• Dashboard still works — falls back to 60s polling.

Metric exists in latest_metrics but not in my block

• Metric names are case-sensitive.
• Check state.metrics in browser console.

How do I add a new chart range?

Edit charts.js — add a range button, extend the data fetch in refreshPowerChart() or refreshEnergyChart().

Can I have a static block (no live updates)?

Yes — the builder runs, the updater can be empty. It still appears and re-renders on tab switches.

---

Related Docs

• Main README — setup, features, configuration reference
• GitHub Issues — bug reports, feature requests

---

Happy coding!