Skip to content

fredgis/SkyNav

Repository files navigation

🎈 SkyNav

SkyNav Fleet Manager is a Microsoft Fabric workload designed to help operators monitor and manage a fleet of hot air balloons in real time. It brings together live telemetry, operational intelligence, semantic modeling, and conversational AI inside a single Fabric experience, and it is built with the Microsoft Fabric Extensibility Toolkit.

The scenario is a Loire Valley balloon operator that needs to monitor telemetry, understand operational relationships between balloons, pilots, flights, and sites, and ask natural-language questions about the fleet.

The platform is demonstrated end to end. An IoT telemetry simulator generates realistic balloon data such as position, altitude, speed, flight phase, wind conditions, temperature, and propane level. That telemetry is streamed through Eventstream, can trigger operational alerts in Activator, and is stored in Eventhouse, which becomes the analytical backbone for the live dashboard and the agent.

On top of the raw data, SkyNav adds a business ontology and a connected graph linking balloons, pilots, flights, sites, restricted zones, and weather windows. This semantic layer gives structure and meaning to the telemetry and becomes especially important when users interact with SkyNav Agent, a conversational assistant powered by gpt-5.2 in Microsoft AI Foundry and connected to Fabric data plus MCP-backed weather and wind tools.

Visual preview

SkyNav workload overview SkyNav live map view

SkyNav ontology and graph view SkyNav agent and operations view

🎬 Demo videos

SkyNavFullTelemetry.mp4
SkyNav.mp4

Open SkyNav demo video · Open full telemetry demo video

What is currently implemented

  • An IoT telemetry simulator that drives the end-to-end demo scenario
  • An Eventstream -> Activator -> Eventhouse pipeline with operational alerting, including low-propane scenarios
  • A Fabric workload frontend built with React, Fluent UI, and Leaflet
  • A backend API for chat, telemetry, ontology, health checks, and webhook handling
  • A conversational sidebar powered by SkyNavAgent
  • A gpt-5.2 deployment hosted in Microsoft AI Foundry and consumed through the Azure OpenAI-compatible endpoint
  • KQL-backed fleet queries plus MCP-backed external weather and wind
  • An ontology viewer built from the deployed Fabric ontology definition

/api/copilotkit remains available for compatibility and future evolution, but the current primary user-facing chat path is the custom AgentSidebar.tsx posting directly to /api/chat. Within that backend flow, the core fleet tools remain direct, and skynav-wind is now bridged additively through MCP for external weather and wind.

Architecture at a glance

graph TB
    subgraph Fabric[" Microsoft Fabric "]
        Portal["Fabric Portal"]
        Eventhouse[("KQL Eventhouse<br/>telemetry + weather + alerts")]
        Ontology["Fabric IQ Ontology<br/>(getDefinition)"]
        Activator["Fabric Activator"]
    end

    subgraph Azure[" Azure "]
        SWA["Static Web App<br/>Workload frontend"]

        subgraph Frontend[" Frontend "]
            MapTab["Map tab<br/>Leaflet + telemetry"]
            OntologyTab["Ontology tab<br/>entity graph"]
            AgentSidebar["AgentSidebar<br/>headless chat UX"]
        end

        subgraph Backend[" App Service backend "]
            ChatApi["/api/chat"]
            TelemetryApi["/api/telemetry"]
            OntologyApi["/api/ontology"]
            WebhookApi["/api/webhook/activator"]
            SkyNavAgent["SkyNavAgent"]
            CopilotRuntime["/api/copilotkit<br/>(compatibility path)"]
        end

        Foundry["Microsoft AI Foundry<br/>GPT-5.2 deployment<br/>(Azure OpenAI-compatible)"]
    end

    subgraph MCP[" MCP servers "]
        WindMCP["skynav-wind MCP<br/>external weather + wind"]
        OtherMCP["data / ontology MCPs<br/>(adjacent capabilities)"]
    end

    OpenMeteo["Open-Meteo API"]

    Portal -->|iframe| SWA
    SWA --> MapTab & OntologyTab & AgentSidebar

    AgentSidebar -->|POST /api/chat| ChatApi
    MapTab -->|poll every 5s when active| TelemetryApi
    OntologyTab -->|load graph data| OntologyApi

    ChatApi --> SkyNavAgent
    CopilotRuntime --> SkyNavAgent
    SkyNavAgent -->|chat completions + tool loop| Foundry
    SkyNavAgent -->|KQL tools| Eventhouse
    SkyNavAgent -->|MCP weather / wind| WindMCP
    WindMCP -->|HTTP| OpenMeteo
    TelemetryApi -->|KQL queries| Eventhouse
    OntologyApi -->|POST getDefinition + LRO| Ontology
    Activator -->|webhook| WebhookApi

    OtherMCP -.->|available, not primary chat path| SkyNavAgent

    style Fabric fill:transparent,stroke:#1565c0,stroke-width:2px,color:#1565c0
    style Azure fill:transparent,stroke:#e36209,stroke-width:2px,color:#e36209
    style Frontend fill:transparent,stroke:#2ea44f,stroke-dasharray:4,color:#555
    style Backend fill:transparent,stroke:#e36209,stroke-dasharray:4,color:#555
    style MCP fill:transparent,stroke:#8957e5,stroke-width:2px,color:#8957e5

    style Portal fill:#1565c0,stroke:#0d47a1,color:#fff
    style Eventhouse fill:#1565c0,stroke:#0d47a1,color:#fff
    style Ontology fill:#8957e5,stroke:#6e40c9,color:#fff
    style Activator fill:#d73a49,stroke:#b31d28,color:#fff

    style SWA fill:#2ea44f,stroke:#238636,color:#fff
    style MapTab fill:#2ea44f,stroke:#238636,color:#fff
    style OntologyTab fill:#2ea44f,stroke:#238636,color:#fff
    style AgentSidebar fill:#2ea44f,stroke:#238636,color:#fff

    style ChatApi fill:#e36209,stroke:#c24e00,color:#fff
    style TelemetryApi fill:#e36209,stroke:#c24e00,color:#fff
    style OntologyApi fill:#e36209,stroke:#c24e00,color:#fff
    style WebhookApi fill:#d73a49,stroke:#b31d28,color:#fff
    style SkyNavAgent fill:#8957e5,stroke:#6e40c9,color:#fff
    style CopilotRuntime fill:#8957e5,stroke:#6e40c9,color:#fff

    style Foundry fill:#8957e5,stroke:#6e40c9,color:#fff
    style WindMCP fill:#8957e5,stroke:#6e40c9,color:#fff
    style OtherMCP fill:#8957e5,stroke:#6e40c9,color:#fff
    style OpenMeteo fill:#6e5494,stroke:#553c7b,color:#fff
Loading

Blue = Fabric/data, green = frontend, orange = backend, purple = AI/adjacent capabilities, red = automation.

Current runtime flows

1. Conversational agent

The active chat loop is:

AgentSidebar.tsx -> POST /api/chat -> SkyNavAgent -> gpt-5.2 in Microsoft AI Foundry -> direct backend tools + MCP-backed external weather/wind -> response rendered in the sidebar

The backend tool loop currently focuses on:

  • get_fleet_status
  • get_balloon_detail
  • get_active_alerts
  • get_weather
  • geocode_location
  • get_external_weather
  • get_wind_at_altitude
  • get_wind_forecast

2. Ontology viewer

The ontology tab does not depend on undocumented entity endpoints. The backend:

  1. acquires a backend token with Managed Identity
  2. calls POST /ontologies/{id}/getDefinition
  3. polls the long-running operation
  4. decodes the returned definition parts
  5. normalizes entities and relationships for the frontend graph

3. Fleet map

The map is backed by KQL telemetry and trajectory queries. The current frontend behavior is optimized to reduce UI contention:

  • telemetry polling runs every 5 seconds only while the map tab is active
  • returning to the map triggers an immediate refresh
  • full fleet payloads are no longer mirrored into co-agent state every 5 seconds
  • the agent sidebar is decoupled from the large fleet context to reduce unnecessary rerenders

Key capabilities

Area What it does now
Fleet map Shows live balloon positions, selection, trajectories, and operational context
Balloon detail Displays detailed telemetry and per-balloon drill-down from the map
Ontology viewer Displays ontology entities, properties, and relationships as a graph
Agent sidebar Answers fleet and weather questions, focuses the map, and summarizes operational state
Security Uses Entra JWT validation, Managed Identity, rate limiting, and sanitized backend errors

Repository layout

Path Purpose
specs/001-* to specs/005-* Feature specs, quickstarts, and per-feature installation or deployment guides for the simulator, data platform, alerts, ontology, and MCP layer
src/Workload/Frontend React workload UI: map, ontology, agent sidebar, Fabric bootstrap
src/Workload/Backend Express API: chat, telemetry, ontology proxy, auth, health, webhooks
src/mcp-servers MCP servers kept in the repo, with skynav-wind now actively bridged by the backend for external weather and wind
specs/006-skynav-agent Workload specification, architecture, deployment documentation, contracts
docs Supporting documentation and integration notes

Deployment and authentication

The Fabric workload was initially developed on branch 006-SkyNav-Agent and is now documented here on main together with the supporting platform features.

  • Frontend hosting: Azure Static Web App
  • Backend hosting: Azure App Service
  • Model hosting: Microsoft AI Foundry, using a gpt-5.2 deployment exposed through the Azure OpenAI-compatible endpoint
  • User authentication: Fabric workload token validated by the backend through Entra JWKS
  • Backend access to Fabric and model resources: Managed Identity
  • Production path: no client secrets required for the main runtime

🎈 3D Fleet Viewer (IoT Simulator)

Real-time 3D visualization of the balloon fleet using CesiumJS on a globe with live telemetry via WebSocket. The viewer embeds the IoT telemetry simulator — no separate process needed. Balloons are rendered as montgolfière icons with color-coded flight phases, trails, and vertical tethers to ground.

Prerequisites

  • Python 3.11+ installed and available in PATH
  • Internet access (CesiumJS is loaded from CDN, OpenStreetMap tiles as fallback)
  • Optional: an Azure Event Hubs namespace for dual-publish (telemetry goes to both the browser and Event Hubs simultaneously)
  • Optional: a Cesium Ion access token for high-definition 3D terrain (without it, the viewer uses OpenStreetMap tiles)

Quick Start

# Windows — launch with 20 balloons, opens browser automatically
.\launch.ps1 -FleetSize 20

# Linux / macOS
chmod +x launch.sh && ./launch.sh --fleet-size 20

The launcher will:

  1. Create a Python virtual environment (.venv) if it doesn't exist
  2. Install all pip dependencies from src/requirements.txt
  3. Load environment variables from .env at the project root
  4. Start the viewer server on http://localhost:8080
  5. Open your default browser automatically
  6. Wait for you to click "Start Simulation"

Environment Variables (.env)

Copy .env.template to .env and fill in values. Never commit .env — it is in .gitignore.

Variable Default Required Description
SKYNAV_FLEET_SIZE from fleet.yaml (3) No Number of balloons to simulate (1–20+). Beyond 3, extra balloons are auto-generated with French names, random pilots, and Loire Valley launch sites.
SKYNAV_SCENARIO normal No Flight scenario: normal, storm, calm. Affects wind layers, propane burn rate, and cruise altitude.
SKYNAV_EMIT_INTERVAL 5.0 No Telemetry emission interval in seconds. Lower = faster simulation.
SKYNAV_VIEWER_PORT 8080 No HTTP port for the viewer server.
SKYNAV_EVENTHUB_CONNECTION_STRING No Azure Event Hubs connection string. When set, telemetry is published to both WebSocket (browser) and Event Hubs simultaneously.
SKYNAV_EVENTHUB_NAMESPACE No Event Hub FQDN for Entra ID (managed identity) auth. Alternative to connection string.
SKYNAV_EVENTHUB_NAME No Event Hub entity name. Required if using Event Hubs.
CESIUM_ION_TOKEN No Cesium Ion access token for HD 3D terrain with satellite imagery. Without it, the viewer falls back to OpenStreetMap 2D tiles — still fully functional.

Launcher Options

PowerShell Bash Description
-FleetSize 20 --fleet-size 20 Number of balloons (overrides .env and fleet.yaml)
-Scenario storm --scenario storm Flight scenario
-Port 9090 --port 9090 Override viewer HTTP port
-NoBrowser --no-browser Don't auto-open browser
-SkipInstall --skip-install Skip pip install (use after first run for faster startup)

Usage Examples

# Quick demo — 5 balloons, default scenario
.\launch.ps1 -FleetSize 5

# Full fleet, storm scenario, custom port
.\launch.ps1 -FleetSize 20 -Scenario storm -Port 9090

# Fast restart (skip pip install)
.\launch.ps1 -FleetSize 20 -SkipInstall

# Headless mode (no browser auto-open)
.\launch.ps1 -FleetSize 20 -NoBrowser
# Linux/macOS equivalents
./launch.sh --fleet-size 20 --scenario storm --port 9090
./launch.sh --fleet-size 20 --skip-install --no-browser

Fleet Configuration

The base fleet is defined in src/config/fleet.yaml (3 balloons by default). When SKYNAV_FLEET_SIZE exceeds the YAML count, extra balloons are auto-generated with:

  • French balloon names (Château Volant, Vent d'Ouest, Ciel de Loire...)
  • Random Loire Valley launch sites (Amboise, Chenonceau, Chaumont-sur-Loire...)
  • Varied envelope types and pilot names
  • Staggered launch delays (each subsequent balloon launches later)

Flight Phases & Balloon Colors

Phase Color Description
Preflight Gray Balloon on ground, waiting for launch delay
Ascending Blue Climbing to cruise altitude
Cruising Green At target altitude, drifting with wind
Descending Orange Returning to ground (propane low or cruise complete)
Landed Red On ground, flight complete

Viewer Features

  • Montgolfière icons — Balloons rendered as hot air balloon icons with envelope, basket, and flame, color-coded by phase
  • Real-time telemetry — Position, altitude, speed, heading, wind, temperature, propane level updated every 5s via WebSocket
  • Flight trails — Colored polyline trails showing each balloon's flight path
  • Vertical tethers — Dashed lines from balloon to ground showing height visually
  • Dashboard HUD — Active/ascending/landed counts, simulation tick counter
  • Info popup — Click a balloon for detailed telemetry table
  • Alert simulator — Trigger low propane, rapid descent, or strong wind alerts on any active balloon
  • Auto-reconnect — WebSocket reconnects automatically if connection drops

Architecture

Browser (CesiumJS)  <──WebSocket──>  Python Server (aiohttp)
                                          │
                                     Simulator Engine
                                     (tick every 5s)
                                          │
                                    ┌─────┼─────┐
                                    ▼           ▼
                              WebSocket    Event Hubs
                             (browser)    (optional)

The server runs entirely in one process: HTTP server (serves index.html) + WebSocket server (pushes telemetry) + simulator engine (ticks balloons). No external databases required for the viewer.

Documentation

Document Description
Global project plan Broader project plan across simulator, data platform, and workload
Toolchain plan How Agent Forge, Spec-Kit, and Squad are used across the project
Feature squads plan Breakdown of the work into feature squads and contracts-based parallelism
Feature 001 deployment guide Install and run the IoT telemetry simulator locally or with Azure Event Hubs
Feature 002 deployment guide Provision Event Hub, Eventhouse schema, Eventstream, and KQL validation flow
Feature 003 deployment guide Configure Activator rules, schema extensions, alert destinations, and validation
Feature 004 deployment guide Seed reference data, deploy the Fabric ontology, and validate the graph bindings
Workload deployment guide Manual deployment flow for Azure resources, Entra ID, workload packaging, frontend and backend
Workload architecture Detailed runtime architecture, data flows, security model, and deployment diagrams
Workload specification Functional and technical specification for the Fabric workload
Deployment via PowerShell Automated deployment path with deploy-skynav.ps1
MCP servers README Repository MCP server architecture and development notes

Notes

  • The KQL Eventhouse and Fabric ontology are existing Fabric workspace resources consumed by the workload
  • The current headless sidebar is the main production chat UX; CopilotKit runtime remains available for compatibility and future streaming scenarios
  • skynav-wind is now used additively from the backend for external Open-Meteo weather and wind lookups
  • The root README is intentionally concise; use the workload architecture document for the full technical breakdown

About

An aerial tourism company operates a fleet of IoT-equipped hot air balloons in the Vallée de la Loire. The SkyNav agent helps pilots and dispatch plan flights, monitor the fleet in real time, and ensure safety by cross-referencing internal telemetry with external weather/regulatory data

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors