DataAggregator is a modular infrastructure for collecting, registering, and processing industrial data. It provides a flexible backbone to acquire machine signals, store them in time-series and relational databases, and run ML-based processing pipelines.
It is designed to scale from a single machine to an entire plant, handling heterogeneous environments through dedicated collectors for each data source. Data can be visualized in real time using tools like Grafana or any custom frontend, while processing can run either at the edge or centrally. This enables the application of custom algorithms or ONNX-based models directly on the collected signals.
demo.webm
DataAggregator/
├─ DataAggregator.sln # Visual Studio solution entry point
├─ .github/workflows/ # CI/CD pipelines (build & test)
│ └─ buildAndTest.yml
├─ docker/ # Dockerfiles for each service
│ ├─ Dockerfile.Collector
│ ├─ Dockerfile.Processor
│ └─ Dockerfile.Registration
├─ env/ # Developement environment (docker-compose, volumes)
│ ├─ docker-compose.yml
│ └─ volumes/
│ └─ grafana/
├─ lib/ # External local libraries
│ └─ Capnp.Net.Runtime/ # Cap’n Proto .NET runtime
├─ src/ # Main application source code
│ ├─ DataAggregator.Shared/ # Common DTOs, models, helpers
│ ├─ DataAggregator.Collector/ # Collector host (ASP.NET Core service)
│ ├─ DataAggregator.Collector.Shared/ # Collector abstractions & config
│ ├─ DataAggregator.Collector.OpenCNCapnProtoConnector/ # OpenCN connector
│ ├─ DataAggregator.Collector.FileCollector/ # File-based collector for demo/testing
│ ├─ DataAggregator.Registration/ # Registration service (ASP.NET Core)
│ └─ DataAggregator.Processor/ # Processing service (ONNX models, ML logic)
├─ tests/
│ ├─ DataAggregator.Registration.Tests/ # Unit tests for Registration
│ ├─ DataAggregator.Collector.Tests/ # Unit tests for Registration
│ └─ DataAggregator.Processor.Tests/ # Unit tests for Processor
-
🔧 Deployment example (full stack with configs, Grafana, etc.) 👉 DataAggregator-demo
-
📊 Model training & analysis pipeline 👉 Data-analysis
-
🛠️ OpenCN fork with connector support (work in progress) 👉 OpenCN branch
cmctl-bufferization
- .NET 9 → Backend services (Collector, Registration, Processor)
- ML.NET -> Data processing
- Docker / Docker Compose → Deployment & orchestration
- InfluxDB v3 → Time-series data storage
- PostgreSQL → Relational storage & metadata
- Grafana (in demo repo) → Data visualization
DataAggregator is built around a modular and distributed architecture:
- Collector → Acquires machine signals (e.g., currents from CNC drives) via OpenCN or file input.
- Registration → Manages device registration and configuration in PostgreSQL.
- Processor → Extracts features and runs ML inference (ONNX models) to classify machine states.
- Databases → InfluxDB v3 stores raw and processed signals; PostgreSQL stores device metadata.
This design ensures scalability across multiple machines and flexibility for integration with existing industrial setups.
👉 For a ready-to-use deployment example (including Docker setup, Grafana dashboards, and configs), check out: 📦 DataAggregator-demo
This project is licensed under the PolyForm Noncommercial License 1.0.0. See the LICENSE file for details.
This project was developed as part of a Bachelor thesis and is published with permission. Any commercial use is not permitted without prior discussion with the author.
this readme was enhanced by AI to clean the formatting and structure the content