Skip to content

HydrAIDE - The Adaptive, Intelligent Data Engine, built in Go. Production-ready and optimized for real-time, large-scale data operations.

License

Notifications You must be signed in to change notification settings

hydraide/hydraide

HydrAIDE – Adaptive Intelligent Data Engine

HydrAIDE - The Adaptive, Intelligent Data Engine

License Version Status Speed Go Join Discord


🚀 Major Update: V2 Storage Engine (Version 3.0)

HydrAIDE 3.0 introduces a completely redesigned storage engine that delivers:

Improvement Gain
Write Speed 32-112x faster
Storage Size 50% smaller
File Count 95% fewer files
SSD Lifespan 100x longer

👉 Read the full V2 Storage Engine documentation to understand how it works under the hood.

✅ Backward Compatible

The V2 engine is fully backward compatible with existing V1 data. Both engines work side-by-side:

  • V1 Engine: Multi-chunk file storage (legacy)
  • V2 Engine: Single-file append-only storage (recommended)

⚠️ Migration Strongly Recommended

While V1 continues to work, we strongly recommend migrating to V2 for optimal performance.

⚡ IMPORTANT: Always create a full backup before migration!

Use hydraidectl to migrate your data:

# 1. Create a backup first!
# 2. Dry-run to verify (no changes made)
hydraidectl migrate --source /path/to/hydraide/data --dry-run

# 3. Run actual migration
hydraidectl migrate --source /path/to/hydraide/data --workers 4

👉 See full migration guide: hydraidectl Migration Guide


🧠 What is HydrAIDE?

One engine that replaces your database, cache, and pub/sub — just save your structs.

No schema design. No queries. No cleanup scripts. HydrAIDE automatically handles persistence, real-time events, distribution, and memory for you.

For developers who want:

  • Less code and infrastructure — everything in one place
  • Instant data access with O(1) folder-based routing
  • Native reactivity — every change emits a real-time event
  • Memory-efficient operation — data only lives in RAM when needed
  • Safe concurrency — built-in per-key distributed locking

With HydrAIDE, you don’t adapt to the database — the database adapts to your intent.


⚙️ What HydrAIDE Does – In One Stack

Feature Description
🗂️ Database Engine A NoSQL-like, structure-first data store — no schemas, no query language. Just save your Go structs. 👉 read more
💾 V2 Storage Engine Append-only, single-file-per-Swamp storage with 32-112x faster writes, 50% smaller files, and automatic compaction. 👉 read more
🔄 Built-in Reactivity & Subscription logic Native real-time subscriptions on every write/update/delete. Like Redis Pub/Sub, but smarter. 👉 read more
⚡️ O(1) access Deterministic, constant-time O(1) access to data — every Swamp name maps directly to a fixed folder location, ensuring instant lookup without indexing or scanning. 👉 read more
🔐 Concurrency-Safe Per-object locking with deadlock-free critical sections. Easy and safe for business rules. 👉 read more
🛡️ Built-in business lock Per-key, distributed locking that works across services and servers — ideal for enforcing business-level rules without race conditions. HydrAIDE automatically queues lock requests (FIFO), applies a TTL to prevent deadlocks, and releases locks safely even if a service crashes 👉 read more
🧠 Memory-Efficient Swamps live in memory only when accessed. Instant hydration, automatic disposal when idle. 👉 read more
🧹 Zero Garbage No daemons. No cron jobs. No cleanup scripts. Swamps manage themselves via lifecycle logic. 👉 read more
✍️ No More Queries No SELECT, no WHERE, no JOINS, no Aggregates. Your struct is the query. 👉 read more
🛰️ Pure gRPC Control Fully gRPC-native. Works with or without SDKs. Ideal for CLI tools, edge services, and IoT nodes. 👉 read more
🌐 Scaling Without Orchestrator Deterministic folder-based distribution. No orchestrators. Just spawn instances where needed. 👉 read more
🤯 Less Infra Headache No need to combine Redis + Kafka + Mongo + scheduler. HydrAIDE is the backend stack itself. 👉 read more

🚀 Start HydrAIDE in 2 Minutes

The fastest way to run HydrAIDE is using the hydraidectl CLI. No config files. No docker. No complexity.

✅ Recommended: Install with hydraidectl

  1. Download the CLI (Linux):

    curl -sSfL https://raw.githubusercontent.com/hydraide/hydraide/main/scripts/install-hydraidectl.sh | bash

    👉 For Windows, and full install guide, see the How to install hydraidectl

  2. Create a new instance:

    hydraidectl init

    👉 Available command details: hydraidectl user manual

  3. Start HydrAIDE as a background service:

    sudo hydraidectl service --instance <your-instance-name>

    👉 Read the full installation guide for more details: How to install HydrAIDE under 2 minutes


🐳 Prefer Docker?
You can also install and run HydrAIDE using Docker.
👉 Docker Installation Guide


💡 Proven in the Real World

HydrAIDE already powers platforms like Trendizz.com, indexing millions of websites and billions of structured relationships, with real-time search across hundreds of millions of words in under 1 seconds, without preloading.

Read the full story behind the hydrAIDE: How I Made Europe Searchable From a Single Server - The HydrAIDE Story

In production for over 2 years.
Replaces Redis, MongoDB, Kafka, cron jobs, and their glue code.


🚀 Demo Applications & Model examples

Explore ready-to-run demo applications built in Go to better understand the HydrAIDE Go SDK and its unique data modeling approach.

These examples are a great starting point to learn how to:

  • Structure your HydrAIDE-powered services
  • Use profile and catalog models
  • Handle real-time, reactive data flows efficiently

✅ Primary SDK: Go

HydrAIDE is written in Go, and hydraidego is the official SDK powering production at scale.

  • Supports everything: save/read, locking, subscriptions, TTLs, indexes – all native
  • Zero boilerplate: just write structs, and it just works
  • Fast, typed, reactive – built to feel like part of Go itself

🧠 Designed for real-time systems.
🔥 Used in live infrastructure today.
📚 Comes with full docs, examples, and patterns.


🤖 ChatGPT Support: Learn & Build with HydrAIDE Faster

The HydrAIDE documentation is purposefully structured to make it fully compatible with LLM-based assistants like ChatGPT — so you can focus on building instead of spending hours reading.

⚠️ HydrAIDE was not created using ChatGPT or any LLM. It is the result of years of real-world engineering experience. However, we believe in leveraging AI tools wherever they can accelerate your work — especially when learning new architectures or building production-grade systems.

✅ Turn ChatGPT into your personal HydrAIDE expert

To do that, simply create a ChatGPT project, and upload the following files:

File Type Path
Installation Guide HydrAIDE installation guide
LLM-Friendly Q&A Set hydraide-questions-answers-for-llm.md
Go SDK Documentation go-sdk.md
Go Example Models All .go files from models
Go SDK Core Logic hydraidego.go

Once uploaded, ChatGPT will be able to:

  • answer any question about HydrAIDE’s architecture or APIs,
  • help you write HydrAIDE-style Go code interactively,
  • explain example models, functions, and patterns,
  • and guide you through debugging, architecture design, or optimization steps.

💡 The documentation is written to be semantically consumable by AI, which means ChatGPT will understand not just APIs, but the design philosophy, naming logic, and intent behind each HydrAIDE feature.

A prebuilt ChatGPT is also available for the HydrAIDE Knowledge Engine. You can use it via the ChatGPT store or directly through this link: https://chatgpt.com/g/g-688779751c988191b975beaf7f68801d-hydraide-knowledge-engine Feel free to ask it anything! If it can’t answer your question, open an issue, or build your own custom GPT project with enhanced responses, as we described above.


📊 Comparisons - HydrAIDE vs Other Databases

Want to see how HydrAIDE compares to the most popular databases and engines?
We’re building a full series of deep comparisons, mindset-first, not config-first.


🌱 Every commit builds more than just code. It builds a mindset. HydrAIDE is not just a tool. It’s a way of thinking. If you see potential here, don’t just watch — contribute. Because we’re not just building a system. We’re building a community of systems thinkers.

Ready to leave your mark? Join us on Discord and let’s build the HydrAIDE together. 🚀

  • Start by reading the Contributor Introduction, it explains why HydrAIDE exists, what kind of people we’re looking for, and how you can join.
  • Then check out our Contribution Guide, it walks you through the practical steps.

Once you're ready, open your first issue or pull request. We’ll be waiting! 🚀


📩 Contact & Enterprise

HydrAIDE is used in production at Trendizz.com. Interested in enterprise licensing, SDK development, or embedding HydrAIDE in your own platform?

📧 Peter Gebripeter.gebri@hydraide.io (Founder of HydrAIDE & Trendizz) 🌐 Websitehttps://HydrAIDE.io Currently in progress and directly linked to GitHub.

Join the movement. Build different.

About

HydrAIDE - The Adaptive, Intelligent Data Engine, built in Go. Production-ready and optimized for real-time, large-scale data operations.

Topics

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Packages

 
 
 

Contributors 11

Languages