CognitiveBlocks is a Minecraft mod + SDK built with the Fabric API that brings GPT-powered intelligence directly into the game world.
Imagine agents, NPCs, and in-game assistants that can understand and respond to your natural language — with memory, tools, and the ability to act in the world.
This is the start of that vision.
- 🤖
/chatcommand to talk to an AI model (OpenAI GPT) - 🧠 Memory for player conversations and context
- 🌍 Real-time player state logging (biome, location, inventory)
- 🎭 Dynamic AI personalities (friendly assistant, sarcastic gamer, etc.)
- 🛠️ Extensible prompt system (
SystemPrompt) with variable injection - 🔧 Foundation for future agent-driven tools and behaviors
This is not just a mod — it's a platform:
Goal: Create controllable, autonomous AI agents in Minecraft that:
- Understand the world state
- Accept natural language commands
- Execute multi-step plans (with LLM guidance)
- Adapt to the player and environment over time
- AI-controlled NPCs (companions, workers, scouts)
- Natural language task execution
- Pathfinding (A* with smart caching)
- Tool use + interaction with modded content
- Dynamic personality switching
- In-game model diagnostics and debugging
- Java 17+
- IntelliJ IDEA (recommended)
- Minecraft 1.20.x
- Fabric Mod Loader
- OpenAI API Key
git clone https://github.com/yourname/CognitiveBlocks.git
cd CognitiveBlocks
# Open in IntelliJ and sync Gradle