Codex Mod for Minecraft (made with codex and chatgpt for openai build week hackathon)
MinecraftGPT is a client-side Fabric mod for Minecraft Java Edition. It lets a player ask questions and request visual help from the standalone Codex CLI that is authenticated with ChatGPT.
The mod can answer text questions in Minecraft chat, optionally inspect the visible game environment, track custom visual goals, and build certain structures such as houses.
- Sends ordinary text questions to a locally installed Codex CLI and displays the answer in Minecraft chat.
- Supports an explicit visual-context mode that can send a screenshot and 3D scan the player's environment.
- Stores separate conversation mappings for different world/player combinations.
- Provides local visual goals that can be visually checked by codex with the world context and player's environment.
- Provides a structured builder with preview, confirmation, verification, saved history, and undo.
This is a client-side mod intended primarily for single-player use. The builder requires a Creative single-player world. It does not provide multiplayer world support.
The complete installation guide is in INSTALLATION.md.
Used for general queries.
Note: Running /codex -i captures your inventory and current position, performs a bounded 3D scan of the area around you, and sends a screenshot of the game to Codex. Use this mode for environment-based questions, allowing Codex to understand your surroundings and where you are located.
/codex
/codex help
/codex <question>
/codex -i <question>
/codex new
/codex status
| Command | Behavior |
|---|---|
/codex help |
Shows the in-game command summary. |
/codex <question> |
Sends a text-only question to the current world/player conversation. |
/codex -i <question> |
Sends the question with an explicit screenshot and bounded game context. Uses player's current position to do a 3D scan and inspect the environment. |
/codex new |
Starts a fresh main conversation mapping for the current world/player. |
/codex status |
Checks the executable, CLI version, ChatGPT login state, and local thread state. |
Goals are MinecraftGPT’s own local achievement-style records.
/codex goal add <description>
/codex goal list
/codex goal check <numeric-id>
/codex goal clear
Used to build structures. Please note that this feature is still in development and currently has restrictive safety checks. Block entities/NBT data, such as furnaces and chests, are rejected. Builds near water may also be rejected, along with other unsafe situations.
These checks “fail closed,” meaning that if the mod cannot prove a build is safe, it rejects the entire plan without changing any blocks.
For now, the builder works best with small, simple structures, such as an oak-wood house built on a flat, open area. This feature is still in development and more advanced building support will be added later.
Builds can take quite a bit of time so please be patient when the command is running.
/codex build
/codex build <request>
/codex build --name "name" <request>
/codex build status
/codex build confirm
/codex build cancel
/codex build history [page]
/codex build show <id-or-prefix>
/codex build hide
/codex build rename <id-or-prefix> "new name"
/codex build undo
/codex build undo <id-or-prefix>
| Command | Behavior |
|---|---|
/codex build |
Shows build status when no request is supplied. |
/codex build <request> |
Asks Codex for one structured block plan and displays a no-write preview. |
/codex build --name "name" <request> |
Creates a build plan with a player-supplied history name. |
/codex build status |
Shows planning, preview, saved overlay, and undo state |
/codex build confirm |
Must use this to confirm the preview. This Revalidates, snapshots, applies, and verifies the current preview |
/codex build cancel |
Cancels planning, discards an unconfirmed preview, or hides an applied diff. |
/codex build history [page] |
Lists saved build records for the current world/player and dimension. |
/codex build show <id-or-prefix> |
Displays a saved forward diff without reading live blocks or changing the world. |
/codex build hide |
Hides the saved diff overlay without changing the world or history. |
/codex build rename <id-or-prefix> "new name" |
Changes only the saved history name. |
/codex build undo [id-or-prefix] |
Restores the exact saved before-states after conflict checks. |
The preview uses colors to communicate the planned diff:
- Green: blocks being added.
- Yellow: existing blocks being changed.
- Red: blocks being removed.
Useful if you would want to change the configuration of certain settings.
%APPDATA%\.minecraft\config\minecraftgpt\config.json
This should be:
{
"codexExecutable": "",
"model": "gpt-5.6-luna",
"reasoningEffort": "medium",
"scanRadius": 6,
"verticalScanRadius": 3,
"timeoutSeconds": 180,
"buildTimeoutSeconds": 600,
"maxReplyCharacters": 6000,
"keepContextScreenshots": false
}Important settings:
codexExecutable: leave blank for automatic Codex discovery, or set an absolute executable path.model: model used by the local Codex CLI for Minecraft requests.reasoningEffort: reasoning level passed to Codex.scanRadius: horizontal visual scan radius, clamped to 2–12 blocks.verticalScanRadius: vertical visual scan radius, clamped to 1–6 blocks.timeoutSeconds: timeout for normal chat, visual questions, and goal checks.buildTimeoutSeconds: independent timeout for structured build planning.maxReplyCharacters: maximum reply length shown in Minecraft chat.keepContextScreenshots: when false, temporary context screenshots are removed after use.
Restart Minecraft after changing the configuration.