Skip to content

Prototype agent, controller, and Nakama backend foundation#4

Merged
JOY (JOY) merged 16 commits into
mainfrom
dev
May 16, 2026
Merged

Prototype agent, controller, and Nakama backend foundation#4
JOY (JOY) merged 16 commits into
mainfrom
dev

Conversation

@JOY
Copy link
Copy Markdown
Contributor

What does this PR do?

Promotes the current prototype foundation from dev to main: Simple KCC player control, local visual loading, animation/equipment visual handling, prototype AI agent/NPC brain, Nakama OSS backend runtime, and Cloud Run gateway schema updates.

Linked issue / ADR

  • Motivated by docs/adr/0010-nakama-oss-game-backend.md
  • Updates backend direction from Supabase-first prototype toward Nakama OSS game backend with Supabase identity bridge

Touched areas

  • Unity client (Unity/)
  • Dedicated server build flags / CI
  • Go gateway (backend/gateway/)
  • Supabase schema / RLS policies
  • DOS Chain integration / NFT contracts
  • AI agent runtime
  • Design docs (docs/design/) or ADRs (docs/adr/)
  • CI / project tooling

Test plan

  • cd backend/gateway && go test ./... - passed
  • cd backend/nakama && npm run build - passed
  • cd backend/nakama && npm test - passed
  • Deployed Cloud Run gateway revision second-spawn-gateway-00004-67h
  • Smoke tested POST /v1/agent/decide against Cloud Run with equipment inside context.body - returned 200 OK
  • Unity 6000.5.0b7 via CoplayDev MCP:
    • refreshed assets and waited for compilation to finish
    • entered Play Mode in Assets/_SecondSpawn/Scenes/ZoneTest_Hub.unity
    • waited 20 seconds
    • Unity Console warning/error query returned 0 entries
    • runtime inspection showed players=1, loaders=1, and equipment visual assigned on the authoritative player

Server-authority check (mandatory if touching gameplay)

  • No new gameplay logic runs on the Unity client as authoritative state
  • No new API key embedded in the Unity client
  • LLM outputs are validated as intent server-side, never auto-applied
  • New agent decision path goes through the gateway/Nakama intent contract and prototype validator before Unity applies narrow movement/say intents

Reviewer pass

JOY is non-coder per .claude/CLAUDE.md Hard Rule #7. Before merge, this PR should receive AI reviewer feedback from Claude Code Max and Gemini.

  • AI agent reviewer pass attached

JOY (JOY) and others added 13 commits May 14, 2026 16:07
…e prefab, PlayerSpawner

B-1: Rename Assets/_SecondSpawn/Scenes/SampleScene.unity -> ZoneTest_Hub.unity
     (GUID 99c9720ab356a0642a771bea13969a05 preserved, build settings updated).
B-2: Build scene hierarchy buckets per docs/setup/unity-conventions.md:
     _Managers, _Cameras, _Lights, _UI, World, _DynamicObjects. Reparent
     existing objects (Main Camera renamed to PlayerCamera under _Cameras;
     Directional Light + Global Volume under _Lights; _NetworkBootstrap
     under _Managers).
B-3: Add Player_NetworkCube.prefab (Cube + NetworkObject + NetworkPlayer)
     at Assets/_SecondSpawn/Prefabs/. Attach PlayerSpawner to _NetworkBootstrap
     and wire _playerPrefab + _spawnRoot SerializedObject refs. PlayerSpawner
     implements INetworkRunnerCallbacks; OnPlayerJoined spawns the prefab
     server-side with input authority assigned to the joining player and
     reparents under _DynamicObjects. Server-only spawn gate enforces
     Pillar 4 (server-authoritative) per Hard Rule #2.

Bundled (Phase A follow-through, not standalone):
- ProjectSettings.asset: runInBackground=1 (network tick continues with
  editor unfocused, required for Host Mode smoke test), Fusion 2.1.1
  scripting define symbols added for Standalone + Android.
- PhotonAppSettings.asset: SDK serializer auto-added empty AppIdRealtime,
  AppIdQuantum, NetworkLogging, ClientLogging fields.

Bundled (dev tooling):
- Packages/manifest.json + packages-lock.json: add com.coplaydev.unity-mcp
  package (HTTP localhost:8080 MCP bridge). Replaces Unity AI Assistant
  relay for Claude Code agents; bypasses cap=1 entitlement contention
  documented in CLAUDE.md sustainable MCP workflow. Unity AI Assistant
  package retained for Codex agent.

Code review: APPROVED (one low-urgency should-fix deferred to slice
phase 2 when pets/mounts introduce multi-NetworkObject ownership).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ion 2 reparent pattern (observed not to stick)

- Wrap INetworkRunnerCallbacks.OnUserSimulationMessage with #pragma warning
  disable CS0618 in PlayerSpawner.cs + NetworkInputProvider.cs. Fusion 2.1.1
  marked the SimulationMessagePtr parameter type obsolete ("Not used anymore"
  per Photon/Fusion/release_history.txt line 408) but the interface still
  requires the implementation, so the warning cannot be eliminated by code
  removal. The pragma is scoped to the single method, with a comment citing
  the release_history reference.

- Refactor PlayerSpawner.OnPlayerJoined to use Fusion 2 idiomatic
  runner.Spawn(prefab, pos, rot, player, onBeforeSpawned: callback) overload.
  Captures _spawnRoot into a local before passing the closure so re-spawns
  read the field fresh on each call. Calls Transform.SetParent inside the
  onBeforeSpawned callback (after instantiation, before Spawned()) which is
  the documented Fusion 2 pattern for setting a NetworkObject's parent at
  spawn time.

  KNOWN ISSUE: Play mode verification shows the reparent does NOT stick under
  Fusion 2.1.1 - the cube still ends up at scene root rather than under
  _DynamicObjects. The TODO comment in the callback flags the gap for a
  Phase B follow-up investigation (likely Fusion moves the GO into a
  runner-managed scene after the callback). Tracked for the next slice.

Phase B smoke-test core criteria remain met: cube spawns on PlayerJoined
with NetworkObject + NetworkPlayer + weaver-generated NetworkObjectPrefabData
attached. Reparent organization is cosmetic for the smoke test scope.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… (FieldAccessException workaround)

Add SECOND SPAWN PATCH AssemblyAttributes.cs declaring
`[assembly: System.Runtime.CompilerServices.IgnoresAccessChecksTo("Fusion.Runtime")]`
on SecondSpawn.Networking. Without this, Fusion 2.1.1's IL weaver emits
direct accesses to Fusion.NetworkBehaviour::Ptr (internal field) inside
every [Networked] property getter/setter, but Fusion.Runtime.dll's
[InternalsVisibleTo] list only grants access to Fusion.Unity.Tests/Editor,
Unity.Fusion.CodeGen, Fusion.Plugin/Json/Runtime.Tests - it does NOT
include SecondSpawn.Networking. CLR raises FieldAccessException at runtime
as soon as NetworkPlayer.Spawned() executes set_NetworkedPosition.

Phase A smoke test passed (commit 7aacfe5) because NetworkInputProvider's
INetworkInput is a plain data struct that never accesses Ptr. Phase B
(commit f109928) NetworkPlayer is the first NetworkBehaviour that does.

IgnoresAccessChecksToAttribute is not in BCL - declared the type inline
(Roslyn matches by full name). Confirmed Play mode passes the
FieldAccessException barrier after this patch in Unity 6.5 beta b7
plus Fusion 2.1.1-RC.

Long-term fix: when Photon ships a Fusion version that grants access via
AssembliesToWeave or exposes Ptr publicly, remove this file. Track
Photon/Fusion/release_history.txt; revert when confirmed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@JOY
Copy link
Copy Markdown
Contributor Author

Claude (@claude) review

Please review this PR with focus on:

  • Unity/Fusion 2.1 authority boundaries and whether any prototype code accidentally trusts client-side or LLM-side state.
  • Nakama OSS runtime + Supabase identity bridge architecture, especially whether the custom auth and profile/memory contracts are safe enough for prototype evolution.
  • Unity asset handling around paid/generated visual prefabs, equipment visual selection, and avoiding accidental committed paid Asset Store content.
  • Any merge blockers before this goes from dev to main.

Context: JOY is non-coder, so please call out concrete blocking issues first, then nice-to-have cleanup.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request transitions the game backend to Nakama OSS (ADR 0010) and integrates Photon Fusion Simple KCC for authoritative movement. It establishes a framework for AI agents and OpenClaw-connected NPCs, including character profiles, soul definitions, and memory systems. The changes span Unity client logic, a Go LLM gateway prototype, and Nakama runtime modules. Review feedback highlights critical networking issues, specifically the need to use networked state for velocity calculations in NetworkAnimatorBridge and to network agent inputs in NetworkPlayer to avoid simulation divergence. Further recommendations include optimizing object discovery in CharacterMemorySync, adding error handling for JSON parsing in Nakama modules, and adopting more robust methods for JWT parsing and unique ID generation.

Comment on lines +127 to +135
var worldVelocity = (transform.position - _previousPosition) / deltaTime;
worldVelocity.y = 0f;
_previousPosition = transform.position;

var localVelocity = transform.InverseTransformDirection(worldVelocity);
var referenceMoveSpeed = Mathf.Max(0.01f, _referenceMoveSpeed);
NetSpeed = Mathf.Clamp01(worldVelocity.magnitude / referenceMoveSpeed);
NetVelocityX = Mathf.Clamp(localVelocity.x / referenceMoveSpeed, -1f, 1f);
NetVelocityZ = Mathf.Clamp(localVelocity.z / referenceMoveSpeed, -1f, 1f);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The velocity calculation manually computes a delta from _previousPosition, which is not a [Networked] variable. In Photon Fusion, non-networked state is not rolled back during resimulations, leading to incorrect velocity values and animation glitches on the client when mispredictions occur. Since SimpleKCC is already a dependency on this object, it is more reliable and resimulation-safe to use _kcc.RealVelocity directly for these calculations.

            var worldVelocity = _kcc.RealVelocity;\n            worldVelocity.y = 0f;\n\n            var localVelocity = transform.InverseTransformDirection(worldVelocity);\n            var referenceMoveSpeed = Mathf.Max(0.01f, _referenceMoveSpeed);\n            NetSpeed = Mathf.Clamp01(worldVelocity.magnitude / referenceMoveSpeed);\n            NetVelocityX = Mathf.Clamp(localVelocity.x / referenceMoveSpeed, -1f, 1f);\n            NetVelocityZ = Mathf.Clamp(localVelocity.z / referenceMoveSpeed, -1f, 1f);

Comment on lines +103 to +111
public void SetPrototypeAgentInput(NetworkInputData input)
{
_prototypeAgentInput = input;
_hasPrototypeAgentInput = true;
if (HasStateAuthority)
{
IsAgentControlled = true;
}
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The SetPrototypeAgentInput method sets a local _prototypeAgentInput field which is not [Networked]. In a dedicated server topology, the server will not have access to this input if it is set by a client (e.g., via the PrototypeLLMAgentDriver which reads local keyboard input), causing the simulation to diverge. Furthermore, for the 'offline' agent feature to function, the server must be the authority driving the agent; if the input is only stored locally on a client, the server cannot simulate the agent's movement when the player is disconnected.


private static bool TryApplyProfileEquipment(int equipmentVisualId)
{
var players = Object.FindObjectsByType<NetworkPlayer>(FindObjectsInactive.Exclude, FindObjectsSortMode.None);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Object.FindObjectsByType is an expensive operation that searches the entire scene hierarchy. Calling it every frame within the ApplyProfileEquipmentWhenAvailable coroutine loop (line 114) for up to 10 seconds can cause significant performance spikes. Consider using a more efficient discovery mechanism, such as having the NetworkPlayer register itself with a manager upon spawning or using a static registry.

Comment on lines +435 to +443
var start = json.IndexOf(marker, StringComparison.Ordinal);
if (start < 0)
{
return "";
}

start += marker.Length;
var end = json.IndexOf('"', start);
return end > start ? json.Substring(start, end - start) : "";
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Manual parsing of JWT claims using string searching (json.IndexOf) is brittle and prone to failure if the JSON formatting changes (e.g., whitespace, field order, or minification). It is safer to use a proper JSON parser after base64-decoding the JWT payload segment to ensure compatibility with different serialization outputs.

Comment thread backend/nakama/modules/index.ts Outdated
return null;
}

var body = JSON.parse(response.body);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

JSON.parse(response.body) is called without a try-catch block. If the external Supabase Auth service returns an unexpected non-JSON response (such as an HTML error page from a proxy or an empty body), this will throw a runtime exception, causing the beforeAuthenticateCustom hook to fail and preventing user login.

  var body;\n  try {\n    body = JSON.parse(response.body);\n  } catch (e) {\n    logger.error("Supabase Auth returned invalid JSON: " + response.body);\n    return null;\n  }

memory.importance = clampNumber(memory.importance || 5, 1, 10);
if (!memory.id) {
memory.id = "mem-" + nowId() + "-" + (context.body.memory.length + 1);
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Generating IDs using Date.getTime() is prone to collisions if multiple memory records are added in the same millisecond. While the array length suffix provides some mitigation, it is not robust against concurrent requests or deletions in a distributed environment. Using a UUID or a more robust unique ID generator is recommended for production-ready data.

@JOY
Copy link
Copy Markdown
Contributor Author

Addressed Gemini review findings in 96a446f:

  • NetworkAnimatorBridge now uses SimpleKCC.RealVelocity instead of a non-networked _previousPosition delta.
  • NetworkPlayer.SetPrototypeAgentInput / ClearPrototypeAgentInput now only run on state authority, so prototype agent movement cannot be driven by a non-authoritative client.
  • SecondSpawnGatewayClient now parses Nakama JWT payload via JsonUtility instead of string searching.
  • Nakama Supabase auth bridge now handles invalid JSON responses safely.
  • Nakama memory IDs now include player id + timestamp + random suffix + sequence instead of timestamp-only.

Re-verified locally:

  • cd backend/gateway && go test ./...
  • cd backend/nakama && npm run build && npm test
  • Unity MCP refresh/compile and Play Mode smoke: no new warning/error after 20 seconds.

@JOY
Copy link
Copy Markdown
Contributor Author

Codex local review pass after recovered thread continuation.

Verdict: approved with merge held for optional/local Claude Code review, per repo hard rule.

What I checked:

  • PR diff against main for Unity/Fusion authority boundaries, LLM intent boundaries, Nakama ADR 0010 alignment, paid asset leakage, and secret leakage.
  • Existing Gemini findings are addressed in 96a446f.
  • Added one extra fix in 6bf29ce: PrototypeLLMAgentDriver now refuses to start the prototype agent loop on non-state-authority player instances, so non-authoritative clients cannot trigger unnecessary gateway/agent calls.
  • Removed a Unity 6.5 obsolete API warning in CharacterMemorySync.
  • Cleaned Unity-generated trailing whitespace / EOF noise so git diff --check passes locally.

Verification:

  • git diff --check passes locally.
  • backend/gateway: go test ./... passes.
  • backend/nakama: npm run build passes.
  • backend/nakama: npm test passes.
  • Unity MCP Play Mode smoke after the patch produced no C# compile errors and no new SecondSpawn script warnings/errors. Remaining console entries are existing Fusion/Unity plugin/editor-domain noise:
    • invalid Fusion CodeGen AssetDatabase path warning
    • duplicate UnityEditor.iOS.Extensions.Xcode reference warning
    • Fusion ScriptsReloaded assert from plugin settings
  • GitHub checks for head 6bf29ce are green: backend-test, markdown lint, CodeQL actions/csharp/go, GitBook.

No merge-blocking issues found by Codex local review. Holding actual dev -> main merge until JOY either runs local Claude Code review or explicitly waives that project-specific requirement for this PR.

@JOY JOY (JOY) merged commit 154ac15 into main May 16, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant