Swift for TensorFlow ...for Unity ML-Agents (Gym)

Minimal Swift-Client for Unity ML-Agents
(old and outdated)

Required:

• Unity ML-Agents 0.8.0
• S4TF Release 0.8.0
• Only working with Environment Executables
• MacOS Version 10.13+; using Swift-Protobuf and BlueSocket

Examples

• DQN
• PPO
• UDRL

Setup (Unity-Env)

• Academy.cs using SocketCommunicator.cs instead of RpcCommunicator.cs (L292 & L307):

communicator = new RPCCommunicator(...);
// =>
communicator = new SocketCommunicator(...);

• SocketCommunicator (L15): Environment timeout value

API

• Create/close environment:

let path = "/...PATHTO/Env.app"
guard let env = try? UnityGym(path) else {
    exit(0)
}
defer { env.close() }

• Environment Data:

let actionDims = env.actionSpace.shape
// Cont. actionSpace:
let actionDim = actionDims[0]
// (Multi)-Disc. actionSpace:
let actionDim0 = actionDims[0]
let actionDim1 = actionDims[1]
...
let nO = env.numStackedObservations
let inputLen = env.observationSpace.shape.contiguousSize * nO
let n = env.nAgents

• Environment Interaction:

let obs = env.observation
let obs = try env.reset()
let (obs, reward, done, maxStepReached) = try env.step(action)

step(Tensor<Int32> / Tensor<Float>? = nil)
// TensorShape: (nAgents, actionDim), 
// nil/ opt. => Random action

/* 
obs: Tensor<Float>, TensorShape: (nAgents, observationSpace)
reward: Tensor<Float>, TensorShape: (nAgents)
done: Tensor<Bool>, TensorShape: (nAgents)
maxStepReached: Tensor<Bool>, TensorShape: (nAgents) 
*/

do {
    var o1 = try env.reset()
    let a = model(o1)
    let (o2, r, d, m) = try env.step(a)
} catch {
    print(error)
}