Client server architecture

The toolbox has been implemented in Java, and it is therefore recommended to implement and test models and algorithms directly in Java files. In case you want to perform constrained planning in another programming language, then it is possible to communicate with the toolbox based on a client-server architecture. The toolbox acts as a server, and external processes can connect to the server in order to request solutions to planning problems. A high-level overview of the architecture is shown below. An example implementation of the client-server communication is provided in the python folder, which provides a full implementation of a Python client.

Starting the server

The toolbox server can be started by running the file executables.Server, which waits for clients that connect on port 8080.

Overview of requests

A request is represented by a string, and the server responds to a request by sending a response string. Below we give an overview of the requests that can be handled by the server.

Instance requests

• dumpDefaultDomain_domainName_numAgents_numDecisions - Writes a problem instance to an XML file for the given number of agents and decisions. The instance is stored in the file javaInstance.xml. The domainName should be one of the following options: advertising, cbm, maze, tclFixedLimit, tclMultiLevel, webad. The response of the server is an empty string. Example request: dumpDefaultDomain_advertising_3_10

Solve requests

Solve requests can be used to compute a solution for a given problem instance using an algorithm. The server does not send the solution back to the client. Instead, it keeps the solution in memory, such that is can be used afterwards when receiving simulation requests.

• solveDefaultDomain_domainName_numAgents_numDecisions_algName - Loads a problem instance and computes a solution using the specified algorithm. The domainName should be one of the following options: advertising, cbm, maze, tclFixedLimit, tclMultiLevel, webad. The algName should be one of the following options: cmdp, colgen, deterministicpreallocation, dynamicrelaxation, cgcp, calp. The response of the server is the expected reward of the computed solution. Example request: solveDefaultDomain_advertising_3_10
• solveXMLDomainMDP_algName - Loads an MDP problem instance from pythonInstance.xml and computes a solution using the specified algorithm. The algName should be one of the following options: cmdp, colgen, deterministicpreallocation, dynamicrelaxation|alpha|beta. For dynamic relaxation the tolerance alpha and the relaxation factor beta are part of the algorithm name. The response of the server is the expected reward of the computed solution. Example requests: solveXMLDomainMDP_cmdp, solveXMLDomainMDP_dynamicrelaxation|0.05|2
• solveXMLDomainPOMDP_algName - Loads a POMDP problem instance from pythonInstance.xml and computes a solution using the specified algorithm. The algName should be one of the following options: cgcp, calp. The response of the server is the expected reward of the computed solution. Example request: solveXMLDomainPOMDP_calp

Simulation requests

The client can perform evaluation of the solution by running simulation runs. The client never receives the solution to the problem explicitly. Instead, it needs to send requests to the server in order to get the actions to be executed by the agents. In addition, the client needs to communicate information back to the server, depending on the outcome of the actions executed.

• startSimulation - This request must be sent prior to starting simulation runs. The response of the server is an empty string.
• startNewSimulationRun_agentID - This request must be sent prior to starting a new run for an agent. A run consists of one sequence of actions, starting from the initial (belief) state. The agentID is the id of the agent. The response of the server is an empty string. Example request: startNewSimulationRun_3
• getActionCMDP_agentID_t_s - After solving an MDP problem instance, this request can be used to obtain the action to be executed by agent agentID in state s at time t. The response of the server is an action. Example request: getActionCMDP_2_3_89
• getActionCPOMDP_agentID_t_b - After solving a POMDP problem instance, this request can be used to obtain the action to be executed by agent agentID in belief b at time t. The response of the server is an action. Example request: getActionCMDP_2_3_[0.25,0.25,0.25,0.25]
• updatePolicy_agentID_a_o - During POMDP simulations it may be necessary to update the policy data structure, depending on the observation received by the agent. In the request, a represents the executed action, and o represents the observation. The response of the server is an empty string. Example request: updatePolicy_2_10_8

Server requests

The client can send server requests to close the connection. It is also possible to shut down the server process completely.

• disconnect - This request must be sent when client wants to close the connection. The response of the server is an empty string.
• shutdown - This request kills the server process. The response of the server is an empty string.