-
Notifications
You must be signed in to change notification settings - Fork 1
Domains
The toolbox has several built-in problem domains which can be obtained using a CMDPInstanceGenerator or CPOMDPInstanceGenerator in the domains package. Each domain provides a function getInstance(int numAgents, int numDecisions) which returns a CMDPInstance or CPOMDPInstance with the given number of agents and decisions. This instance also defines the resource limits which impose constraints on the behavior of the agents.
describe notion of problem instance in the code
Several domains have been integrated in the toolbox already. A brief description of the domains is provided below, including references to the literature which either uses or describes the domain.
Online advertising involves presenting advertisements to users that browse the internet in such a way that they become interested in, e.g., buying a product in a webshop. If there is only a limited amount of money available for advertising, then it is required to decide how this budget is spent in order to maximize revenue. Each user browsing on the internet is modeled as a Markov Decision Process in which states represent the level of interest of the user and actions represent the advertisements that can be shown to the user. Each action has cost associated with it, corresponding to the amount of money that is required to show the advertisement. The global budget imposes a constraint on the advertisements that can be shown to the users.
Type of constraints: budget
Literature: Boutilier, C., & Lu, T. (2016). Budget Allocation using Weakly Coupled, Constrained Markov Decision Processes. In Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence (pp. 823–830).
De Nijs, F., Walraven, E., de Weerdt, M. M., & Spaan, M. T. J. (2017). Bounding the Probability of Resource Constraint Violations in Multi-Agent MDPs. In Proceedings of the 31st AAAI Conference on Artificial Intelligence (pp. 3562–3568).
todo
todo
todo
todo
explain how new domain can be used
The ConstrainedPlanningToolbox has been developed by the Algorithmics group at Delft University of Technology, The Netherlands. Please visit our website for more information.