XPP Benchmarks

IPC Benchmarks

• Original domain files and upper bounds from: http://api.planning.domains/
• We used all instances from the optimal IPC trak until 2018.
• ipc_cost_bounds.py contains the cost bounts for x * best known cost bound with x in {0.25, 0.5, 0.75}

Used in the following publications:

• R. Eifler, M. Cashmore, J. Hoffmann, D. Magazzeni, and M. Steinmetz A New Approach to Plan-Space Explanation: Analyzing Plan-Property Dependencies in Oversubscription Planning Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI'20), New York City, USA, 2020
• R. Eifler, M. Steinmetz, A. Torralba, and J. Hoffmann, Plan-Space Explanation via Plan-Property Dependencies: Faster Algorithms & More Powerful Properties Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI'20), 2020

Resource Constraint Domains

Folder Structure

• <domain name>
  • versions with action costs
  • versions without resourc constraints
  • domain.pddl
  • <goals_X>
    • properties
      • AS_vs_LTL
      • LTL_only
      • all
      • learning_test_125
      • learning_test_150
    • `<scale_X>

Plan Properties

Attention: This property files still use the old syntax. The new syntax is based on JSON. An update is comming soon.

• AS_vs_LTL: Same plan properties once encoded in as 'Action Sets' once as LTLf formula
• LTL_only: Properties which need LTLf, for example orderings.
• all: All properties using either AS or LTL definition depending on what is easier to use.

Used in the following publications:

• R. Eifler, M. Cashmore, J. Hoffmann, D. Magazzeni, and M. Steinmetz A New Approach to Plan-Space Explanation: Analyzing Plan-Property Dependencies in Oversubscription Planning Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI'20), New York City, USA, 2020
• R. Eifler, M. Steinmetz, A. Torralba, and J. Hoffmann, Plan-Space Explanation via Plan-Property Dependencies: Faster Algorithms & More Powerful Properties Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI'20), 2020

Temporal Preference Learning Template Formulas

• Reference formulas used in the paper are in learning_test_150. f_x_pos.json contains the original template formula and f_x_neg.json the negated version.
• For each domain we fixed the number of goals:
  • blocksworld: 6
  • nomystery: 4
  • rovers: 6
  • TPP: 4

Used in the following publications:

• V. Seimetz, R. Eifler, and J. Hoffmann, Learning Temporal Plan Preferences from Examples: An Empirical Study Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI'21), 2021

Data Sets

Temporal Preference Learning Data Set (IJCAI21_data)

free_plan_generation

• This data belongs to the application setup, where the planners do not know the target formula.
• plans: Contains the first 50 plans generated by the corresponding planner.
• annotated: Run folders with plans annotated according to the satisfaction of the target formula. Each folder corresponds to one target formula.
• learning: Results file of the learning step.

target_formula

• This data belongs to the idealized setup where the planner knows the target formula and can generate positive and negative examples explicitly.
• plan_generation: Contains the generated examples plans by enforcing the target formulas and the negated target formula.
• learning: Results file of the learning step.