Feature: Implementing Continuous OPE Estimators

[usaito]

new feature

estimators_continuous.py

• implement the following Continuous OPE estimators
  • KernelizedInverseProbabilityWeighting
  • KernelizedSelfNormalizedInverseProbabilityWeighting
  • KernelizedDoublyRobust
• implement some kernel functions
  • triangular_kernel
  • gaussian_kernel
  • epanechnikov_kernel
  • cosine_kernel

reference
Nathan Kallus and Angela Zhou.
"Policy Evaluation and Optimization with Continuous Treatments", AISTATS, 2018.

meta_continuous.py

• implement ContinuousOffPolicyEvaluation that streamlines OPE with continuous actions. This works as follows.

# (1) Synthetic Data Generation
dataset = SyntheticContinuousBanditDataset(dim_context=5)
bandit_feedback = dataset.obtain_batch_bandit_feedback(
    n_rounds=10000, min_action_value=-10, max_action_value=10,
)

# (2) Off-Policy Evaluation
ope = ContinuousOffPolicyEvaluation(
    bandit_feedback=bandit_feedback,
    ope_estimators=[KernelizedIPW(kernel="epanechnikov", bandwidth=0.02)]
)
estimated_policy_value = ope.estimate_policy_values(
     action_by_evaluation_policy=action_by_evaluation_policy,
)

tests

• add the following tests wrt the new features
  1. test_ipw_estimators_continuous.py
  2. test_dr_estimators_continuous.py
  3. test_meta_continuous.py
  4. test_kernel_functions.py

1 and 2 include some performance tests of the continuous OPE estimators using synthetic data as well as input checks.
4 checks whether the kernel functions satisfy the conditions described on page 9 of the following lecture slides: http://ibis.t.u-tokyo.ac.jp/suzuki/lecture/2015/dataanalysis/L9.pdf

• fix descriptions (mostly English expressions) of docstrings

[nomuramasahir0]

I'm just letting you know:
IDE (PyCharm) generates a non-default argument error, though I do not find any specific runtime error.

If you would like to avoid this error, it might be better to inherit BaseContinuousOffPolicyEstimator, rather than KernelizedInverseProbabilityWeighting.

[nomuramasahir0]

Other than the above one point, LGTM!

[usaito]

@nmasahiro Thanks!