[BUG] InitialStateValueEstimationEvaluator assertion errors

[jdesman1]

Describe the bug
Using an InitialStateValueEstimationEvaluator in minimal examples causes assertion errors in the pip distributed v2.0.3.

To Reproduce

!pip install d3rlpy

from d3rlpy.datasets import get_cartpole
from d3rlpy.algos import DiscreteCQLConfig
from d3rlpy.metrics import InitialStateValueEstimationEvaluator

dataset, env = get_cartpole()
cql = DiscreteCQLConfig().create(device=None)
cql.build_with_dataset(dataset)

initial_estimator = InitialStateValueEstimationEvaluator(episodes=dataset.episodes)

cql.fit(
    dataset,
    n_steps=10000,
    evaluators={
        'init': initial_estimator
    },
)

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
[<ipython-input-11-5173f046dee1>](https://localhost:8080/#) in <cell line: 11>()
      9 initial_estimator = InitialStateValueEstimationEvaluator(episodes=dataset.episodes)
     10 
---> 11 cql.fit(
     12     dataset,
     13     n_steps=10000,

3 frames
[/usr/local/lib/python3.10/dist-packages/d3rlpy/algos/qlearning/base.py](https://localhost:8080/#) in fit(self, dataset, n_steps, n_steps_per_epoch, experiment_name, with_timestamp, logger_adapter, show_progress, save_interval, evaluators, callback, epoch_callback)
    402             List of result tuples (epoch, metrics) per epoch.
    403         """
--> 404         results = list(
    405             self.fitter(
    406                 dataset,

[/usr/local/lib/python3.10/dist-packages/d3rlpy/algos/qlearning/base.py](https://localhost:8080/#) in fitter(self, dataset, n_steps, n_steps_per_epoch, experiment_name, with_timestamp, logger_adapter, show_progress, save_interval, evaluators, callback, epoch_callback)
    546             if evaluators:
    547                 for name, evaluator in evaluators.items():
--> 548                     test_score = evaluator(self, dataset)
    549                     logger.add_metric(name, test_score)
    550 

[/usr/local/lib/python3.10/dist-packages/d3rlpy/metrics/evaluators.py](https://localhost:8080/#) in __call__(self, algo, dataset)
    264             ):
    265                 # estimate action-value in initial states
--> 266                 actions = algo.predict([batch.observations[0]])
    267                 values = algo.predict_value([batch.observations[0]], actions)
    268                 total_values.append(values[0])

[/usr/local/lib/python3.10/dist-packages/d3rlpy/algos/qlearning/base.py](https://localhost:8080/#) in predict(self, x)
    256         """
    257         assert self._impl is not None, IMPL_NOT_INITIALIZED_ERROR
--> 258         assert check_non_1d_array(x), "Input must have batch dimension."
    259 
    260         # TODO: support tuple inputs

AssertionError: Input must have batch dimension.

Expected behavior
Evaluation and standard logging.

Additional context
N/A

[takuseno]

@jdesman1 Thank you for reporting this! This has been fixed at this commit: c27f0a4 . I'll release a patch that includes this fix later today.

[takuseno]

The latest patch has been released.
https://github.com/takuseno/d3rlpy/releases/tag/v2.0.4