[BUG] v2.0.3 DiscreteCQL loss is not a scalar

[jdesman1]

Describe the bug
The loss of DiscreteCQL is not being logged as a scalar.

To Reproduce

from d3rlpy.datasets import get_cartpole
from d3rlpy.algos import DiscreteCQLConfig
from d3rlpy.metrics import TDErrorEvaluator
from d3rlpy.metrics import EnvironmentEvaluator

dataset, env = get_cartpole()
cql= DiscreteCQLConfig().create(device=None)
cql.build_with_dataset(dataset)
td_error_evaluator = TDErrorEvaluator(episodes=dataset.episodes)
env_evaluator = EnvironmentEvaluator(env)
rewards = env_evaluator(cql, dataset=None)

cql.fit(
    dataset,
    n_steps=10000,
    evaluators={
        'td_error': td_error_evaluator,
        'environment': env_evaluator,
    },
)

Inspection of the logged CQL loss reveals it is now of form [float, float], although the tqdm display shows it as a scalar.

Expected behavior
Loss should be logged as a scalar, instead now producing a list.

Additional context
N/A

[takuseno]

@jdesman1 I really appreciate your checks! This has been fixed at the latest commit: 62100ab . I'll wait for a couple of days before I release the next patch since it seems that I missed some bugs at this big release. Thanks!

[takuseno]

If you need, you can use the latest implementation by installing from source.

[jdesman1]

Happy to help! I'll open more issues if I come across anything else.

[takuseno]

The latest patch has been released.
https://github.com/takuseno/d3rlpy/releases/tag/v2.0.4