run bug: TypeError: Argument 'freq' has incorrect type

[tongnie]

Hi, this is an error when I try to run the command "python ./experiments/run_imputation.py --config imputation/spin.yaml --model-name spin --dataset-name bay_block", and my pandas version is 1.5.3, is there an easy way to fix this?
Thanks for your help!

===============================
Global seed set to 561122547
Traceback (most recent call last):
File "./experiments/run_imputation.py", line 301, in
run_experiment(args)
File "./experiments/run_imputation.py", line 146, in run_experiment
dataset = get_dataset(args.dataset_name)
File "./experiments/run_imputation.py", line 64, in get_dataset
return add_missing_values(PemsBay(), p_fault=p_fault, p_noise=p_noise,
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\tsl\datasets\pems_bay.py", line 32, in init
df, dist, mask = self.load()
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\tsl\datasets\pems_bay.py", line 78, in load
df, dist = self.load_raw()
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\tsl\datasets\pems_bay.py", line 69, in load_raw
datetime_idx = sorted(df.index)
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\pandas\core\arrays\datetimes.py", line 590, in iter
converted = ints_to_pydatetime(
TypeError: Argument 'freq' has incorrect type (expected pandas.libs.tslibs.offsets.BaseOffset, got numpy.bytes)

[marshka]

Hi, I run the code with pandas version 1.4.4 and it works. However, you can change the code to make it work with your version. Lines

datetime_idx = sorted(df.index)
date_range = pd.date_range(datetime_idx[0], datetime_idx[-1], freq='5T')
df = df.reindex(index=date_range)

make the sequence regularly sampled, adding missing timestamps with nan values. You can change this sequence of instructions with

date_range = pd.date_range(df.index.min(), df.index.max(), freq='5T')
df = df.reindex(index=date_range)

or simply

df = df.asfreq('5T')

to obtain the same result.

[tongnie]

Thanks for your quick reply! This solution works for me, however, another new issue occurs and it seems to have something to do with the pytorch-lightning version:

Traceback (most recent call last):
File "./experiments/run_imputation.py", line 301, in
run_experiment(args)
File "./experiments/run_imputation.py", line 274, in run_experiment
trainer.fit(imputer,
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 740, in fit
self._call_and_handle_interrupt(
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 685, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 777, in _fit_impl
self._run(model, ckpt_path=ckpt_path)
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1145, in _run
self.accelerator.setup(self)
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\pytorch_lightning\accelerators\gpu.py", line 46, in setup
return super().setup(trainer)
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\pytorch_lightning\accelerators\accelerator.py", line 93, in setup
self.setup_optimizers(trainer)
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\pytorch_lightning\accelerators\accelerator.py", line 354, in setup_optimizers
optimizers, lr_schedulers, optimizer_frequencies = self.training_type_plugin.init_optimizers(
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\pytorch_lightning\plugins\training_type\training_type_plugin.py", line 245, in init_optimizers
return trainer.init_optimizers(model)
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\pytorch_lightning\trainer\optimizers.py", line 44, in init_optimizers
lr_schedulers = self._configure_schedulers(lr_schedulers, monitor, not pl_module.automatic_optimization)
File "C:\ProgramData\Anaconda3\envs\spin\lib\site-packages\pytorch_lightning\trainer\optimizers.py", line 192, in _configure_schedulers
raise ValueError(f'The provided lr scheduler "{scheduler}" is invalid')
ValueError: The provided lr scheduler "<spin.scheduler.CosineSchedulerWithRestarts object at 0x000001F29D41DCD0>" is invalid

When I change the default lr scheduler to 'cosine', this problem still happens. And my pytorch-lightning version is 1.5.10.