xresnet for vision classification - TypeError

[JakobPuehringerIpercept]

I try to follow the 06_TS_to_image_classification notebook on the latest version of tsai and the following dataset specs:

print(X.shape)
print(y.shape)

> (10, 3, 50000)
> (10,)

Running the following line, results in a TypeError:
learn = TSClassifier(X, y, splits=splits, bs=[2, 2], tfms=[None, Categorize()], batch_tfms=[TSNormalize(), TSToPlot()], arch=xresnet34, metrics=accuracy)

Error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[42], line 1
----> 1 learn = TSClassifier(X, y, splits=splits, bs=[2, 2], tfms=[None, Categorize()], batch_tfms=[TSNormalize(), TSToPlot()], arch=xresnet34, metrics=accuracy)

File [~/tsai/tsai/tslearner.py:67](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a226177735f564d5f677075227d.vscode-resource.vscode-cdn.net/home/ubuntu/InternJP/nbs_prototypes/~/tsai/tsai/tslearner.py:67), in TSClassifier.__init__(self, X, y, splits, tfms, inplace, sel_vars, sel_steps, weights, partial_n, vocab, train_metrics, valid_metrics, bs, batch_size, batch_tfms, pipelines, shuffle_train, drop_last, num_workers, do_setup, device, seed, arch, arch_config, pretrained, weights_path, exclude_head, cut, init, loss_func, opt_func, lr, metrics, cbs, wd, wd_bn_bias, train_bn, moms, path, model_dir, splitter, verbose)
     59     elif isinstance(arch, str): arch = get_arch(arch)
     60     # if 'xresnet' in arch.__name__.lower() and not '1d' in arch.__name__.lower():
     61     #     model = build_tsimage_model(arch, dls=dls, pretrained=pretrained, init=init, device=device, verbose=verbose, arch_config=arch_config)
     62     # elif 'tabularmodel' in arch.__name__.lower():
   (...)
     65     #     model = build_ts_model(arch, dls=dls, device=device, verbose=verbose, pretrained=pretrained, weights_path=weights_path,
     66     #                            exclude_head=exclude_head, cut=cut, init=init, arch_config=arch_config)
---> 67     model = build_ts_model(arch, dls=dls, device=device, verbose=verbose, pretrained=pretrained, weights_path=weights_path,
     68                             exclude_head=exclude_head, cut=cut, init=init, arch_config=arch_config)
     69 try:
     70     setattr(model, "__name__", arch.__name__)

File [~/tsai/tsai/models/utils.py:181](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a226177735f564d5f677075227d.vscode-resource.vscode-cdn.net/home/ubuntu/InternJP/nbs_prototypes/~/tsai/tsai/models/utils.py:181), in build_ts_model(arch, c_in, c_out, seq_len, d, dls, device, verbose, pretrained, weights_path, exclude_head, cut, init, arch_config, **kwargs)
    179 else:
    180     pv(f'arch: {arch.__name__}(c_in={c_in} c_out={c_out} arch_config={arch_config} kwargs={kwargs})', verbose)
--> 181     model = arch(c_in, c_out, **arch_config, **kwargs).to(device=device)
    183 try:
    184     model[0], model[1]

TypeError: xresnet34() takes from 0 to 1 positional arguments but 2 were given

After some digging into the source code (still a newbie to tsai), I discovered that the fastai model xresnet34() expects the parameter n_out, instead of c_out. The latest update in notebook nbs/030_models.utils.ipynb makes some changes to create xresnet models, so this could probably have to do something with it.

Setup:
python : 3.9.16
tsai : 0.3.6
fastai : 2.7.11
fastcore : 1.5.28
torch : 1.13.1+cu117

[oguiza]

Thanks for sharing this @JakobPuehringer.
There was indeed a bug in the code. Good catch! I've added support for xresnet now, so it should work.
As to your dataset, it's very strange. You seem to have only 10 samples with 50k time steps. I don't think the model will learn anything with such a small dataset. And if it works with such a long time series, it will be extremely slow.

[JakobPuehringerIpercept]

@oguiza no worries, glad to help
I downsampled the dataset for testing purposes and speeding up debugging, but still wanted to use my real data. Since I transform the ts to images of size 224x224 it should reduce the dimensions. Still, the transformation to image data is quite slow, so therefore I am thinking of collapsing the timeseries even further by taking the mean over x samples, but that is another topic.

[JakobPuehringerIpercept]

Also, I tested the workflow again and the initial bug does not occur anymore, but another error gets thrown.

1. When using the Learner class, the model gets now correctly initialized and training works.

model = create_model(xresnet34, dls=dls)
learn = Learner(dls, model, metrics=accuracy)
learn.fit_one_cycle(2)

2. When using the scikit-like TSClassifier class, initializing the model works too, but training does not.

learn = TSClassifier(X, y, splits=splits, bs=[2, 2], batch_tfms=batch_tfms, tfms=tfms, arch=xresnet34, metrics=accuracy)
learn.fit_one_cycle(2)

this results in the following error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[19], line 1
----> 1 learn.fit_one_cycle(2)

File [~/.pyenv/versions/3.9.16/lib/python3.9/site-packages/fastai/callback/schedule.py:114](https://vscode-remote+ssh-002.vscode-resource.vscode-cdn.net/home/ubuntu/InternJP/nbs_prototypes/~/.pyenv/versions/3.9.16/lib/python3.9/site-packages/fastai/callback/schedule.py:114), in fit_one_cycle(self, n_epoch, lr_max, div, div_final, pct_start, wd, moms, cbs, reset_opt, start_epoch)
    110 @patch
    111 def fit_one_cycle(self:Learner, n_epoch, lr_max=None, div=25., div_final=1e5, pct_start=0.25, wd=None,
    112                   moms=None, cbs=None, reset_opt=False, start_epoch=0):
    113     "Fit `self.model` for `n_epoch` using the 1cycle policy."
--> 114     if self.opt is None: self.create_opt()
    115     self.opt.set_hyper('lr', self.lr if lr_max is None else lr_max)
    116     lr_max = np.array([h['lr'] for h in self.opt.hypers])

File [~/.pyenv/versions/3.9.16/lib/python3.9/site-packages/fastai/learner.py:188](https://vscode-remote+ssh-002dremote.vscode-resource.vscode-cdn.net/home/ubuntu/InternJP/nbs_prototypes/~/.pyenv/versions/3.9.16/lib/python3.9/site-packages/fastai/learner.py:188), in Learner.create_opt(self)
    186     self.opt.clear_state()
    187 else:
--> 188     self.opt = self.opt_func(self.splitter(self.model), lr=self.lr)
    189 if not self.wd_bn_bias:
    190     for p in self._bn_bias_state(True ): p['do_wd'] = False

File [~/tsai/tsai/models/utils.py:111](https://vscode-remote+ssh-002dremote.vscode-resource.vscode-cdn.net/home/ubuntu/InternJP/nbs_prototypes/~/tsai/tsai/models/utils.py:111), in ts_splitter(m)
    109 def ts_splitter(m):
    110     "Split of a model between body and head"
--> 111     return L(m.backbone, m.head).map(params)

File [~/.pyenv/versions/3.9.16/lib/python3.9/site-packages/torch/nn/modules/module.py:1269](https://vscode-remote+ssh-002dremote.vscode-resource.vscode-cdn.net/home/ubuntu/InternJP/nbs_prototypes/~/.pyenv/versions/3.9.16/lib/python3.9/site-packages/torch/nn/modules/module.py:1269), in Module.__getattr__(self, name)
   1267     if name in modules:
   1268         return modules[name]
-> 1269 raise AttributeError("'{}' object has no attribute '{}'".format(
   1270     type(self).__name__, name))

AttributeError: 'XResNet' object has no attribute 'backbone'

Is it not intended to use xresnet models here?

[oguiza]

Hi @JakobPuehringer,
There was a bug in the code that I've just fixed in GitHub (will be available in tsai 0.3.6 which will be available soon). You can now use TSClassifier or TSRegressor with xresnet. Here's an example:

from tsai.basics import *
from tsai.data.image import TSToPlot
from tsai.models.utils import create_model
from fastai.vision.models.xresnet import xresnet34

X = np.random.rand(8, 3, 100)
y = np.random.randint(0, 3, (8))
splits = TimeSplitter()(y)
tfms = [None, TSCategorize()]
batch_tfms = [TSNormalize(), TSToPlot()]
learn = TSClassifier(X, y, splits=splits, tfms=tfms, bs=[64, 128], batch_tfms=batch_tfms, arch=xresnet34, metrics=accuracy)
learn.fit_one_cycle(1)

It'd be good if you could test it with your own code and confirm it works as expected.

[JakobPuehringerIpercept]

Hi @oguiza!
I pulled the new changes and tested the workflow again. I can confirm it works now flawlessly. Thanks for the effort.