You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
print(X.shape,y.shape)
arch_config = dict(
n_layers=3, # number of encoder layers
n_heads=4, # number of heads
d_model=16, # dimension of model
d_ff=128, # dimension of fully connected network
attn_dropout=0.0, # dropout applied to the attention weights
dropout=0.3, # dropout applied to all linear layers in the encoder except q,k&v projections
patch_len=24, # length of the patch applied to the time series to create patches
stride=2, # stride used when creating patches
padding_patch=True, # padding_patch
)
learn = TSForecaster(X, y, splits=splits, batch_size=16, path="models",
arch="PatchTST", arch_config=arch_config, metrics=[mse, mae], cbs=ShowGraph())
learn.summary()
It ran well till here:
(790157, 21, 36) (790157, 1, 1)
not enough values to plot a chart
PatchTST (Input shape: 16 x 21 x 36)
============================================================================
Layer (type) Output Shape Param # Trainable
============================================================================
16 x 21 x 1
RevIN 42 True
____________________________________________________________________________
16 x 21 x 38
ReplicationPad1d
____________________________________________________________________________
16 x 24 x 8
Unfold
____________________________________________________________________________
16 x 21 x 8 x 16
Linear 400 True
Dropout
Linear 272 True
Linear 272 True
Linear 272 True
Dropout
Linear 272 True
Dropout
Dropout
____________________________________________________________________________
16 x 16 x 8
Transpose
BatchNorm1d 32 True
____________________________________________________________________________
16 x 8 x 16
Transpose
____________________________________________________________________________
16 x 8 x 128
Linear 2176 True
GELU
Dropout
____________________________________________________________________________
16 x 8 x 16
Linear 2064 True
Dropout
____________________________________________________________________________
16 x 16 x 8
Transpose
BatchNorm1d 32 True
____________________________________________________________________________
16 x 8 x 16
Transpose
Linear 272 True
Linear 272 True
Linear 272 True
Dropout
Linear 272 True
Dropout
Dropout
____________________________________________________________________________
16 x 16 x 8
Transpose
BatchNorm1d 32 True
____________________________________________________________________________
16 x 8 x 16
Transpose
____________________________________________________________________________
16 x 8 x 128
Linear 2176 True
GELU
Dropout
____________________________________________________________________________
16 x 8 x 16
Linear 2064 True
Dropout
____________________________________________________________________________
16 x 16 x 8
Transpose
BatchNorm1d 32 True
____________________________________________________________________________
16 x 8 x 16
Transpose
Linear 272 True
Linear 272 True
Linear 272 True
Dropout
Linear 272 True
Dropout
Dropout
____________________________________________________________________________
16 x 16 x 8
Transpose
BatchNorm1d 32 True
____________________________________________________________________________
16 x 8 x 16
Transpose
____________________________________________________________________________
16 x 8 x 128
Linear 2176 True
GELU
Dropout
____________________________________________________________________________
16 x 8 x 16
Linear 2064 True
Dropout
____________________________________________________________________________
16 x 16 x 8
Transpose
BatchNorm1d 32 True
____________________________________________________________________________
16 x 8 x 16
Transpose
____________________________________________________________________________
16 x 21 x 128
Flatten
____________________________________________________________________________
16 x 21 x 1
Linear 129 True
____________________________________________________________________________
Total params: 16,747
Total trainable params: 16,747
Total non-trainable params: 0
Optimizer used: <function Adam at 0x7ec3f1a4b0a0>
Loss function: FlattenedLoss of MSELoss()
Callbacks:
- TrainEvalCallback
- CastToTensor
- Recorder
- ProgressCallback
- ShowGraph
But either lr_find or fit_one_cycle failed with following error. Where was I wrong and how to fix? Thanks.
0.00% [0/1 00:00<?]
0.00% [0/39507 00:00<?]
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
[<ipython-input-9-332f966dfa8c>](https://localhost:8080/#) in <cell line: 1>()
----> 1 lr_max = learn.lr_find().valley
20 frames
[/usr/local/lib/python3.10/dist-packages/fastai/callback/schedule.py](https://localhost:8080/#) in lr_find(self, start_lr, end_lr, num_it, stop_div, show_plot, suggest_funcs)
291 n_epoch = num_it//len(self.dls.train) + 1
292 cb=LRFinder(start_lr=start_lr, end_lr=end_lr, num_it=num_it, stop_div=stop_div)
--> 293 with self.no_logging(): self.fit(n_epoch, cbs=cb)
294 if suggest_funcs is not None:
295 lrs, losses = tensor(self.recorder.lrs[num_it//10:-5]), tensor(self.recorder.losses[num_it//10:-5])
[/usr/local/lib/python3.10/dist-packages/fastai/learner.py](https://localhost:8080/#) in fit(self, n_epoch, lr, wd, cbs, reset_opt, start_epoch)
262 self.opt.set_hypers(lr=self.lr if lr is None else lr)
263 self.n_epoch = n_epoch
--> 264 self._with_events(self._do_fit, 'fit', CancelFitException, self._end_cleanup)
265
266 def _end_cleanup(self): self.dl,self.xb,self.yb,self.pred,self.loss = None,(None,),(None,),None,None
[/usr/local/lib/python3.10/dist-packages/fastai/learner.py](https://localhost:8080/#) in _with_events(self, f, event_type, ex, final)
197
198 def _with_events(self, f, event_type, ex, final=noop):
--> 199 try: self(f'before_{event_type}'); f()
200 except ex: self(f'after_cancel_{event_type}')
201 self(f'after_{event_type}'); final()
[/usr/local/lib/python3.10/dist-packages/fastai/learner.py](https://localhost:8080/#) in _do_fit(self)
251 for epoch in range(self.n_epoch):
252 self.epoch=epoch
--> 253 self._with_events(self._do_epoch, 'epoch', CancelEpochException)
254
255 def fit(self, n_epoch, lr=None, wd=None, cbs=None, reset_opt=False, start_epoch=0):
[/usr/local/lib/python3.10/dist-packages/fastai/learner.py](https://localhost:8080/#) in _with_events(self, f, event_type, ex, final)
197
198 def _with_events(self, f, event_type, ex, final=noop):
--> 199 try: self(f'before_{event_type}'); f()
200 except ex: self(f'after_cancel_{event_type}')
201 self(f'after_{event_type}'); final()
[/usr/local/lib/python3.10/dist-packages/fastai/learner.py](https://localhost:8080/#) in _do_epoch(self)
245
246 def _do_epoch(self):
--> 247 self._do_epoch_train()
248 self._do_epoch_validate()
249
[/usr/local/lib/python3.10/dist-packages/fastai/learner.py](https://localhost:8080/#) in _do_epoch_train(self)
237 def _do_epoch_train(self):
238 self.dl = self.dls.train
--> 239 self._with_events(self.all_batches, 'train', CancelTrainException)
240
241 def _do_epoch_validate(self, ds_idx=1, dl=None):
[/usr/local/lib/python3.10/dist-packages/fastai/learner.py](https://localhost:8080/#) in _with_events(self, f, event_type, ex, final)
197
198 def _with_events(self, f, event_type, ex, final=noop):
--> 199 try: self(f'before_{event_type}'); f()
200 except ex: self(f'after_cancel_{event_type}')
201 self(f'after_{event_type}'); final()
[/usr/local/lib/python3.10/dist-packages/fastai/learner.py](https://localhost:8080/#) in all_batches(self)
203 def all_batches(self):
204 self.n_iter = len(self.dl)
--> 205 for o in enumerate(self.dl): self.one_batch(*o)
206
207 def _backward(self): self.loss_grad.backward()
[/usr/local/lib/python3.10/dist-packages/tsai/learner.py](https://localhost:8080/#) in one_batch(self, i, b)
38 b_on_device = to_device(b, device=self.dls.device) if self.dls.device is not None else b
39 self._split(b_on_device)
---> 40 self._with_events(self._do_one_batch, 'batch', CancelBatchException)
41
42 # %% ../nbs/018_learner.ipynb 7
[/usr/local/lib/python3.10/dist-packages/fastai/learner.py](https://localhost:8080/#) in _with_events(self, f, event_type, ex, final)
197
198 def _with_events(self, f, event_type, ex, final=noop):
--> 199 try: self(f'before_{event_type}'); f()
200 except ex: self(f'after_cancel_{event_type}')
201 self(f'after_{event_type}'); final()
[/usr/local/lib/python3.10/dist-packages/fastai/learner.py](https://localhost:8080/#) in _do_one_batch(self)
217 self('after_pred')
218 if len(self.yb):
--> 219 self.loss_grad = self.loss_func(self.pred, *self.yb)
220 self.loss = self.loss_grad.clone()
221 self('after_loss')
[/usr/local/lib/python3.10/dist-packages/fastai/losses.py](https://localhost:8080/#) in __call__(self, inp, targ, **kwargs)
52 if targ.dtype in [torch.int8, torch.int16, torch.int32]: targ = targ.long()
53 if self.flatten: inp = inp.view(-1,inp.shape[-1]) if self.is_2d else inp.view(-1)
---> 54 return self.func.__call__(inp, targ.view(-1) if self.flatten else targ, **kwargs)
55
56 def to(self, device:torch.device):
[/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py](https://localhost:8080/#) in _call_impl(self, *args, **kwargs)
1499 or _global_backward_pre_hooks or _global_backward_hooks
1500 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501 return forward_call(*args, **kwargs)
1502 # Do not call functions when jit is used
1503 full_backward_hooks, non_full_backward_hooks = [], []
[/usr/local/lib/python3.10/dist-packages/torch/nn/modules/loss.py](https://localhost:8080/#) in forward(self, input, target)
534
535 def forward(self, input: Tensor, target: Tensor) -> Tensor:
--> 536 return F.mse_loss(input, target, reduction=self.reduction)
537
538
[/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py](https://localhost:8080/#) in mse_loss(input, target, size_average, reduce, reduction)
3279 """
3280 if has_torch_function_variadic(input, target):
-> 3281 return handle_torch_function(
3282 mse_loss, (input, target), input, target, size_average=size_average, reduce=reduce, reduction=reduction
3283 )
[/usr/local/lib/python3.10/dist-packages/torch/overrides.py](https://localhost:8080/#) in handle_torch_function(public_api, relevant_args, *args, **kwargs)
1549 # Use `public_api` instead of `implementation` so __torch_function__
1550 # implementations can do equality/identity comparisons.
-> 1551 result = torch_func_method(public_api, types, args, kwargs)
1552
1553 if result is not NotImplemented:
[/usr/local/lib/python3.10/dist-packages/fastai/torch_core.py](https://localhost:8080/#) in __torch_function__(cls, func, types, args, kwargs)
380 if cls.debug and func.__name__ not in ('__str__','__repr__'): print(func, types, args, kwargs)
381 if _torch_handled(args, cls._opt, func): types = (torch.Tensor,)
--> 382 res = super().__torch_function__(func, types, args, ifnone(kwargs, {}))
383 dict_objs = _find_args(args) if args else _find_args(list(kwargs.values()))
384 if issubclass(type(res),TensorBase) and dict_objs: res.set_meta(dict_objs[0],as_copy=True)
[/usr/local/lib/python3.10/dist-packages/torch/_tensor.py](https://localhost:8080/#) in __torch_function__(cls, func, types, args, kwargs)
1293
1294 with _C.DisableTorchFunctionSubclass():
-> 1295 ret = func(*args, **kwargs)
1296 if func in get_default_nowrap_functions():
1297 return ret
[/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py](https://localhost:8080/#) in mse_loss(input, target, size_average, reduce, reduction)
3292 reduction = _Reduction.legacy_get_string(size_average, reduce)
3293
-> 3294 expanded_input, expanded_target = torch.broadcast_tensors(input, target)
3295 return torch._C._nn.mse_loss(expanded_input, expanded_target, _Reduction.get_enum(reduction))
3296
[/usr/local/lib/python3.10/dist-packages/torch/functional.py](https://localhost:8080/#) in broadcast_tensors(*tensors)
72 if has_torch_function(tensors):
73 return handle_torch_function(broadcast_tensors, tensors, *tensors)
---> 74 return _VF.broadcast_tensors(tensors) # type: ignore[attr-defined]
75
76
The size of tensor a (336) must match the size of tensor b (16) at non-singleton dimension 0
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi, I tried PatchTST in colab :
It ran well till here:
But either lr_find or fit_one_cycle failed with following error. Where was I wrong and how to fix? Thanks.
Beta Was this translation helpful? Give feedback.
All reactions