Bayesian GRU

[Anusha851]

I get the following error for Bayesian GRU implementation.

---

ValueError Traceback (most recent call last)
in
9 labels=labels.to(device),
10 criterion=criterion,
---> 11 sample_nbr=3)
12 loss.backward()
13 optimizer.step()

/opt/conda/lib/python3.7/site-packages/blitz/utils/variational_estimator.py in sample_elbo(self, inputs, labels, criterion, sample_nbr, complexity_cost_weight)
63 loss = 0
64 for _ in range(sample_nbr):
---> 65 outputs = self(inputs)
66 loss += criterion(outputs, labels)
67 loss += self.nn_kl_divergence() * complexity_cost_weight

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
725 result = self._slow_forward(*input, **kwargs)
726 else:
--> 727 result = self.forward(*input, **kwargs)
728 for hook in itertools.chain(
729 _global_forward_hooks.values(),

in forward(self, encodings, hidden_states, sharpen_loss)
17
18 #pass the inputs to the model
---> 19 x,t = self.bGRU1(encodings,hidden_states = None, sharpen_loss = None)
20 x,t = self.bGRU2(x,t)
21 return x

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
725 result = self._slow_forward(*input, **kwargs)
726 else:
--> 727 result = self.forward(*input, **kwargs)
728 for hook in itertools.chain(
729 _global_forward_hooks.values(),

/opt/conda/lib/python3.7/site-packages/blitz/modules/gru_bayesian_layer.py in forward(self, x, hidden_states, sharpen_loss)
221 sharpen_loss = None
222
--> 223 return self.forward_(x, hidden_states, sharpen_loss)
224

/opt/conda/lib/python3.7/site-packages/blitz/modules/gru_bayesian_layer.py in forward_(self, x, hidden_states, sharpen_loss)
135
136 #Assumes x is of shape (batch, sequence, feature)
--> 137 bs, seq_sz, _ = x.size()
138 hidden_seq = []
139

ValueError: not enough values to unpack (expected 3, got 2)

For this line of code:

for i, (datapoints, labels) in enumerate(train_dataloader):
optimizer.zero_grad()

    loss = gru.sample_elbo(inputs=datapoints.to(device),
                       labels=labels.to(device),
                       criterion=criterion,
                       sample_nbr=3)

My datapoints have shape torch.Size([4, 768]). How am I expected to reshape it? Please advise. Thanks.

[donhauser]

You can add a new axis by using None as selector for your tensor/array.

Like this you can "prepend" an extra dimension to 2-D data in numpy and torch:

# datapoints have shape [4, 768]
datapoints = datapoints[None, :, :]
# datapoints now have shape [1, 4, 768]

By swapping None with :, you could also reshape to [4, 1, 768] or [4, 768, 1].
If you need to swap 4 and 768 for whatever reason, have a look at .transpose()

Source: https://sparrow.dev/adding-a-dimension-to-a-tensor-in-pytorch/

[Anusha851]

Thank you.