How to convert the [:,t,:] syntax of python language into C# language with high performance

[toolgood]

How to convert the [:,t,:] syntax of python language into C# language with high performance

source code

    def forward(self, input_data):
        input_weighted = torch.zeros(input_data.shape[0], self.T, self.input_size).to(device)
        input_encoded = torch.zeros(input_data.shape[0], self.T, self.hidden_size).to(device)

        # Eq. 8, parameters not in nn.Linear but to be learnt
        hidden = init_hidden(input_data, self.hidden_size).to(device)  # 1*batch_size*hidden_size
        cell = init_hidden(input_data, self.hidden_size).to(device)

        for t in range(self.T): 
            x = torch.cat((hidden.repeat(self.input_size, 1, 1).permute(1, 0, 2),
                           cell.repeat(self.input_size, 1, 1).permute(1, 0, 2),
                           input_data.permute(0, 2, 1)), dim=2)  # batch_size * input_size * (2*hidden_size + T)
            x = self.attn_linear(x.view(-1, self.hidden_size * 2 + self.T))  # (batch_size * input_size) * 1

            attn_weights = F.softmax(x.view(-1, self.input_size),dim=1) 

            weighted_input = torch.mul(attn_weights, input_data[:, t, :])  # (batch_size, input_size)

            self.lstm_layer.flatten_parameters()
            _, lstm_states = self.lstm_layer(weighted_input.unsqueeze(0), (hidden, cell))
            hidden = lstm_states[0]
            cell = lstm_states[1]

            # save output
            input_weighted[:, t, :] = weighted_input
            input_encoded[:, t, :] = hidden

        return input_weighted, input_encoded

And lstm does not support the flatten_parameters method.

[NiklasGustafsson]

How to convert the [:,t,:] syntax of python language into C# language with high performance

[..,t,..] if you're using .NET 5.0 and later. [(null,null),t,(null,null)] if you're using .NET Framework 4.7.2 and later.

And lstm does not support the flatten_parameters method.

Please file a separate issue report on this.

[NiklasGustafsson]

The flatten_parameters issue has been addressed in PR #636

[NiklasGustafsson]

I'm going to close this as having been answered. Please re-open if there is a bug of some sort.

[pkese]

@toolgood

If this is still relevant: input_data[:, t, :] can also be written as input_data.select(1,t).

https://pytorch.org/docs/stable/generated/torch.select.html

[toolgood]

@NiklasGustafsson @pkese
thank you.
I'm not familiar with the usage of Python. The method of Python feels very strange and can't be converted into c# language.

_mask[None, None, :] written as _mask[null, null, (null, null)] , is error.

_mask[None, None, :] written as _mask[(null, null), (null, null), (null, null)] ,

Grammar doesn't report errors, but intuition is wrong.

source code：

class ProbMask():
    def __init__(self, B, H, L, index, scores, device="cpu"):
        _mask = torch.ones(L, scores.shape[-1], dtype=torch.bool).to(device).triu(1)
        _mask_ex = _mask[None, None, :].expand(B, H, L, scores.shape[-1])
        indicator = _mask_ex[torch.arange(B)[:, None, None],
                             torch.arange(H)[None, :, None],
                             index, :].to(device)
        self._mask = indicator.view(scores.shape).to(device)
    
    @property
    def mask(self):
        return self._mask