Flip a Tensor

[nh9k]

Area Select

react-native-pytorch-core (core package)

Description

Hi, thanks for contributions.

How can i flip a tensor like torch.flip?
There is no implementation of flip function in Playtorch docs

Another solution i thought is to make a new tensor and assign reversed values using for loop such as the tensor[::, ::-1] .
However, i got a result Cannot assign to property ~
My code and the corresponding error are as follows.

CODE: e.g., (.js file)

  let tempTensor = torch.zeros(1);
  console.log('temp value', tempTensor.data());
  console.log('temp shape', tempTensor.shape);
  tempTensor[0].data = 5; //or tempTensor[0]
  console.log('result', tempTensor.data());

ERROR: e.g.,

 LOG  temp value [0]
 LOG  temp shape [1]
 WARN  Possible Unhandled Promise Rejection (id: 10):
Error: Exception in HostObject::set for prop 'data': TypeError: Cannot assign to property 'data' on HostObject with default setter

How can i solve this problem? Many thanks for your help :)

[raedle]

Hi @nh9k, thanks for reporting the need for this operation!

It's correct, there is no implementation for torch.flip or Tensor.flip operation in the PlayTorch SDK for version 0.2.0. I submitted a PR #101, which will add this operation. When the PR is reviewed and merged, it will be available in the following react-native-pytorch-core@nightly.

In the meantime, if an operation is TorchScriptable, it is possible to build these operations into the model. Have a look at the following example:

import torch
import torch.nn as nn

class ModelWrapper(nn.Module):
    def __init__(self, model):
      super().__init__()
      self.model = model

    def forward(self, x: torch.Tensor) -> torch.Tensor:
      x = x.flip([0, 1])
      return self.model(x)

wrapped_model = ModelWrapper(model)

And then export the model for the lite interpreter runtime as usual

from torch.utils.mobile_optimizer import optimize_for_mobile

scripted_model = torch.jit.script(wrapped_model)
optimized_model = optimize_for_mobile(scripted_model)
optimized_model._save_for_lite_interpreter("wrapped_model.ptl")

Hope that helps!

[raedle]

@nh9k, we merged #101 and it is available through the react-native-pytorch-core@nightly version.

Are you using the PlayTorch app and Expo Snack or are you developing in a local React Native app?

Note: if you are using the PlayTorch app, then Tensor.flip will be available with the next planned app release

[raedle]

@nh9k, there is another option via an exported model to polyfill the torch.flip operation until the next release of PlayTorch brings support for Tensor.flip.

The example Snack demonstrates the option: https://playtorch.dev/snack/@raedle/flip-polyfill

Basically, it wraps the torch.flip operation in an torch.nn.Module and exports it as a lite interpreter model:

import torch
import torch.nn as nn
from typing import List
from torch.utils.mobile_optimizer import optimize_for_mobile

class Flip(nn.Module):
    def __init__(self):
      super().__init__()

    def forward(self, x: torch.Tensor, dims: List[int]) -> torch.Tensor:
      return torch.flip(x, dims)

flip = Flip()

scripted_model = torch.jit.script(flip)
optimized_model = optimize_for_mobile(scripted_model)
optimized_model._save_for_lite_interpreter("flip.ptl")

The exported flip.ptl lite interpreter model can be loaded with PlayTorch (even in the PlayTorch app) and used like the torch.flip operation.

const filePath = await MobileModel.download(require('./flip.ptl'));
const flipModule = await torch.jit._loadForMobile(filePath);

In React Native, the flip operation is loaded via the usePolyfillFlip hook for convenience:

const flip = usePolyfillFlip();

Then, flip can be used to flip a tensor:

let tensor = torch.arange(8).reshape([2, 2, 2]);
const flippedTensor = flip(tensor, [1, 0]);

Hope that unblocks you until PlayTorch comes natively with Tensor.flip!

[nh9k]

Hi! @raedle, Thank you so much your kindness.

I am developing in a local React Native app, and your idea which is building flip operation into the model is very good.
Thank you for your suggestion!
In my case, our models need to flip image once more before inference, because the image on the app display already is fliped.
I will use one among the your solutions.

1. released nightly version
2. torchscriptable operation
3. your flip-polyfill method

I hope that the selected method will be efficient.
Additionaly, I would like to know how to use the nigthly version.

[raedle]

@nh9k, do I understand correctly that flip is required to flip the output image before rendering it on screen?

Option 1 and 2 sound reasonable if you have control over the model or you can upgrade react-native-pytorch-core to the nightly version. In that case, it's mostly a preference where to execute the flip, and if you don't want to change the model, then using the nightly should get you the expected result.

You can add the latest nightly for your local React Native app with the following command:

cd <YOUR_LOCAL_REACT_NATIVE_APP_DIRECTORY>
yarn add react-native-pytorch-core@nightly

After executing the command, you should be able to use flip as follows:

Example:

let tensor = torch.arange(8).reshape([2, 2, 2]);
const flippedTensor = flip(tensor, [1, 0]);

Depending on the image format and the flip axis, you may want to choose the width or height axis to flip.

Please let us know if this is successful, and also please do share what you built with PlayTorch, if you can 😉!

[nh9k]

@raedle, Thank you so much 😄
My app gets a fliped image, so i have to flip image before our models infer this image.
In detail, my app will use tensor.permute ([2,1,0]) and flip, because it receives images rotated 90 degrees.
I will report back if I have any questions or if this problem is solved.

[nh9k]

Hi @raedle, I solved the problem!
I just switched to this code.

const flippedTensor = tensor.flip([1,0]);

Thank you :)

@nh9k, do I understand correctly that flip is required to flip the output image before rendering it on screen?

Option 1 and 2 sound reasonable if you have control over the model or you can upgrade react-native-pytorch-core to the nightly version. In that case, it's mostly a preference where to execute the flip, and if you don't want to change the model, then using the nightly should get you the expected result.

You can add the latest nightly for your local React Native app with the following command:

cd <YOUR_LOCAL_REACT_NATIVE_APP_DIRECTORY>
yarn add react-native-pytorch-core@nightly

After executing the command, you should be able to use flip as follows:

Example:

let tensor = torch.arange(8).reshape([2, 2, 2]);
const flippedTensor = flip(tensor, [1, 0]);

Depending on the image format and the flip axis, you may want to choose the width or height axis to flip.

Please let us know if this is successful, and also please do share what you built with PlayTorch, if you can 😉!

[raedle]

Fantastic to hear, @nh9k! I'll close the issue, then