[pytorch] [java/scala] new version remove Module class convert to layerImpl ,cause the layerImpl cannot covert to Module ! #1393

mullerhai · 2023-07-28T12:29:15Z

HI ,
when I use new version pytorch 2.0.1 in scala 2.11.10, meet the class convert error, the example java code I rewrite in scala, but it cannot work ,because in javacpp-pytorch new version ,Module.class has remove all the layerImpl convert to Module ,use register_module method, why remove them? now the error is

Exception in thread "main" java.lang.ClassCastException: class org.bytedeco.pytorch.Module cannot be cast to class org.bytedeco.pytorch.LinearImpl (org.bytedeco.pytorch.Module and org.bytedeco.pytorch.LinearImpl are in unnamed module of loader 'app')
	at SimpleMNIST$Net.<init>(hell.scala:23)
	at SimpleMNIST$.main(hell.scala:52)
	at SimpleMNIST.main(hell.scala)

how to solve that error ,do I need import some method sugar dependency in scala code?
if I scala code remove asInstanceOf[LinearImpl] these code ,the scala code cannot compile, Please help me ,thanks
dependency:

ThisBuild / version := "0.1.0-SNAPSHOT"

ThisBuild / scalaVersion := "2.12.10"

lazy val root = (project in file("."))
  .settings(
    name := "torchSa"
  )


scalaVersion := "2.12.10"

//idePackagePrefix := Some("org.example")
resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"

val sparkVersion = "3.1.1"

//libraryDependencies ++= Seq(
//  "org.apache.spark" %% "spark-core" % sparkVersion,
//  "org.apache.spark" %% "spark-sql" % sparkVersion,
//  "org.apache.spark" %% "spark-mllib" % sparkVersion,
//  "org.apache.spark" %% "spark-streaming" % sparkVersion
//)
// https://mvnrepository.com/artifact/org.apache.parquet/parquet-common
libraryDependencies += "org.apache.parquet" % "parquet-common" % "1.12.3"

libraryDependencies += "org.bytedeco" % "pytorch" %  "2.0.1-1.5.10-SNAPSHOT" // "1.12.1-1.5.8" // "1.10.2-1.5.7"
// https://mvnrepository.com/artifact/org.bytedeco/pytorch-platform
libraryDependencies += "org.bytedeco" % "pytorch-platform" % "2.0.1-1.5.10-SNAPSHOT"  //  "1.12.1-1.5.8" //"1.10.2-1.5.7"
//libraryDependencies += "org.bytedeco" % "pytorch-platform-gpu" %  "2.0.1-1.5.10-SNAPSHOT" // "1.12.1-1.5.8" // "1.10.2-1.5.7"
//// https://mvnrepository.com/artifact/org.bytedeco/pytorch-platform

libraryDependencies += "org.bytedeco" % "mkl-platform-redist" % "2023.1-1.5.10-SNAPSHOT"  //  "1.12.1-1.5.8" //"1.10.2-1.5.7"
//

code : convert the example java code to scala

import org.bytedeco.javacpp._
import org.bytedeco.pytorch._
import org.bytedeco.pytorch.Module
import org.bytedeco.pytorch.global.torch._
import java.io.File
import scala.collection.mutable.ListBuffer
import scala.io.Source
object SimpleMNIST { // Define a new Module. :LinearImpl :LinearImpl=
  class Net() extends Module { // Construct and register two Linear submodules.
    //fc1 = register_module("fc1", new LinearImpl(784, 64));
    var fc1 = register_module("fc1", new LinearImpl(784, 64)).asInstanceOf[LinearImpl]
    var  fc2 = register_module("fc2", new LinearImpl(64, 32)).asInstanceOf[LinearImpl]
    var  fc3 = register_module("fc3", new LinearImpl(32, 10)).asInstanceOf[LinearImpl]

    // Implement the Net's algorithm.
    def forward(xl: Tensor): Tensor = { // Use one of many tensor manipulation functions.
      var x = xl
      x = relu(fc1.forward(x.reshape(x.size(0), 784)))
      x = dropout(x,  0.5,  is_training)
      x = relu(fc2.asInstanceOf[LinearImpl].forward(x))
      x = log_softmax(fc3.asInstanceOf[LinearImpl].forward(x),  1)
      x
    }

//     Use one of many "standard library" modules.
//    var fc1: LinearImpl = null
//    var fc2: LinearImpl = null
//    var fc3: LinearImpl = null
  }

  @throws[Exception]
  def main(args: Array[String]): Unit = {
    /* try to use MKL when available */
    System.setProperty("org.bytedeco.openblas.load", "mkl")

    // Create a multi-threaded data loader for the MNIST dataset.
    val data_set = new MNIST("./data").map(new ExampleStack)
    val data_loader = new MNISTRandomDataLoader(data_set, new RandomSampler(data_set.size.get), new DataLoaderOptions(/*batch_size=*/ 64))
    // Create a new Net.
    val net = new SimpleMNIST.Net
    // Instantiate an SGD optimization algorithm to update our Net's parameters.
    val optimizer = new SGD(net.parameters, new SGDOptions(/*lr=*/ 0.01))
    for (epoch <- 1 to 10) {
      var batch_index = 0
      // Iterate the data loader to yield batches from the dataset.
      var it = data_loader.begin
      while ( {
        !(it == data_loader.end)
      }) {
        val batch = it.access
        // Reset gradients.
        optimizer.zero_grad()
        // Execute the model on the input data.
        val prediction = net.forward(batch.data)
        // Compute a loss value to judge the prediction of our model.
        val loss = nll_loss(prediction, batch.target)
        // Compute gradients of the loss w.r.t. the parameters of our model.
        loss.backward()
        // Update the parameters based on the calculated gradients.
        optimizer.step
        // Output the loss and checkpoint every 100 batches.
        if ( {
          batch_index += 1; batch_index
        } % 100 == 0) {
          System.out.println("Epoch: " + epoch + " | Batch: " + batch_index + " | Loss: " + loss.item_float)
          // Serialize your model periodically as a checkpoint.
          val archive = new OutputArchive
          net.save(archive)
          archive.save_to("net.pt")
        }

        it = it.increment
      }
    }
  }
}

The text was updated successfully, but these errors were encountered:

HGuillemet · 2023-07-28T13:13:24Z

The prototype of the register_module method is:

 public Module register_module(String name, Module module);

In previous version, there was, in addition, specialized methods like:

 public LinearImpl register_module(String name, LinearImpl module);

but these were a workaround for a bug that has been fixed and they are not needed anymore.
You can now write:

LinearImpl fc1 = new LinearImpl(784, 64);
register_module("fc1", fc1);

mullerhai · 2023-07-28T13:28:05Z

The prototype of the register_module method is:
 public Module register_module(String name, Module module);
In previous version, there was, in addition, specialized methods like:
 public LinearImpl register_module(String name, LinearImpl module);
but these were a workaround for a bug that has been fixed and they are not needed anymore. You can now write:
LinearImpl fc1 = new LinearImpl(784, 64);
register_module("fc1", fc1);

It is work fine ,but now I has another question, the new version ,we has implement the SequentialImpl.class, for the normal layer in pytorch. we can easy put or push_back to the SequentialImpl, It is can work . but now I need to user_defined layer or model list want to add to the SequentialImpl ,but it is is not work.
for example

    val seqs = new SequentialImpl()
    var fc4= new LinearImpl(784, 64)
    var fc5 = new LinearImpl(64, 32)
    var fc6 = new LinearImpl(32, 10)
    seqs.push_back(fc4)
    seqs.push_back(fc5)
    seqs.push_back(fc6)

it can work

but next user_defined model add to SequentialImpl cannot work

  class HelenLayer() extends Module { // Construct and register two Linear submodules.

    var fc1= new LinearImpl(784, 64)
    register_module("fc1",fc1)
    var fc2 = new LinearImpl(64, 32)
    register_module("fc2",fc2)
    var fc3 = new LinearImpl(32, 10)
    register_module("fc3",fc3)

    // Implement the Net's algorithm.
    def forward(xl: Tensor): Tensor = { // Use one of many tensor manipulation functions.
      var x = xl
      x = relu(fc1.forward(x.reshape(x.size(0), 784)))
      x = dropout(x,  0.5,  is_training)
      x = relu(fc2.forward(x))
      x = log_softmax(fc3.forward(x),  1)
      x
    }

  }

val  layerSeqs = new SequentialImpl()

val helenLayer = new HelenLayer()
layerSeqs.push_back(helenLayer)  // compiler error
layerSeqs.put(helenLayer)  //  meet  jvm error

so how to use SequentialImpl for a list of user_defined layer or model? thanks

mullerhai · 2023-07-28T14:45:08Z

Now I has implement user defined layer use SequentialImple named SeqNow like our example code Net Model simpleMnist, the code can running ,but cannot decrease the loss, I don't know why.

  class SeqNow() extends Module {
    var seqs = new SequentialImpl()
    var fc4 = new LinearImpl(784, 64)
    var relu = new ReLUImpl()
    val dropOpt = new DropoutOptions()
    var drop = new DropoutImpl(0.5)
    var fc5 = new LinearImpl(64, 32)
    var relu2 = new ReLUImpl()
    var fc6 = new LinearImpl(32, 10)
    val log_softmax = new LogSoftmaxImpl(1)
    seqs.push_back(fc4)
    seqs.push_back(relu)
    seqs.push_back(drop)
    seqs.push_back(fc5)
    seqs.push_back(relu2)
    seqs.push_back(fc6)
    seqs.push_back(log_softmax)
    def forward(xl: Tensor): Tensor = {
      var x = xl.reshape(xl.size(0), 784)
      x = seqs.forward(x)
      x
    }
  }
  class Net() extends Module { // Construct and register two Linear submodules.

    var fc1 = new LinearImpl(784, 64)
    register_module("fc1", fc1)
    var fc2 = new LinearImpl(64, 32)
    register_module("fc2", fc2)
    var fc3 = new LinearImpl(32, 10)
    register_module("fc3", fc3)

    // Implement the Net's algorithm.
    def forward(xl: Tensor): Tensor = { // Use one of many tensor manipulation functions.
      var x = xl
      x = relu(fc1.forward(x.reshape(x.size(0), 784)))
      x = dropout(x, 0.5, is_training)
      x = relu(fc2.forward(x))
      x = log_softmax(fc3.forward(x), 1)
      x
    }
  }

  @throws[Exception]
  def main(args: Array[String]): Unit = {
    /* try to use MKL when available */
    System.setProperty("org.bytedeco.openblas.load", "mkl")
    // Create a new Net.
    val net = new SimpleMNIST.Net
    val seqs = new SequentialImpl()
    val seqNow = new SimpleMNIST.SeqNow()
    // Create a multi-threaded data loader for the MNIST dataset.
    val data_set = new MNIST("./data").map(new ExampleStack)
    val data_loader = new MNISTRandomDataLoader(data_set, new RandomSampler(data_set.size.get), new DataLoaderOptions(/*batch_size=*/ 32))

    // Instantiate an SGD optimization algorithm to update our Net's parameters.
    val optimizer = new SGD(seqNow.parameters, new SGDOptions(/*lr=*/ 0.01))
    //    val optimizer = new SGD(net.parameters, new SGDOptions(/*lr=*/ 0.01))
    for (epoch <- 1 to 10) {
      var batch_index = 0
      // Iterate the data loader to yield batches from the dataset.
      var it = data_loader.begin
      while ( {
        !it.equals(data_loader.end)
      }){
      //        while ( {
      //        !(it == data_loader.end)
      //      }) {
        val batch = it.access
        // Reset gradients.
        optimizer.zero_grad()
        // Execute the model on the input data.
        //        val prediction = net.forward(batch.data)
        val prediction = seqNow.forward(batch.data)
        // Compute a loss value to judge the prediction of our model.
        val loss = nll_loss(prediction, batch.target)
        // Compute gradients of the loss w.r.t. the parameters of our model.
        loss.backward()
        // Update the parameters based on the calculated gradients.
        optimizer.step
        // Output the loss and checkpoint every 100 batches.
        if ( {
          batch_index += 1;
          batch_index
        } % 100 == 0) {
          System.out.println("Epoch: " + epoch + " | Batch: " + batch_index + " | Loss: " + loss.item_float)
          // Serialize your model periodically as a checkpoint.
          val archive = new OutputArchive
          //          net.save(archive)
          archive.save_to("net.pt")
        }

        it = it.increment
      }
    }
  }

···

mullerhai · 2023-07-28T14:48:46Z

for the net console
···

···

HGuillemet · 2023-07-28T14:56:55Z

so how to use SequentialImpl for a list of user_defined layer or model?

You cannot. There is no way to have libtorch call a forward method implemented in Java.
That's why I already advise you several times to use a Java alternative to Sequential like this or this.

HGuillemet · 2023-07-28T15:06:27Z

Your SeqNow module has no parameters to optimize and no sub-module.
You should either directly pass the SequentialImpl module to the optimizer, or register_module("seqs", seqs) in SeqNow constructor.

mullerhai · 2023-07-28T15:16:59Z

thanks, add i use the same way solve it
···
class SeqNow() extends Module {
var seqs = new SequentialImpl()
var fc4 = new LinearImpl(784, 64)

var relu = new ReLUImpl()
val dropOpt = new DropoutOptions()
var drop = new DropoutImpl(0.55)
drop.train(true)
var fc5 = new LinearImpl(64, 32)
var relu2 = new ReLUImpl()
var fc6 = new LinearImpl(32, 10)
val log_softmax = new LogSoftmaxImpl(1)
register_module("fc4",fc4 )
register_module("relu",relu )
register_module("drop",drop )
register_module("fc5",fc5 )
register_module("relu2",relu2 )
register_module("fc6",fc6 )
register_module("log_softmax",log_softmax )
seqs.push_back(fc4)
seqs.push_back(relu)
seqs.push_back(drop)
seqs.push_back(fc5)
seqs.push_back(relu2)
seqs.push_back(fc6)
seqs.push_back(log_softmax)
def forward(xl: Tensor): Tensor = {
  var x = xl.reshape(xl.size(0), 784)
  x = seqs.forward(x)
  x
}

}

···

HGuillemet · 2023-07-28T15:21:08Z

You just need to register seqs. fc4, relu, ... are already submodules of seqs after the push_back.

mullerhai · 2023-07-28T15:22:40Z

register_module("seqs", seqs)

I want to use the reshape layer or view layer in SequentialImpl ,but not found , so I want to know if we have the method to add reshape layer in model middle postion?

mullerhai · 2023-07-28T15:23:36Z

You just need to register seqs. fc4, relu, ... are already submodules of seqs after the push_back.

yes ,you say is correct

mullerhai · 2023-07-28T15:35:29Z

by the way , in the new version ,does ModuleListImpl ModuleDictImpl could perfect implement for really use in coding? thanks

HGuillemet · 2023-07-28T15:37:33Z

I want to use the reshape layer or view layer in SequentialImpl ,but not found , so I want to know if we have the method to add reshape layer in model middle postion?

There is not such modules in libtorch. You'll have to do the reshape in a forward method of your own, like you did above.

HGuillemet · 2023-07-28T15:40:26Z

by the way , in the new version ,does ModuleListImpl ModuleDictImpl could perfect implement for really use in coding? thanks

No tested, but I think it's usable.

mullerhai · 2023-07-28T15:46:13Z

by the way , in the new version ,does ModuleListImpl ModuleDictImpl could perfect implement for really use in coding? thanks

No tested, but I think it's usable.

next day I will test them ，then give you answer

mullerhai · 2023-07-29T02:26:22Z

by the way , in the new version ,does ModuleListImpl ModuleDictImpl could perfect implement for really use in coding? thanks

No tested, but I think it's usable.

HI @HGuillemet , In my opinoin, the ModuleListImpl ModuleDictImpl two class could usable only a part not all,
first them could organize the layer , I just print them layer constructor

JavaCPP_torch_0003a_0003ann_0003a_0003aModule(
  (fc4): torch::nn::Linear(in_features=784, out_features=64, bias=true)
  (relu): torch::nn::ReLU()
  (drop): torch::nn::Dropout(p=0.55, inplace=false)
  (fc5): torch::nn::Linear(in_features=64, out_features=32, bias=true)
  (relu2): torch::nn::ReLU()
  (fc6): torch::nn::Linear(in_features=32, out_features=10, bias=true)
  (log_softmax): torch::nn::LogSoftmax(dim=1)
  (seqs): torch::nn::ModuleList(
    (0): torch::nn::Linear(in_features=784, out_features=64, bias=true)
    (1): torch::nn::ReLU()
    (2): torch::nn::Dropout(p=0.55, inplace=false)
    (3): torch::nn::Linear(in_features=64, out_features=32, bias=true)
    (4): torch::nn::ReLU()
    (5): torch::nn::Linear(in_features=32, out_features=10, bias=true)
    (6): torch::nn::LogSoftmax(dim=1)
  )
)

JavaCPP_torch_0003a_0003ann_0003a_0003aModule(
  (seqs): torch::nn::ModuleDict(
    (fc4): torch::nn::Linear(in_features=784, out_features=64, bias=true)
    (relu): torch::nn::ReLU()
    (drop): torch::nn::Dropout(p=0.55, inplace=false)
    (fc5): torch::nn::Linear(in_features=64, out_features=32, bias=true)
    (relu2): torch::nn::ReLU()
    (fc6): torch::nn::Linear(in_features=32, out_features=10, bias=true)
    (log_softmax): torch::nn::LogSoftmax(dim=1)
  )
)

but we can not invoke the forward method ,because in the ModuleListImpl ModuleDictImpl container all element is Module class ,Module class doesn't have forward method no more, If I want to convert each element to it origin layer type, also meet error , I don't know how to forward them element layer in ModuleListImpl ModuleDictImpl , if convinient please give me some example to do this. thanks

···
class DictNow() extends Module {
var fc4 = new LinearImpl(784, 64)
var relu = new ReLUImpl()
val dropOpt = new DropoutOptions()
var drop = new DropoutImpl(0.55)
drop.train(true)
var fc5 = new LinearImpl(64, 32)
var relu2 = new ReLUImpl()
var fc6 = new LinearImpl(32, 10)
val log_softmax = new LogSoftmaxImpl(1)
// var vector = new StringSharedModuleVector()
var arrayName= ArrayString
var arrayModule= ArrayModule
var subDict = new StringSharedModuleDict()
subDict.insert("fc4",fc4)
subDict.insert("relu",relu)
subDict.insert("drop",drop)
subDict.insert("fc5",fc5)
subDict.insert("relu2",relu2)
subDict.insert("fc6",fc6)
subDict.insert("log_softmax",log_softmax)

var seqs = new ModuleDictImpl(subDict)
register_module("seqs", seqs)
import org.bytedeco.pytorch.functions._
def forward(xl: Tensor): Tensor = {
  var x = xl.reshape(xl.size(0), 784)
  //      x = seqs.forward(x)
  arrayName.foreach(ele =>{
   x= seqs.get(ele).asInstanceOf[AnyModule].forward(x)
  })

// var count = 0
// var it = seqs.begin
// while ( {
// !it.equals(seqs.end)
// }){
//// seqs.get(1).apply(NamedModuleApplyFunction)
// x = seqs.get(count).asInstanceOf[AnyModule].forward(x)
// count +=1
// }
x
}
}

class ListNow() extends Module {
var seqs = new ModuleListImpl()
var fc4 = new LinearImpl(784, 64)
var relu = new ReLUImpl()
val dropOpt = new DropoutOptions()
var drop = new DropoutImpl(0.55)
drop.train(true)
var fc5 = new LinearImpl(64, 32)
var relu2 = new ReLUImpl()
var fc6 = new LinearImpl(32, 10)
val log_softmax = new LogSoftmaxImpl(1)

    register_module("fc4",fc4 )
    register_module("relu",relu )
    register_module("drop",drop )
    register_module("fc5",fc5 )
    register_module("relu2",relu2 )
    register_module("fc6",fc6 )
    register_module("log_softmax",log_softmax )
register_module("seqs", seqs)
seqs.push_back(fc4)
seqs.push_back(relu)
seqs.push_back(drop)
seqs.push_back(fc5)
seqs.push_back(relu2)
seqs.push_back(fc6)
seqs.push_back(log_softmax)
var arrayName= Array[String]("fc4","relu","drop","fc5","relu2","fc6","log_softmax")
var arrayModule= Array[Module](fc4,relu,drop,fc5,relu2,fc6,log_softmax)
def forward(xl: Tensor): Tensor = {
  var x = xl.reshape(xl.size(0), 784)
  var count = 0
  arrayModule.foreach(ele =>{
    val cla = ele.asInstanceOf[AnyRef].getClass

    println(s"count ${count}")
    val module = seqs.get(count)
    x =module.asInstanceOf[LinearImpl].forward(x)
    count +=1
    println(s"count2 ${count}")
  })

// x = seqs.forward(x)
// var it = seqs.begin
// while ( {
// !it.equals(seqs.end)
// }){
//// seqs.get(1).apply(NamedModuleApplyFunction)
// }
x
}
}

···

mullerhai · 2023-07-29T03:01:58Z

I also found some convertion make me confuse, example like LinearImpl could convert to Module,but Module can not convert to LinearImpl
Exception in thread "main" java.lang.ClassCastException: class org.bytedeco.pytorch.Module cannot be cast to class org.bytedeco.pytorch.LinearImpl (org.bytedeco.pytorch.Module and org.bytedeco.pytorch.LinearImpl are in unnamed module of loader 'app')
at SimpleMNIST$ListNow.$anonfun$forward$2(hell.scala:121)

mullerhai · 2023-07-29T03:12:48Z

but if in ModuleList ModuleDict forward method write code like that ,them can run perfectly.


    def forward(xl: Tensor): Tensor = {
      var x = xl.reshape(xl.size(0), 784)
      x = relu.forward(fc4.forward(x.reshape(x.size(0), 784)))
      x = drop.forward(x)
      x = relu2.forward(fc5.forward(x))
      x = log_softmax.forward(fc6.forward(x))
      x
    }

but as you know ,we want to foreach module list or dict layer element like python pytorch yield to invoke each layer forward method,it will very easy to write

mullerhai · 2023-07-29T03:14:20Z

so most important,how to foreach module list or dict layer element like python pytorch yield to invoke each layer forward method for ModuleList ModuleDict,

mullerhai · 2023-07-29T04:19:50Z

like you see ,in python ,we use ModuleList for a range Layer, like that

class AutomaticFeatureInteractionModel(torch.nn.Module):
    """
    A pytorch implementation of AutoInt.

    Reference:
        W Song, et al. AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks, 2018.
    """

    def __init__(self, field_dims, embed_dim, atten_embed_dim, num_heads, num_layers, mlp_dims, dropouts, has_residual=True):
        super().__init__()
       self.self_attns = torch.nn.ModuleList([
            torch.nn.MultiheadAttention(atten_embed_dim, num_heads, dropout=dropouts[0]) for _ in range(num_layers)
        ])


    def forward(self, x):
        """
        :param x: Long tensor of size ``(batch_size, num_fields)``
        """

        for self_attn in self.self_attns:
            cross_term, _ = self_attn(cross_term, cross_term, cross_term)

if in javacpp pytorch, how to coding that

HGuillemet · 2023-07-29T07:06:32Z

I also found some convertion make me confuse, example like LinearImpl could convert to Module,but Module can not convert to LinearImpl
Exception in thread "main" java.lang.ClassCastException: class org.bytedeco.pytorch.Module cannot be cast to class

There is no reason for this. I suspect this is a problem related to Scala.
Is it possible that you hit this bug ?

HGuillemet · 2023-07-29T08:03:49Z

Note that if you have a Java module, for instance:

LinearImpl linear = new LinearImpl(3,3);

you pass it to libtorch and get it back:

  ModuleListImpl list = new ModuleListImpl();
  list.push_back(linear);
  Module m = list.get(0);

Module m you are getting back is NOT linear. It's not even an instance of LinearImpl. This is an new instance of Java Module class that wraps the C++ Module instance returned by get:

      System.err.println(linear == m); // false
      System.err.println(linear.equals(m)); // false
      System.err.println(m instanceof LinearImpl); // false
      System.err.println(linear.address() == m.address()); // false
      System.err.println(linear.asModule() == m); // false
      System.err.println(linear.asModule().equals(m)); // true
      System.err.println(linear.asModule().address() == m.address()); // true

There is nothing we can do about this.

HGuillemet · 2023-07-29T08:07:33Z

Also even in C++, you cannot directly call forward on a items of ModuleList.

mullerhai · 2023-07-29T08:47:33Z

Note that if you have a Java module, for instance:

LinearImpl linear = new LinearImpl(3,3);

you pass it to libtorch and get it back:

  ModuleListImpl list = new ModuleListImpl();
  list.push_back(linear);
  Module m = list.get(0);

Module m you are getting back is NOT linear. It's not even an instance of LinearImpl. This is an new instance of Java Module class that wraps the C++ Module instance returned by get:

      System.err.println(linear == m); // false
      System.err.println(linear.equals(m)); // false
      System.err.println(m instanceof LinearImpl); // false
      System.err.println(linear.address() == m.address()); // false
      System.err.println(linear.asModule() == m); // false
      System.err.println(linear.asModule().equals(m)); // true
      System.err.println(linear.asModule().address() == m.address()); // true

There is nothing we can do about this.

oh , I think ，if the Module class just is java wrapper obj, and it cannot convert back to real layer obj to invoke forward method, so what the meanning for ModuleDict and ModuleList in pytorch? the Module need forward method or must has anyway to invoke element layer forward method , how do you think?

mullerhai · 2023-07-29T08:48:25Z

I also found some convertion make me confuse, example like LinearImpl could convert to Module,but Module can not convert to LinearImpl
Exception in thread "main" java.lang.ClassCastException: class org.bytedeco.pytorch.Module cannot be cast to class

There is no reason for this. I suspect this is a problem related to Scala. Is it possible that you hit this bug ?

I don't think so, maybe in java the Module cannot convert back to layer obj

mullerhai · 2023-07-29T09:03:28Z

give you one important example , when layer organize a array , they have some father class Module, then they lost forward method? can not image that bad things!!! do you think this rule is legal?


      var fc4 = new LinearImpl(784, 64)
      var relu = new ReLUImpl()
      val dropOpt = new DropoutOptions()
      var drop = new DropoutImpl(0.55)
      drop.train(true)
      var fc5 = new LinearImpl(64, 32)
      var relu2 = new ReLUImpl()
      var fc6 = new LinearImpl(32, 10)
      val log_softmax = new LogSoftmaxImpl(1)

 var arrayModule= Array(fc4,relu,drop,fc5,relu2,fc6,log_softmax)   // Module Array
      arrayModule.foreach(ele=> {
           ele.forward() // no  ,we need it !!!
          println(ele.asModule().equals(fc4.asModule()))
//          ele.asModule().asInstanceOf[LinearImpl]
        })

HGuillemet · 2023-07-29T13:12:01Z

We could probably add a constructor for native modules to downcast from Module (similarly to what must be done in C+):

  Module m = list.get(0);
  LinearImpl linear = new LinearImpl(m);
  Tensor output = linear.forward(input);

What do you think ?

mullerhai · 2023-07-29T13:36:58Z

We could probably add a constructor for native modules to downcast from Module (similarly to what must be done in C+):
  Module m = list.get(0);
  LinearImpl linear = new LinearImpl(m);
  Tensor output = linear.forward(input);
What do you think ?

I think we need only one step ,just add forward method to Module class like Sequential class , only that we could use Module array for layer forward method yield perfect,
···
Module m = list.get(0);
Tensor output = m.forward(input);
···

if we do not , I only use switch case judge Layer type then process it , it not grace .like this
···

class ModuleListYieldNow() extends Module {
var seqs = new ModuleListImpl()
var fc4 = new LinearImpl(784, 64)
var relu = new ReLUImpl()
val dropOpt = new DropoutOptions()
var drop = new DropoutImpl(0.55)
drop.train(true)
var fc5 = new LinearImpl(64, 32)
var relu2 = new ReLUImpl()
var fc6 = new LinearImpl(32, 10)
val log_softmax = new LogSoftmaxImpl(1)
register_module("seqs", seqs)
seqs.push_back(fc4)
seqs.push_back(relu)
seqs.push_back(drop)
seqs.push_back(fc5)
seqs.push_back(relu2)
seqs.push_back(fc6)
seqs.push_back(log_softmax)
var arrayName= ArrayString
var arrayModule= Array(fc4,relu,drop,fc5,relu2,fc6,log_softmax)
var tupleModule= Tuple7(fc4,relu,drop,fc5,relu2,fc6,log_softmax)
var arrayLayerClass = Tuple7( classOf[LinearImpl],classOf[ReLUImpl],classOf[DropoutImpl],classOf[LinearImpl],classOf[ReLUImpl],classOf[LinearImpl],classOf[LogSoftmaxImpl])
def forward(xl: Tensor): Tensor = {
var x = xl.reshape(xl.size(0), 784)
var cnt =1
val iter = arrayModule.iterator
// val iter = tupleModule.productIterator
// val iterClass = arrayLayerClass.productIterator
while(iter.hasNext){
val layer = iter.next()
layer match {
case layer:LinearImpl =>
x = layer.asInstanceOf[LinearImpl].forward(x)
case layer:ReLUImpl =>
x = layer.asInstanceOf[ReLUImpl].forward(x)
case layer:DropoutImpl=>
x = layer.asInstanceOf[DropoutImpl].forward(x)
case layer:LogSoftmaxImpl =>
x = layer.asInstanceOf[LogSoftmaxImpl].forward(x)
}
// if(layer.isInstanceOf[LinearImpl]){
// x = layer.asInstanceOf[LinearImpl].forward(x)
// }else if(layer.isInstanceOf[ReLUImpl]){
// x = layer.asInstanceOf[ReLUImpl].forward(x)
// }else if(layer.isInstanceOf[DropoutImpl]){
// x = layer.asInstanceOf[DropoutImpl].forward(x)
// }else{
// x = layer.asInstanceOf[LogSoftmaxImpl].forward(x)
// }
cnt +=1
}
x
}
}
···

HGuillemet · 2023-07-29T13:51:12Z

Yes, either you use some Java list or array of Module, and you can cast normally to subclasses, like you do here with the switch.
Or you want to use a torchlib structure like ModuleList, but then we cannot cast as normal, we need this extra step (new LinearImpl(m)).
Sequential is different since the C++ implementation knows about the classes of modules it contains and is able to chain the forward calls dynamically.

mullerhai · 2023-07-29T14:34:36Z

Yes, either you use some Java list or array of Module, and you can cast normally to subclasses, like you do here with the switch. Or you want to use a torchlib structure like ModuleList, but then we cannot cast as normal, we need this extra step (new LinearImpl(m)). Sequential is different since the C++ implementation knows about the classes of modules it contains and is able to chain the forward calls dynamically.

maybe hard for you to select the best way solve it, so now just follow you way like new LinearImpl(m) ,thanks

HGuillemet · 2023-07-29T14:53:00Z

The first option already works. So if it's enough please use it.
The second option with new LinearImpl(m) needs developments.

mullerhai · 2023-07-29T15:08:23Z

The first option already works. So if it's enough please use it. The second option with new LinearImpl(m) needs developments.

thanks , need the second option,

HGuillemet · 2023-09-06T14:47:39Z

This issue is finally addressed by PR bytedeco/javacpp#700
Once merged, you will be able to do:

  t2 = new LinearImpl(m).forward(t)

where m is an instance of Module returned by torchlib, for instance by:

m = list.get(0);

where list is a ModuleList. Of course m must be a Linear module.

mullerhai · 2023-09-06T15:30:35Z

This issue is finally addressed by PR bytedeco/javacpp#700 Once merged, you will be able to do:
  t2 = new LinearImpl(m).forward(t)
where m is an instance of Module returned by torchlib, for instance by:
m = list.get(0);
where list is a ModuleList. Of course m must be a Linear module.

perfect ,now javacpp pytorch is forward to the useable tools for java /scala /jvm,waiting for PR merge,thanks @HGuillemet

saudet · 2023-10-04T23:36:33Z

That pull request has been merged, so this should be working now!

mullerhai · 2023-10-09T16:07:09Z

That pull request has been merged, so this should be working now!

perfect，thanks,waiting 1.5.10 version release publish to mvn repos

saudet assigned HGuillemet Jul 28, 2023

saudet added enhancement help wanted question labels Aug 1, 2023

HGuillemet mentioned this issue Aug 5, 2023

Add of for downcasting when c-style cast cannot work bytedeco/javacpp#700

Merged

saudet removed the help wanted label Oct 4, 2023

mullerhai closed this as completed Oct 9, 2023

[pytorch] [java/scala] new version remove Module class convert to layerImpl ,cause the layerImpl cannot covert to Module ! #1393

[pytorch] [java/scala] new version remove Module class convert to layerImpl ,cause the layerImpl cannot covert to Module ! #1393

Comments

mullerhai commented Jul 28, 2023

HGuillemet commented Jul 28, 2023

mullerhai commented Jul 28, 2023

mullerhai commented Jul 28, 2023

mullerhai commented Jul 28, 2023

HGuillemet commented Jul 28, 2023

HGuillemet commented Jul 28, 2023

mullerhai commented Jul 28, 2023

HGuillemet commented Jul 28, 2023 • edited

mullerhai commented Jul 28, 2023

mullerhai commented Jul 28, 2023

mullerhai commented Jul 28, 2023

HGuillemet commented Jul 28, 2023

HGuillemet commented Jul 28, 2023

mullerhai commented Jul 28, 2023

mullerhai commented Jul 29, 2023

mullerhai commented Jul 29, 2023

mullerhai commented Jul 29, 2023

mullerhai commented Jul 29, 2023

mullerhai commented Jul 29, 2023

HGuillemet commented Jul 29, 2023

HGuillemet commented Jul 29, 2023

HGuillemet commented Jul 29, 2023

mullerhai commented Jul 29, 2023

mullerhai commented Jul 29, 2023

mullerhai commented Jul 29, 2023

HGuillemet commented Jul 29, 2023

mullerhai commented Jul 29, 2023

HGuillemet commented Jul 29, 2023

mullerhai commented Jul 29, 2023

HGuillemet commented Jul 29, 2023

mullerhai commented Jul 29, 2023

HGuillemet commented Sep 6, 2023

mullerhai commented Sep 6, 2023

saudet commented Oct 4, 2023

mullerhai commented Oct 9, 2023 • edited

HGuillemet commented Jul 28, 2023 •

edited

mullerhai commented Oct 9, 2023 •

edited