Save and load RandomForest

[YossiShasha]

Hi,
I would like to save and load the result of the RandomForest training.
When I used Java serialization and I got an Exception since getParam in the Parameter class creates Parameter Object that contains a Method field member (the getMethod variable is final and saved as part of the created Parameter object). The same happend when I used FST.
I also tried to use Kryo but got an Exception since RandomDecisionTree does not have no-arg Constructor.
I have millions of training data so I can't train it every time I want to use it, What should I do?

Thanks.

[EdwardRaff]

Please provide a code sample that causes the error and I'll look into it

Sent from my iPhone

On May 6, 2016, at 12:45 PM, YossiShasha notifications@github.com wrote:

Hi,
I would like to save and load the result of the RandomForest training.
When I used Java serialization and I got an Exception since getParam in the Parameter class creates Parameter Object that contains a Method field member (the getMethod variable is final and saved as part of the created Parameter object). The same happend when I used FST.
I also tried to use Kryo but got an Exception since RandomDecisionTree does not have no-arg Constructor.
I have millions of training data so I can't train it every time I want to use it, What should I do?

Thanks.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

[YossiShasha]

I wrote simple code for Kryo and FST:

 package tests;

 import com.esotericsoftware.kryo.Kryo;
 import com.esotericsoftware.kryo.io.Input;
 import com.esotericsoftware.kryo.io.Output;
 import de.ruedigermoeller.serialization.FSTObjectInput;
 import de.ruedigermoeller.serialization.FSTObjectOutput;
 import jsat.classifiers.trees.RandomForest;

 import java.io.*;

 /**
 * Created by Yossi on 03/05/2016.
  */
public class SaveRandomForest {
      public static void main(String[] args){
           RandomForest randomForest = new RandomForest();
           System.out.println(randomForest.getMaxForestSize());
           randomForest.setMaxForestSize(1000);
           System.out.println(randomForest.getMaxForestSize());
           try {
               Kryo kryo = new Kryo();
               Output output = new Output(new FileOutputStream("C://RandomForest.ser"));
               kryo.writeObject(output, randomForest);
               output.close();
               Input input = new Input(new FileInputStream("C://RandomForest.ser"));
               kryo.readObject(input, RandomForest.class);
               RandomForest savedRandomForest = kryoReadObject(kryo, input);
               input.close();
               System.out.println(randomForest.getMaxForestSize());
           } catch (Exception e) {
               e.printStackTrace();
           }

           try{
               ObjectOutputStream outputStream = new ObjectOutputStream(new        FileOutputStream("C://RandomForest.ser"));
               FSTObjectOutput out = new FSTObjectOutput(outputStream);
               out.writeObject(randomForest);
               out.close();
               ObjectInputStream inputStream = new ObjectInputStream(new FileInputStream("C://RandomForest.ser"));
               FSTObjectInput in = new FSTObjectInput(inputStream);
               RandomForest savedRandomForest = (RandomForest)in.readObject();
               in.close();
               System.out.println(randomForest.getMaxForestSize());
           } catch (Exception e) {
               e.printStackTrace();
           }
     }
}

Thanks!

[EdwardRaff]

Will this error with normal Java serialization? I don't intend on debuting kyro

Sent from my iPhone

On May 6, 2016, at 1:39 PM, YossiShasha notifications@github.com wrote:

I wrote simple code for Kryo and FST:

package tests;

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;
import de.ruedigermoeller.serialization.FSTObjectInput;
import de.ruedigermoeller.serialization.FSTObjectOutput;
import jsat.classifiers.trees.RandomForest;

import java.io.*;

/**

• Created by Yossi on 03/05/2016.
  */
  public class SaveRandomForest {
  public static void main(String[] args){
  RandomForest randomForest = new RandomForest();
  System.out.println(randomForest.getMaxForestSize());
  randomForest.setMaxForestSize(1000);
  System.out.println(randomForest.getMaxForestSize());
  try {
  Kryo kryo = new Kryo();
  Output output = new Output(new FileOutputStream("C://RandomForest.ser"));
  kryo.writeObject(output, randomForest);
  output.close();
  Input input = new Input(new FileInputStream("C://RandomForest.ser"));
  kryo.readObject(input, RandomForest.class);
  RandomForest savedRandomForest = kryoReadObject(kryo, input);
  input.close();
  System.out.println(randomForest.getMaxForestSize());
  } catch (Exception e) {
  e.printStackTrace();
  }

     try{
         ObjectOutputStream outputStream = new ObjectOutputStream(new        FileOutputStream("C://RandomForest.ser"));
         FSTObjectOutput out = new FSTObjectOutput(outputStream);
         out.writeObject(randomForest);
         out.close();
         ObjectInputStream inputStream = new ObjectInputStream(new FileInputStream("C://RandomForest.ser"));
         FSTObjectInput in = new FSTObjectInput(inputStream);
         RandomForest savedRandomForest = (RandomForest)in.readObject();
         in.close();
         System.out.println(randomForest.getMaxForestSize());
     } catch (Exception e) {
         e.printStackTrace();
     }
  

  }
  }
  Thanks!

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

[YossiShasha]

The only readon I tried different serialization than Java serialization is because Java serialization failed to serialize since Method object is not serializable and the Parameter object that you create as part of DecisionTree contains such object (Method object). So as I said, Kyro fails since RandomDecidionTree doesn't have no-arg Constructor and both regular Java serialization and FST fails while trying to write a Method object. I am not sure if Kyro can handle objects that are not marked with Serializable or not, maybe even if there were a no-arg Constructor to RandomDecisionTree it will fail while trying to write Method like Java and FST serialization.

[EdwardRaff]

Should be fixed, please confirm it works for Java serialization. I understand trying other serializers if something isn't working, but that makes it more difficult for me in resolving a bug report.

[YossiShasha]

I am using maven and new to this, and I don't know how to download the snapshot version :/
I saw that you suggested to add maven-snapshot-repo for the url tag but I am not sure what it means.

[YossiShasha]

I copied your source code to my project just to test it (yeah it is really ugly and wrong). I tried the serialization right now and it worked for Java, Kryo and FST serialization!

[EdwardRaff]

If you clone the project locally and build it, it should install the snapshot artifact into your local maven repo.

[YossiShasha]

Thank you very much for your really quick support!