Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kyro throw buffer underFlow without input/output NOT closing, without multi-thread #373

Closed
wyzssw opened this issue Nov 23, 2015 · 7 comments

Comments

@wyzssw
Copy link

wyzssw commented Nov 23, 2015

this is my exception stack

Exception in thread "main" com.esotericsoftware.kryo.KryoException: Buffer underflow.
Serialization trace:
right (org.apache.commons.lang3.tuple.ImmutablePair)
backingMap (com.google.common.collect.HashBasedTable)
userTable (com.xxxx.ecar.item.model.ItemStates)
    at com.esotericsoftware.kryo.io.Input.require(Input.java:199)
    at com.esotericsoftware.kryo.io.Input.readAscii_slow(Input.java:616)
    at com.esotericsoftware.kryo.io.Input.readAscii(Input.java:594)
    at com.esotericsoftware.kryo.io.Input.readString(Input.java:472)
    at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:150)
    at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
    at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670)
    at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:118)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
    at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790)
    at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:153)
    at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
    at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
    at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:686)
    at com.xxxxx.ecar.item.rpc.serialize.KryoSerializers.readObject(KryoSerializers.java:61)
    at com.xxxxx.ecar.item.rpc.serialize.KryoSerializers.main(KryoSerializers.java:97)

this is my whole code

package com.xxxxx.serialize;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.io.OutputStream;

import org.apache.commons.lang3.tuple.ImmutablePair;
import org.objenesis.strategy.StdInstantiatorStrategy;

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;
import com.xxxx.ecar.item.model.ItemStates;

/**
 * 
 * @author 
 *
 */
public final class KryoSerializers {

  private KryoSerializers(){}

  private static ThreadLocal<Kryo> kryos = new ThreadLocal<Kryo>() {
    protected Kryo initialValue() {
        Kryo kryo = new Kryo();
        ((Kryo.DefaultInstantiatorStrategy) kryo.getInstantiatorStrategy()).setFallbackInstantiatorStrategy(new StdInstantiatorStrategy());
        return kryo;
    };
  };

  public static void register(Class<?> type){
    kryos.get().register(type);
  }

  /**
   * 
   * @param source
   * @param type
   * @return
   */
  public static <T> T readObject (InputStream source, Class<T> type) {
    register(type);
    Input input = new Input(source);
    T result =   kryos.get().readObject(input, type);
    return result;
  }

  /**
   * 
   * @param source
   * @param type
   * @return
   */
  public static <T> T readObject (byte[] source, Class<T> type) {
    ByteArrayInputStream is = new ByteArrayInputStream(source);
    Input input = new Input(is);
    T result =   kryos.get().readObject(input, type);
    input.close();
    return result;
  }

  /***
   * 
   * @param sink 
   * @param object
   */
  public static void writeObject (OutputStream sink, Object object) {
    Output output = new Output(sink);
    kryos.get().writeObject(output, object);
    output.flush();
  }

  /**
   * 
   * @param object
   * @return
   */
  public static byte[] getBytes(Object object){
    ByteArrayOutputStream os = new ByteArrayOutputStream();
    Output output = new Output(os);
    kryos.get().writeObject(output, object);
    output.close();
    return os.toByteArray();
  }


  public static void main(String[] args) {
    ItemStates states = ItemStates.create();
    states.getOrderTable().put(1L, 2, "coupon");
    states.getStockTable().put(1, ImmutablePair.of(1, 2), ImmutablePair.of(3, 4));
    states.getUserTable().put(ImmutablePair.of(1L, 2), 3L, 5);
    byte[] data  =  KryoSerializers.getBytes(states);
    System.out.println(KryoSerializers.readObject(data,ItemStates.class));
  }

}

and Itemstats.java

package com.xxxx.model;


import java.io.Serializable;

import org.apache.commons.lang3.tuple.Pair;

import com.google.common.collect.HashBasedTable;
import com.google.common.collect.Table;

/**
 *
 *
 * @author 
 */
public class ItemStates implements Serializable{

  private static final long serialVersionUID = -329529021217842031L;

  private Table<Integer, Pair<Integer,Integer>, Pair<Integer,Integer>> stockTable = HashBasedTable.create();

  private Table<Long, Integer,String> orderTable = HashBasedTable.create();

  private Table<Pair<Long,Integer>, Long,Integer> userTable = HashBasedTable.create();

  /**
   * 
   * <p>hashBasedTable [modelId,Pair[attr1Id,attr2id],Pair[itemId,stock]]
   * @return
   */
  public  Table<Integer, Pair<Integer,Integer>, Pair<Integer,Integer>> getStockTable(){
    return stockTable;
  }

  /**
   * 
   * <p>hashBasedTable [orderId,status,coupon]
   * @return
   */
  public Table<Long, Integer,String> getOrderTable(){
    return orderTable;
  }

  /**
   * 
   * <p>hashBasedTable [Pair[uderId,modelId],orderId,status]
   * @return
   */
  public  Table<Pair<Long,Integer>, Long,Integer> getUserTable(){
    return userTable;
  }



  @Override
  public String toString() {
    return "ItemStates [stockTable=" + stockTable + ", orderTable=" + orderTable + ", userTable="
        + userTable + "]";
  }

  public static  ItemStates  create(){
    return new ItemStates();
  }

}

@wyzssw
Copy link
Author

wyzssw commented Nov 23, 2015

@magro @NathanSweet Please help me ,please ,please

@leixinwei12
Copy link

我们现在也出现了这个原因,
是缓存对象已经改了,
各个子系统没全部更新,
结果读取序列化出来的对象跟写的对象不一致时导致。
希望能帮到后面的人。

@magro
Copy link
Collaborator

magro commented Apr 8, 2016

Sorry for the late response, is this still relevant?

@magro
Copy link
Collaborator

magro commented Jul 16, 2016

Closed for now, please reopen if still relevant.

@magro magro closed this as completed Jul 16, 2016
@wyzssw
Copy link
Author

wyzssw commented Jul 17, 2016

Is kryo new version resolve this bug?

@magro
Copy link
Collaborator

magro commented Jul 17, 2016

To be honest I don't know. Can you try this? If it doesn't work can you please provide a self contained test case that allows to reproduce the issue (see also http://sscce.org)?

@wyzssw
Copy link
Author

wyzssw commented Jul 18, 2016

After try my code,it works good,thanks.
Kryo has a defect which must register all class to guarantee serialization/deserialization work well.
So finally i turn to Fast-serialization instead

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants