Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

StringByteIterator toArray does not return the entire byte representation of the value #69

Closed
ShayHassidim opened this Issue Mar 4, 2012 · 4 comments

Comments

Projects
None yet
4 participants

GemFireClient convertToBytearrayMap method does not store the entire value

retVal.put(key, values.get(key).toArray());
should be:
retVal.put(key, values.get(key).toString().getBytes());

If you will run this in debug mode you will find only few bytes are actually stored in cache.
Shay

m1ch1 pushed a commit that referenced this issue Mar 4, 2012

Contributor

m1ch1 commented Mar 4, 2012

Hi Shay,

ByteIterator.toArray() and ByteIterator.toString().getBytes() should return the same byte array. Could you try something like this and see what ByteIterator.toArray() is returning?

  private Map<String, byte[]> convertToBytearrayMap(Map<String,ByteIterator> values) {
    Map<String, byte[]> retVal = new HashMap<String, byte[]>();
    for (String key : values.keySet()) {
      System.out.println("key: " + key + ", bytesLeft: " + values.get(key).bytesLeft());
      byte[] value = values.get(key).toArray();
      System.out.println("value size: " + value.length);
      System.out.println("key: " + key + ", bytesLeft: " + values.get(key).bytesLeft());
      retVal.put(key, value);
    }
    return retVal;             
  }     

Thanks!
--Michi

Contributor

westonplatter commented Mar 23, 2014

@ShayHassidim - also something that may impact gemfire is that we recently merged a gemfire change that fixed a bug, #123. It might be worth pulling a fresh copy of the codebase.

Collaborator

busbey commented Jun 6, 2015

I think this issue comes down to incorrect handling of String <-> byte[]. In addition to #279, StringByteIterator incorrectly uses the String's character length to determine how many bytes should be made and then incorrectly pulls each character and truncates it to a byte.

That means for Strings made up of non-ascii, toArray() will return a smaller array than if you immediately call toString().getBytes() (if you don't call toString immediately then you get the wrong answer still).

@busbey busbey changed the title from GemFireClient convertToBytearrayMap method does not store the entire value to StringByteIterator toArray does not return the entire byte representation of the value Jun 6, 2015

Collaborator

busbey commented Aug 4, 2015

Fixed by #288

@busbey busbey closed this Aug 4, 2015

jaricftw pushed a commit to jaricftw/YCSB that referenced this issue Jul 19, 2016

jaricftw pushed a commit to jaricftw/YCSB that referenced this issue Jul 19, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment