Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

WireFormat fails hard when writing long strings #34

Merged
merged 1 commit into from

2 participants

@stephenroller

Due to the use of writeUTF, writing intermediate strings longer than 64k throw a UTFDataFormatException and fail hard. This is documented in the Java API:
http://docs.oracle.com/javase/1.4.2/docs/api/java/io/DataOutput.html#writeUTF(java.lang.String)

@espringe espringe merged commit b58d1f9 into NICTA:master
@espringe

Nice catch! Thanks very much for providing code to fix this too!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
Showing with 9 additions and 2 deletions.
  1. +9 −2 src/main/scala/com/nicta/scoobi/WireFormat.scala
View
11 src/main/scala/com/nicta/scoobi/WireFormat.scala
@@ -559,9 +559,16 @@ object WireFormat {
implicit def StringFmt = new WireFormat[String] {
def toWire(x: String, out: DataOutput) {
require(x != null, "Error, trying to serialize a null String. Consider using an empty string or Option[String]")
- out.writeUTF(x)
+ val b = x.getBytes("utf-8")
+ out.writeInt(b.length)
+ out.write(b)
+ }
+ def fromWire(in: DataInput): String = {
+ val l = in.readInt()
+ val b = new Array[Byte](l)
+ in.readFully(b, 0, l)
+ new String(b, "utf-8")
}
- def fromWire(in: DataInput): String = in.readUTF()
def show(x: String) = x.toString
}
Something went wrong with that request. Please try again.