Skip to content


Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP


WireFormat fails hard when writing long strings #34

merged 1 commit into from

2 participants


Due to the use of writeUTF, writing intermediate strings longer than 64k throw a UTFDataFormatException and fail hard. This is documented in the Java API:

@espringe espringe merged commit b58d1f9 into NICTA:master

Nice catch! Thanks very much for providing code to fix this too!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
Showing with 9 additions and 2 deletions.
  1. +9 −2 src/main/scala/com/nicta/scoobi/WireFormat.scala
11 src/main/scala/com/nicta/scoobi/WireFormat.scala
@@ -559,9 +559,16 @@ object WireFormat {
implicit def StringFmt = new WireFormat[String] {
def toWire(x: String, out: DataOutput) {
require(x != null, "Error, trying to serialize a null String. Consider using an empty string or Option[String]")
- out.writeUTF(x)
+ val b = x.getBytes("utf-8")
+ out.writeInt(b.length)
+ out.write(b)
+ }
+ def fromWire(in: DataInput): String = {
+ val l = in.readInt()
+ val b = new Array[Byte](l)
+ in.readFully(b, 0, l)
+ new String(b, "utf-8")
- def fromWire(in: DataInput): String = in.readUTF()
def show(x: String) = x.toString
Something went wrong with that request. Please try again.