When trying to use XMLEventReader with an utf-8 encoded XML document that starts with a byte order mark (EF BB BF), an empty iterator is returned, and an error message is printed to stderr

[bbary]

When trying to use XMLEventReader with an utf-8 encoded XML document that starts with a byte order mark (EF BB BF), an empty iterator is returned, and an error message is printed to stderr.
test.scala
import scala.io.Source
import scala.xml.pull.XMLEventReader

val t = new XMLEventReader(Source.fromFile("hasBOM.xml"))

println(t)
Output of scala test.scala
file:/tmp/hasBOM.xml:1:1: < expected^
empty iterator

The attached file hasBOM.xml has CR+LF newlines, but LF or CR newlines do not change the behaviour.

https://drive.google.com/file/d/0BzQoj9XC6BxUUWZfdzgwb3J4OWc/view?usp=sharing

[SethTisue]

Is this an issue with XMLEventReader, or with Source.fromFile?

[bbary]

it's an issue with XMLEventReader.
When i convert the file to us-ascii it works, but i mustn't do this

[SethTisue]

The attached file hasBOM.xml

There is no attached file?

[bbary]

ah yes sorry i will attach the file

[SethTisue]

it's an issue with XMLEventReader.

How do you know it isn't just an issue with Source.fromFile?

fromFile has overloads that accept the name of encoding or a Codec object; do you know that there is no possible way to make this work simply by using one of those overloads to tell Source how your file is encoded?

[bbary]

i tested Source.fromFile and it's working fine with this file

[SethTisue]

Attaching the file as a screen shot is useless. You'll need to attach the actual file, or link to the actual file, since the actual bytes matter.

[bbary]

i updated my post with a link, because i coudn't upload an xml file

[SethTisue]

I see this was reported back in 2012 at https://issues.scala-lang.org/browse/SI-6741 scala/bug#6741. (But, this repo is now the right place for the issue, if it is determined to be scala-xml specific.)

[SethTisue]

closely related and perhaps relevant; here is a still-open ticket from 2009 on essentially the same issue, but in the context of scalac rather than scala-xml, and thus involving scala.tools.nsc.io.SourceReader rather than scala.io.Source: https://issues.scala-lang.org/browse/SI-2109 scala/bug#2109

[SethTisue]

judging from http://stackoverflow.com/questions/1835430/byte-order-mark-screws-up-file-reading-in-java and http://mindprod.com/jgloss/bom.html, I think this is just standard JVM stuff and is not scala-xml specific. if it were to be addressed, it would be addressed in io.Source, not in scala-xml. so, closing the issue.

both of the links I've provided suggest multiple strategies for working around this.

[SethTisue]

i tested Source.fromFile and it's working fine with this file

not for me:

scala> io.Source.fromFile("cosi2.xml").getLines.next
res5: String = <root>

that looks OK, but appearances are misleading:

scala> io.Source.fromFile("cosi2.xml").getLines.next.size
res6: Int = 7

scala> io.Source.fromFile("cosi2.xml").getLines.next.head.toInt
res7: Int = 65279

oops — that's garbage from the BOM.

[bbary]

@SethTisue: Thank you for answers
But i found nothing on the internet to solve this, all stackoverflow responses are stacking around Java

[biswanaths]

Hey @thatismypath a working scala snippet,

import org.apache.commons.io.input.BOMInputSteam 
import scala.io.Source
import java.io.FileInputStream
import scala.xml.pull.XMLEventReader

val t = new XMLEventReader( 
               Source.fromInputStream(
                      new BOMInputStream(new FileInputStream("hasBOM.xml"))))

println(t)

For sbt this is the dependency,

libraryDependencies += "commons-io" % "commons-io" % "2.3"

Interestingly enough, XML.loadFile("hasBOM.xml") seems to be working fine, not sure who is handling the BOM, whether the SAXParser which scala-xml uses or something else ?

Regards.

[bbary]

Thank you @biswanaths, it works perfectly with your code. Cool
just correct this:
import org.apache.commons.io.input.BOMInputSteam to
import org.apache.commons.io.input.BOMInputStream

Regards.

[pauloflamyob]

The XML returned from BizTalk contains a BOM character. Java can't handle this. We need to handle this ourselves. See here: #95 (comment)