Support Enumerator as an input source

[yorickpeterse]

The Enumerator class can be used to stream data without having to actually write a dedicated class for it. For example:

enum = Enumerator.new do |yielder| 
  HTTPClient.get('http://some-website.com/some-big-file.xml') do |chunk| 
    yielder << chunk
  end
end

document = Oga.parse_xml(enum)

[yorickpeterse]

Worth mentioning: the Enumerator class does not define a read method, nor does it have each_line. Thinking of it, the lexer should also be changed not to read an entire line but instead read a small buffer.

The different scenarios would be as following (in this order):

• Input is a String? Just yield the whole thing, memory is already allocated anyway
• Input responds to read? Use this in combination with a fixed buffer size
• Last option: input responds to each? Use that instead

The rationale for the 2nd and 3rd option is that reading an entire line can be in-efficient. For example, if a large XML file smacks everything on to a single line then the streaming process wouldn't be very efficient.

Code wise this would look something like the following:

if @data.is_a?(String)
  yield @data
# StringIO/IO instances
elsif @data.respond_to?(:read)
  yield @data.read(4) until @data.eof?
# Enumerator and basically everything else
elsif @data.respond_to?(:each)
  @data.each do |line|
    yield line
  end
end