Need to be able to add an iterator to the index queue instead of raw data

[Shazwazza]

For example, when indexing thousands of items, it would be much better if we can queue up an iterator for the worker thread to process instead of queuing up already serialized items which can consume memory.

[Shazwazza]

So basically instead of this combination of methods:

 void AddNodesToIndex(IEnumerable<XElement> nodes, string type)

and

 void EnqueueIndexOperation(IndexOperation op)

we should be able to pass a callback or something to the IndexOperation so that it can resolve the XElement just before it indexes that way it won't consume memory in the queue.

[Shazwazza]

The AddNodesToIndex is fine, but internally added this method:

protected void EnqueueIndexOperation(IEnumerable ops)

and changed the blocking collection to work with batches.