Feature/summarization

[calebgcc]

Another step forward to solve #16.

I added a prompt for the default summarization method that can be imported from plugins (also changed chain type from "map_reduce" to "stuff").

I tested the new text_splitter and it works great, the problem of reaching maximum token is solved now.
We can talk about passing a group of documents instead of just one document, this could help implement hierarchical summarization very easily (if we iterate this step on the output).

[pieroit]

Yes @calebgcc go for it!

[calebgcc]

@pieroit I just added iterative summarization.
It works fine, the only issue now is that for a long document it takes ages, I tried with alice.txt and it took roughly 20 minutes.

Two possible improvements could be:

• saving intermediate results
• use threads (or similar) to speed up the process