[Pipeline Refactor] Split/Join Functionality for multiple prompts

[dsikka]

Summary

• Introduce functionality such that multiple prompts can run, using SPLIT_ROUTE and JOIN_ROUTE nodes.
• Allows us to enable the num_return_sequences generation attribute, such that multiple generations can be produced
• Potentially still needs a good testing strategy?

Testing

model_path = "hf:mgoin/TinyStories-1M-deepsparse"
pipeline = TextGenerationPipeline(
    model_path, prompt_sequence_length=3
)

def run_requests():
    prompts = ["The dog barked", "Hello!"]
    outputs = []
    input_value = TextGenerationInput(prompt=prompts, generation_kwargs={"num_return_sequences": 2, "do_sample": True, "max_length": 20})
    output = pipeline(input_value)
    return output


output = run_requests()
for x in output.generations:
    print("\n")
    print(x)

Outputs:

[GeneratedText(text=' and ran to the farm. They chased after the birds to chase Sisi and the chubby dog', score=None, finished=True, finished_reason='length'), GeneratedText(text=' and got with far and hotries on his arm. \n\nHis mom grabbed him out and', score=None, finished=True, finished_reason='length')]


[GeneratedText(text=' That was my name, Molly. The elephant was smiling.\n\nThe giant was gentle, and', score=None, finished=True, finished_reason='length'), GeneratedText(text=' Why?" The lady thought for a moment, and then said, "That\'s a tiny bird,', score=None, finished=True, finished_reason='length')]