App to scroll through an entire prompted source

[VictorSanh]

It's sometimes hard to cover all the cases especially if you are not aware of them.
For instance, I was prompting the DROP dataset, but didn't initially realize that:

• the "spans" field always contain multiple entries in the validation split
• there can also be multiple spans in the train split... which made it not straightforward how to combine them (especially if you don't know much about the dataset)

It would be easier to spot these potential introduced bugs if we have a viz app to scroll through the dataset/subset/split for a given prompt, and look at a bunch of prompted examples.
Let's start simple with a second separate app (and can ultimately aim at integrating it into the main tagging app in some way).
At minimum, we need:

• dataset selectbox
• subset selectbox
• split selectbox
  -prompt selectbox
• something like a 2 columns table with on the left the "raw" instances, and on the right the "prompted" instances

I already have a similar code for that, I'll adapt and integrate it

[elsanns]

Hi @VictorSanh,

Great idea.
I have been thinking about adding an Evaluate (or Test) button in the main app, printing out the result of any Jinja expression (from the Prompt field) in the current context. It can be useful for:

• candidate prompts
• parts of Jinja expressions to be used in prompts
• browsing through the dataset
• getting to know Jinja

Once the viz app is integrated, it could be done by making the prompt field editable.

I'm happy to participate in the coding and/or integration of the viz app.

[VictorSanh]

@elsanns isn't what you are describing the actual behavior on the main app? When you update (and save) the "Template" field, it updates on the right ("Template output") the result.
Sorry if I misunderstood something

[elsanns]

@VictorSanh Yes but it takes saving and deleting the template if the user is not happy with the output or just wants to test a part of the prompt. I thought about something more lightweight and, ultimately, applied to a subset of examples, not just one record (like the viz app, after integration)
It is just an idea, I guess we would need to figure out how to integrate the viz app with the main module and then decide how it fits in.

[VictorSanh]

Aaaah I see what you mean!
I think for now we can just use it as it is (saving and then delete if necessary). Jinja is pretty easy to learn!
It makes me think that we should add some "jinja - crash course" section in the template writing guide (#26)

[stephenbach]

+1 to Jinja crash course! A first draft for #26 is on my todo list today.

[VictorSanh]

Closing as we just merged #44