How are you going about cleaning?

[cbsudux]

How are you going about cleaning this?

Manually or with GPT-4.

[HideLord]

I use a regex to search for a specific subset of instructions/inputs/outputs that are highly likely to be wrong. After that, I manually go and verify that something is indeed incorrect and query gpt-3.5-turbo or fix them by hand.

[gururise]

How are you going about cleaning this?

Manually or with GPT-4.

Much of it has been hand curated with the help of various tools and regular expressions (see tools directory). There exists a tool that can use GPT3.5/4 to double-check the answers.

[ehartford]

i bet one could convince the gpt-3.5 api to take in a line, and decide whether it's acceptable or needs fixed, and if it needs fixed to suggest a fix, which could then be quick-reviewed (accept or reject) by a human

[gururise]

i bet one could convince the gpt-3.5 api to take in a line, and decide whether it's acceptable or needs fixed, and if it needs fixed to suggest a fix, which could then be quick-reviewed (accept or reject) by a human

We have this capability already built out in one of the tools. I think the first 1000 entries have been checked in this manner.