benchmark json errors, clean json parsing code and implement json schema

[waynehamadi]

Background

we have a lot of issues with json parsing.

Changes

• Added benchmarks to measure json errors
• refactored json parsing so that it uses the right methods and has less if conditions
• used the right method to clean the json
• implemented json schema so that we have a clean AI input to work with => we need someone to work on feeding the json validations error back to the AI, so that it corrects its json itself (this is a different problem than JSON parsing)

Documentation

Test Plan

PR Quality Checklist

• My pull request is atomic and focuses on a single change.
• I have thoroughly tested my changes with multiple different prompts.
• I have considered potential risks and mitigations for my changes.
• I have documented my changes clearly and comprehensively.
• I have not snuck in any "extra" small tweaks changes

[waynehamadi]

work in progress still, we're testing the benchmark on github action and needs a rebase as well.