GPT 4 Image Evals #411
Description
Hello, I wanted to open an issue/question regarding GPT-4 evals with images. I have a few questions as I'm looking to write some evals to test a few ideas I had.
- Is there a rough timeline for when GPT-4 with image scanning will be available?
- When this feature comes out, will queries have to be done individually? Can we create a batch of images and include a prompt?
- If someone were to process large quantities of large images, say 10k images a day, what would that look like cost wise?
I plan to write an eval to test a significantly large dataset of high-quality 360 images of cars and answer questions the end user of a site like Carvana may have. Questions like:
- Does this car have any visible markings, dents, or damage on the surface?
- Will this car be able to take me off the road when I go camping?
- Is this a good fuel-efficient car for me to use?
- ...
Would an evaluation like this be helpful to the public? I want to test this for my needs, but if this would be useful, I can take a fragment of the set and upload it somewhere once I write the prompts.