Is your feature request related to a problem?
We encountered issues during STT Evaluation in staging with ATREE, where several samples returned an error stating {'message': 'Cannot fetch content from the provided URL.', 'code': 3}. This inconsistency affected the reliability of the evaluation process.
These evaluation runs by ATREE are in staging (sample language: Kannada):
| Evaluation Run |
Run ID |
Samples |
Failed |
| STT Evals 6 |
260 |
49 |
20 |
| STT Evals 5 |
259 |
49 |
16 |
| STT Evals 5 |
252 |
49 |
11 |
| STT Evals 4 |
252 |
49 |
13 |
Describe the solution you'd like
- Debug the current STT evaluation process to identify the root cause of the errors.
- Consider altering our approach for multi-modal batch operations using Gemini.
Screenshot
