-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
Priority Task : start using trulens to evaluate Gemini #1
Comments
hey there @mie-h and @Zochory : https://github.com/Tonic-AI/DataTonic/tree/main/evaluation this is a folder where we will first start working on the trulens evaluations which are a hackathon requirement + good practice while building an app 馃 |
hey there @mie-h & @Zochory : i added default prompts to the baseline prompts folder we can use those in a trulens evaluation. |
consider using this to generate "system prompts" for gemini |
added an incomplete example notebook : https://github.com/Tonic-AI/DataTonic/blob/main/evaluation/results/modelcomparision.ipynb |
big thank you to 馃弳馃槑 @MN-Noor for producing the first TruLens with gemini on RAG using open ai! Open tasks :
we'll all work on this together, normally if everyone does one, or at least contributes to a good one we will have secured this task. |
Est-ce que l'on ajouterait pas d'autres multimodal LLM ? |
馃How To
Check our References
trulens github + notebooks : https://github.com/truera/trulens/tree/main/trulens_eval/examples
https://github.com/truera/trulens/blob/main/trulens_eval/examples/expositional/frameworks/llama_index/llama_index_complex_evals.ipynb
Ideas for Evaluation
Work
What it takes : literally just running a notebook.
we will include the notebooks in the submission and write up
The text was updated successfully, but these errors were encountered: