-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Brainstorming concept : DataTonic, help and find the most optimized LLM model for an user usecase #3
Comments
just one point first : the user intention seems like it's not an objective and more of a technique, which is not exactly what i would expect in the user section. this is too complex to put in work right now , let's do an MVP with a single configuration , that sometimes works at least, then it will be easier to provide alternatives. in fact we can start evaluating alternatives in the context of the TruEra evaluation . |
Data Driven Advisory (Use Case)Phase 1: Engagement Setup (1-2 weeks)
Phase 2: Data Gathering and Analysis (3-6 weeks)
Phase 3: In-Depth Analysis and Hypothesis Testing (4-8 weeks)
Phase 4: Solution Development and Validation (2-4 weeks)
Phase 5: Final Recommendations and Implementation Planning (2-3 weeks)
Phase 6: Implementation Support and Closure (Variable)
Post-Engagement (Optional)
Statement Of Work:(example of a single fixed output useable by autogen)
Project Objectives and Scope
Project Approach and Methodology
Deliverables
Timeline
*6. Roles and Responsibilities
Pricing and Payment Terms
Confidentiality, Legal, and Ethical Considerations
Terms and Conditions
Signatures
|
ok since everyone is aligned, i made a new issue, so we can try it like that now. |
Problem :
Today there is a large choice of custom and sometime not accessible or complex, time consuming, would need to pay a subscription to *some bullshit AI solution over priced", would need to pay fair subscription price, or to pay expensive a freelance or agency to do it.
Concept :
Datatonic uses autogen, truegen, its own prompt engenering, its agent...
is able to evaluate and do testing in order to find the proper LLM model for given scenario
take into account multimodel (sound, image, text) using Gemini and Truegen.
Benefits for end users/companies :
Benefit for us :
Limitation:
Seem complex, too irrealistic ?
Scenario example
User :
↳ Add a credential
↳ Explain what he want to accomplish
System (still need more information so will follow ask follow up until system has all he needs :
↳ Budget per token max
↳ Currently using solutions to add as integration :
↳ Specific demands for it like for eg :
↳ When user has done and system has all infos he needs open draggrable list with each component from users
↳ Enterprise extra security needed ?
↳ ...
↳ ...
↳ User can re-order the list based on the importance
↳ Or other kind of measure like a typical five-level Likert item (trivial , not important, important, very important)
↳ Datatonic through its datasets (can be some benchmark of Transformers, embedding/chunking...), Existing evaluations based on the proper metric, on its own evaluation already made and often updated between each langage models, multimodal models (would be more complex (?))
↳ Provide few bundles possibilities (without to have to create each (possible?)
↳ User choose one of these, or through chat input ask for ajustment
Final step 🕺🏿↳ When user has chosen its bundle, Datatonic starts the work and user wil be notified when it's done
### Final step seem too much and seem to add too much complexity and seems a bit a non sense ? when we could provide detailed documentations instead... and give choice to make us built it for him/company as an agency with a support and on boarding
The text was updated successfully, but these errors were encountered: