Skip to content

[Issue]: Would be Nice to Have a Powerful WebSurfer Team Submitted to LMSys.org #2898

Closed as not planned
@Josephrp

Description

@Josephrp

Describe the issue

LMSys.org

Large Model Systems Organization (LMSYS Org) is an open research organization founded by students and faculty from UC Berkeley in collaboration with UCSD and CMU.

Chatbot Arena

Scalable and gamified evaluation of LLMs via crowdsourcing and Elo rating systems.

Rationale

  • LMSys would/could give visibility to autogen
  • LMSys basically now allows "agents" to be validated/verified
  • hosting a permament provider would provide a good reason to get a pretty huge budget
  • it would be nice for a "basic" autogen agent team to "win" on a leaderboard

Steps to reproduce

  1. get a budget
  2. host a websurfing team
  3. demonstrate autogen in production
  4. register the host on LMSys
  5. get potentially hundreds of thousands of users evaluating autogen

Screenshots and logs

image
image

Additional Information

maybe it's not a good idea , i dont really know :-) close this issue at your convenience , but you heard it here first ;-)

Metadata

Metadata

Assignees

No one assigned

    Labels

    0.2Issues which are related to the pre 0.4 codebaseneeds-triage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions