File tree Expand file tree Collapse file tree 1 file changed +19
-2
lines changed Expand file tree Collapse file tree 1 file changed +19
-2
lines changed Original file line number Diff line number Diff line change 22
33<center >
44
5- Under construction
5+ # SciCode Leaderboard  
6+ 
7+ |  Model                  |  Main Problem Resolve Rate | 
8+ | ------------------------| ---------------------------| 
9+ |  OpenAI o1-preview      |  7.7%                      | 
10+ |  Claude3.5-Sonnet       |  4.6%                      | 
11+ |  Deepseek-Coder-v2      |  3.1%                      | 
12+ |  GPT-4o                 |  1.5%                      | 
13+ |  GPT-4-Turbo            |  1.5%                      | 
14+ |  OpenAI o1-mini         |  1.5%                      | 
15+ |  Gemini 1.5 Pro         |  1.5%                      | 
16+ |  Claude3-Opus           |  1.5%                      | 
17+ |  Claude3-Sonnet         |  1.5%                      | 
18+ |  Qwen2-72B-Instruct     |  1.5%                      | 
19+ |  Llama-3.1-405B-Instruct|  0%                        | 
20+ |  Llama-3.1-70B-Instruct |  0%                        | 
21+ |  Mixtral-8x22B-Instruct |  0%                        | 
22+ |  Llama-3-70B-Chat       |  0%                        | 
623
724<!--  Once you've added the results to the submission repository,
825     bring back the table here -->  
@@ -11,4 +28,4 @@ Under construction
1128</center >
1229
1330!!! tip "How to submit"
14-     Want to submit your own model? Head over to the [ documentation] ( docs/index.md ) .
31+     Want to submit your own model? Head over to the [ documentation] ( docs/index.md ) .
 
 
   
 
     
   
   
          
    
    
     
    
      
     
     
    You can’t perform that action at this time.
  
 
    
  
    
      
        
     
       
      
     
   
 
    
    
  
 
  
 
     
    
0 commit comments