File tree Expand file tree Collapse file tree 1 file changed +8
-1
lines changed Expand file tree Collapse file tree 1 file changed +8
-1
lines changed Original file line number Diff line number Diff line change @@ -127,7 +127,7 @@ bigcodebench.evaluate \
127127  --execution [e2b| gradio| local] \
128128  --split [complete| instruct] \
129129  --subset [full| hard] \
130-   --backend [vllm| openai| anthropic| google| mistral| hf]
130+   --backend [vllm| openai| anthropic| google| mistral| hf| hf-inference ]
131131``` 
132132
133133-  All the resulted files will be stored in a folder named ` bcb_results ` .
@@ -177,6 +177,13 @@ Access Gemini APIs from [Google AI Studio](https://aistudio.google.com/)
177177export  GOOGLE_API_KEY=< your_google_api_key> 
178178``` 
179179
180+ Access the [ Hugging Face Serverless Inference API] ( https://huggingface.co/docs/api-inference/en/index ) 
181+ ``` bash 
182+ export  HF_INFERENCE_API_KEY=< your_hf_api_key> 
183+ ``` 
184+ 
185+ Please make sure your HF access token has the ` Make calls to inference providers `  permission.
186+ 
180187## 💻 LLM-generated Code  
181188
182189We share pre-generated code samples from LLMs we have [ evaluated] ( https://huggingface.co/spaces/bigcode/bigcodebench-leaderboard )  on the full set:
    
 
   
 
     
   
   
          
     
  
    
     
 
    
      
     
 
     
    You can’t perform that action at this time.
  
 
    
  
     
    
      
        
     
 
       
      
     
   
 
    
    
  
 
  
 
     
    
0 commit comments