You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Also curious, as I know, the HumanEval dataset contains 164 problems, according to the latest result in README, the model does not even pass any one of them!
Ya that's correct we discovered issues with the pre-training corpus data. We fixed the issue and released a new pertaining corpus and are in the process of processing the dataset further and pre-training a new GPT-CC.
This is Codex results
![image](https://user-images.githubusercontent.com/84262181/139406446-254257ea-4cf9-47e9-8b91-bcb0c0f231ef.png)
The following result is GPT-CC
![image](https://user-images.githubusercontent.com/84262181/139406526-8fe7f262-8eb4-4b77-8df0-f9ada390e692.png)
Thanks
The text was updated successfully, but these errors were encountered: