{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":706591083,"defaultBranch":"bigdl_xpu","name":"lm-evaluation-harness","ownerLogin":"analytics-zoo","currentUserCanPush":false,"isFork":true,"isEmpty":false,"createdAt":"2023-10-18T08:49:27.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/26344540?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1697619140.0","currentOid":""},"activityList":{"items":[{"before":"f71ae2e5449771007bc94de5a968d61356533e76","after":"23c5ca4b3566c21aedb79772947529615e19b554","ref":"refs/heads/bigdl_xpu","pushedAt":"2023-11-09T01:34:01.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhentaocc","name":"Chen, Zhentao","path":"/zhentaocc","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90437536?s=80&v=4"},"commit":{"message":"Add mixed 4bit option (#10)\n\nCo-authored-by: rnwang04 <ruonan1.wang@intel.com>","shortMessageHtmlLink":"Add mixed 4bit option (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1983166870\" data-permission-text=\"Title is private\" data-url=\"https://github.com/analytics-zoo/lm-evaluation-harness/issues/10\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/analytics-zoo/lm-evaluation-harness/pull/10/hovercard\" href=\"https://github.com/analytics-zoo/lm-evaluation-harness/pull/10\">#10</a>)"}},{"before":"4c56950bc76b78afb483fa94371fb847a00acf03","after":"f71ae2e5449771007bc94de5a968d61356533e76","ref":"refs/heads/bigdl_xpu","pushedAt":"2023-10-30T08:54:52.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhentaocc","name":"Chen, Zhentao","path":"/zhentaocc","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90437536?s=80&v=4"},"commit":{"message":"fix environment setup (#8)","shortMessageHtmlLink":"fix environment setup (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1967785603\" data-permission-text=\"Title is private\" data-url=\"https://github.com/analytics-zoo/lm-evaluation-harness/issues/8\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/analytics-zoo/lm-evaluation-harness/pull/8/hovercard\" href=\"https://github.com/analytics-zoo/lm-evaluation-harness/pull/8\">#8</a>)"}},{"before":"571f9af254602dd332b5c763c3c851f6ca2adeb6","after":"4c56950bc76b78afb483fa94371fb847a00acf03","ref":"refs/heads/bigdl_xpu","pushedAt":"2023-10-30T04:41:45.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhentaocc","name":"Chen, Zhentao","path":"/zhentaocc","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90437536?s=80&v=4"},"commit":{"message":"better describe how to run harness llb.py (#7)\n\n* change to names with BigDL-LLM\r\n\r\n* add gitignore rules\r\n\r\n* add guidance to prepare datasets, model and env\r\n\r\n* update llb.py description\r\n\r\n* fix if error\r\n\r\n* fix missing fi\r\n\r\n* fix xpu setvars\r\n\r\n* check logs and results","shortMessageHtmlLink":"better describe how to run harness llb.py (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1967408858\" data-permission-text=\"Title is private\" data-url=\"https://github.com/analytics-zoo/lm-evaluation-harness/issues/7\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/analytics-zoo/lm-evaluation-harness/pull/7/hovercard\" href=\"https://github.com/analytics-zoo/lm-evaluation-harness/pull/7\">#7</a>)"}},{"before":"b11e0cacc787ee68389b9bd5f7918913caba2882","after":"571f9af254602dd332b5c763c3c851f6ca2adeb6","ref":"refs/heads/bigdl_xpu","pushedAt":"2023-10-27T04:37:30.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhentaocc","name":"Chen, Zhentao","path":"/zhentaocc","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90437536?s=80&v=4"},"commit":{"message":"disable doublt quant for bnb nf4 (#6)","shortMessageHtmlLink":"disable doublt quant for bnb nf4 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1964751256\" data-permission-text=\"Title is private\" data-url=\"https://github.com/analytics-zoo/lm-evaluation-harness/issues/6\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/analytics-zoo/lm-evaluation-harness/pull/6/hovercard\" href=\"https://github.com/analytics-zoo/lm-evaluation-harness/pull/6\">#6</a>)"}},{"before":"7c45b2a931a78d3ccc861eb20dbb47dfb62f068d","after":"b11e0cacc787ee68389b9bd5f7918913caba2882","ref":"refs/heads/bigdl_xpu","pushedAt":"2023-10-27T04:35:02.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhentaocc","name":"Chen, Zhentao","path":"/zhentaocc","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90437536?s=80&v=4"},"commit":{"message":"Add 16 bit support to bigdl llm (#5)\n\n* add 16bit support to llm-xpu\r\n\r\n* fix return code","shortMessageHtmlLink":"Add 16 bit support to bigdl llm (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1964749298\" data-permission-text=\"Title is private\" data-url=\"https://github.com/analytics-zoo/lm-evaluation-harness/issues/5\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/analytics-zoo/lm-evaluation-harness/pull/5/hovercard\" href=\"https://github.com/analytics-zoo/lm-evaluation-harness/pull/5\">#5</a>)"}},{"before":"606fb89f169ebab9c5639b4bd284dbcb9897c6fa","after":"7c45b2a931a78d3ccc861eb20dbb47dfb62f068d","ref":"refs/heads/bigdl_xpu","pushedAt":"2023-10-27T04:24:00.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhentaocc","name":"Chen, Zhentao","path":"/zhentaocc","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90437536?s=80&v=4"},"commit":{"message":"Add xpu fp8 and fp4 (#4)\n\n* add options: fp8 & fp4\r\n\r\n* print model\r\n\r\n* add batch option to llb.py","shortMessageHtmlLink":"Add xpu fp8 and fp4 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1964740762\" data-permission-text=\"Title is private\" data-url=\"https://github.com/analytics-zoo/lm-evaluation-harness/issues/4\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/analytics-zoo/lm-evaluation-harness/pull/4/hovercard\" href=\"https://github.com/analytics-zoo/lm-evaluation-harness/pull/4\">#4</a>)"}},{"before":"783f781b0e975ac74dd63c2e382fa094c7e827f0","after":"606fb89f169ebab9c5639b4bd284dbcb9897c6fa","ref":"refs/heads/bigdl_xpu","pushedAt":"2023-10-20T04:32:46.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhentaocc","name":"Chen, Zhentao","path":"/zhentaocc","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90437536?s=80&v=4"},"commit":{"message":"Add leaderboard  script (#2)\n\n* import ipex only when running on xpu\r\n\r\n* add leaderboard run script\r\n\r\n* update readme to guide how to run","shortMessageHtmlLink":"Add leaderboard script (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1953499708\" data-permission-text=\"Title is private\" data-url=\"https://github.com/analytics-zoo/lm-evaluation-harness/issues/2\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/analytics-zoo/lm-evaluation-harness/pull/2/hovercard\" href=\"https://github.com/analytics-zoo/lm-evaluation-harness/pull/2\">#2</a>)"}},{"before":"56717b2eef8332edf2ae2d225194f628a14a0e2e","after":"783f781b0e975ac74dd63c2e382fa094c7e827f0","ref":"refs/heads/bigdl_xpu","pushedAt":"2023-10-19T00:19:31.000Z","pushType":"pr_merge","commitsCount":6,"pusher":{"login":"zhentaocc","name":"Chen, Zhentao","path":"/zhentaocc","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90437536?s=80&v=4"},"commit":{"message":"Merge pull request #1 from zhentaocc/bigdl_xpu\n\nadd bitsandbytes support","shortMessageHtmlLink":"Merge pull request <a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1950264583\" data-permission-text=\"Title is private\" data-url=\"https://github.com/analytics-zoo/lm-evaluation-harness/issues/1\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/analytics-zoo/lm-evaluation-harness/pull/1/hovercard\" href=\"https://github.com/analytics-zoo/lm-evaluation-harness/pull/1\">#1</a> from zhentaocc/bigdl_xpu"}},{"before":"5b97145993fda6789082bf9fe12736455ab62418","after":"56717b2eef8332edf2ae2d225194f628a14a0e2e","ref":"refs/heads/bigdl_xpu","pushedAt":"2023-10-18T09:24:51.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"zhentaocc","name":"Chen, Zhentao","path":"/zhentaocc","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90437536?s=80&v=4"},"commit":{"message":"add quantization config to hf-causal model","shortMessageHtmlLink":"add quantization config to hf-causal model"}},{"before":null,"after":"5b97145993fda6789082bf9fe12736455ab62418","ref":"refs/heads/bigdl_xpu","pushedAt":"2023-10-18T08:52:20.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"zhentaocc","name":"Chen, Zhentao","path":"/zhentaocc","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90437536?s=80&v=4"},"commit":{"message":"revert back to use self.device instead of xpu","shortMessageHtmlLink":"revert back to use self.device instead of xpu"}}],"hasNextPage":false,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAADqt7bgwA","startCursor":null,"endCursor":null}},"title":"Activity · analytics-zoo/lm-evaluation-harness"}