Skip to content
This repository was archived by the owner on Oct 25, 2024. It is now read-only.

Conversation

@luoyu-intel
Copy link
Contributor

Type of Change

update jblas' bug fix commits

@luoyu-intel luoyu-intel requested a review from airMeng as a code owner December 21, 2023 09:03
@VincyZhang
Copy link
Contributor

@VincyZhang VincyZhang added bug Something isn't working ITREX-1.3 labels Dec 21, 2023
@VincyZhang
Copy link
Contributor

@VincyZhang VincyZhang merged commit 47d8755 into main Dec 21, 2023
@VincyZhang VincyZhang deleted the accuracy_fix branch December 21, 2023 12:38
@DDEle
Copy link
Contributor

DDEle commented Dec 22, 2023

It is confirmed that this PR benefits MLPerf accuracy:

  • reference: "ROUGE1", 42.9865 * 0.99, "ROUGE2", 20.1235 * 0.99, "ROUGEL", 29.9881 * 0.99, "GEN_LEN", 4016878*0.9
  • q4_j1_bf16_pc-gcc12 ([Cpp Graph] Align Cpp Beam Search  #322):
    {'rouge1': 43.3352, 'rouge2': 20.1831, 'rougeL': 29.8832, 'rougeLsum': 40.4381, 'gen_len': 4414947, 'gen_num': 13368}
  • q4_j1_bf16_pc-no_mha (around PR950, 8b9020e) (all):
    {'rouge1': 42.8626, 'rouge2': 19.8439, 'rougeL': 29.4733, 'rougeLsum': 40.0532, 'gen_len': 4576100, 'gen_num': 13368}
    (less than 99%)
  • q4_j_bf16_pc-no_mha-gcc13 (inside PR1057, 369c597) (all) ==>(falsely reported as j1 previously)<==
    {'rouge1': 43.2917, 'rouge2': 20.1802, 'rougeL': 29.9402, 'rougeLsum': 40.4317, 'gen_len': 4413826, 'gen_num': 13368}
  • q4_j1_bf16_pc-no_mha-gcc13 (PR1059+, 6e3a514+)(all):
    {'rouge1': 42.8626, 'rouge2': 19.8439, 'rougeL': 29.4733, 'rougeLsum': 40.0532, 'gen_len': 4576100, 'gen_num': 13368}

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

bug Something isn't working ready for merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants