public/eval-qwen3-0-6b
62
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: public:main-260203-172817
Branches Tags
public:main
public:main-260227-152208
public:main-260203-173052
public:main-260203-172817
public:v1.0.4-260122-105458
public:v1.0.4-260121-154022
public:v1.0.4-260121-150747
public:v1.0.4-260121-112021
public:v1.0.3-251215-145444
public:v1.0.3-251126-101652
public:main-251125-165012
public:main-251125-155217
..
compare: public:main
Branches Tags
public:main-260227-152208
public:main-260203-173052
public:main-260203-172817
public:v1.0.4-260122-105458
public:v1.0.4-260121-154022
public:v1.0.4-260121-150747
public:v1.0.4-260121-112021
public:v1.0.3-251215-145444
public:v1.0.3-251126-101652
public:main-251125-165012
public:main-251125-155217
public:main

No commits in common. "main-260203-172817" and "main" have entirely different histories.
main-26020 ... main

30 changed files with 1 additions and 1779 deletions
Show Stats Expand all files Collapse all files

1
.gitattributes vendored
View File

@ -1 +0,0 @@
*.json filter=lfs diff=lfs merge=lfs -text

0
.gitignore vendored
View File

1
README.md Normal file
View File

@ -0,0 +1 @@
This is a repo for model evaluation.

1347
configs/20260204_013552.py
View File

File diff suppressed because it is too large Load Diff

7
logs/eval/public/qwen3-0-6b@main/GaokaoBench_2010-2013_English_MCQs.out
View File

@ -1,7 +0,0 @@
[RISE-CORE Msg(16239:140152262265856:libvgpu.c:901)]: Initializing.....
[RISE-CORE ERROR (pid:16239 thread=140152262265856 libvgpu.c:958)]: cuInit failed:100
02/04 02:16:10 - OpenCompass - INFO - Task [public/qwen3-0-6b@main/GaokaoBench_2010-2013_English_MCQs]: {'score': 40.0}
02/04 02:16:10 - OpenCompass - INFO - time elapsed: 2.35s
/opt/conda/lib/python3.8/site-packages/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
[RISE-CORE Msg(16239:140152262265856:multiprocess_memory_limit.c:504)]: Calling exit handler 16239

7
logs/eval/public/qwen3-0-6b@main/lambada.out
View File

@ -1,7 +0,0 @@
[RISE-CORE Msg(16051:140385105300480:libvgpu.c:901)]: Initializing.....
[RISE-CORE ERROR (pid:16051 thread=140385105300480 libvgpu.c:958)]: cuInit failed:100
02/04 02:15:59 - OpenCompass - INFO - Task [public/qwen3-0-6b@main/lambada]: {'accuracy': 0.038812342324859306}
02/04 02:15:59 - OpenCompass - INFO - time elapsed: 2.27s
/opt/conda/lib/python3.8/site-packages/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
[RISE-CORE Msg(16051:140385105300480:multiprocess_memory_limit.c:504)]: Calling exit handler 16051

7
logs/eval/public/qwen3-0-6b@main/triviaqa.out
View File

@ -1,7 +0,0 @@
[RISE-CORE Msg(16236:140076091423744:libvgpu.c:901)]: Initializing.....
[RISE-CORE ERROR (pid:16236 thread=140076091423744 libvgpu.c:958)]: cuInit failed:100
02/04 02:16:11 - OpenCompass - INFO - Task [public/qwen3-0-6b@main/triviaqa]: {'score': 0.011316057485572028}
02/04 02:16:11 - OpenCompass - INFO - time elapsed: 3.99s
/opt/conda/lib/python3.8/site-packages/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
[RISE-CORE Msg(16236:140076091423744:multiprocess_memory_limit.c:504)]: Calling exit handler 16236

10
logs/infer/public/qwen3-0-6b@main/GaokaoBench_2010-2013_English_MCQs.out
View File

File diff suppressed because one or more lines are too long

10
logs/infer/public/qwen3-0-6b@main/lambada_0.out
View File

File diff suppressed because one or more lines are too long

10
logs/infer/public/qwen3-0-6b@main/lambada_1.out
View File

File diff suppressed because one or more lines are too long

10
logs/infer/public/qwen3-0-6b@main/lambada_2.out
View File

File diff suppressed because one or more lines are too long

10
logs/infer/public/qwen3-0-6b@main/triviaqa_0.out
View File

File diff suppressed because one or more lines are too long

10
logs/infer/public/qwen3-0-6b@main/triviaqa_1.out
View File

File diff suppressed because one or more lines are too long

10
logs/infer/public/qwen3-0-6b@main/triviaqa_2.out
View File

File diff suppressed because one or more lines are too long

10
logs/infer/public/qwen3-0-6b@main/triviaqa_3.out
View File

File diff suppressed because one or more lines are too long

10
logs/infer/public/qwen3-0-6b@main/triviaqa_4.out
View File

File diff suppressed because one or more lines are too long

BIN
predictions/public/qwen3-0-6b@main/GaokaoBench_2010-2013_English_MCQs.json (Stored with Git LFS)
View File

Binary file not shown.

BIN
predictions/public/qwen3-0-6b@main/lambada_0.json (Stored with Git LFS)
View File

Binary file not shown.

BIN
predictions/public/qwen3-0-6b@main/lambada_1.json (Stored with Git LFS)
View File

Binary file not shown.

BIN
predictions/public/qwen3-0-6b@main/lambada_2.json (Stored with Git LFS)
View File

Binary file not shown.

BIN
predictions/public/qwen3-0-6b@main/triviaqa_0.json (Stored with Git LFS)
View File

Binary file not shown.

BIN
predictions/public/qwen3-0-6b@main/triviaqa_1.json (Stored with Git LFS)
View File

Binary file not shown.

BIN
predictions/public/qwen3-0-6b@main/triviaqa_2.json (Stored with Git LFS)
View File

Binary file not shown.

BIN
predictions/public/qwen3-0-6b@main/triviaqa_3.json (Stored with Git LFS)
View File

Binary file not shown.

BIN
predictions/public/qwen3-0-6b@main/triviaqa_4.json (Stored with Git LFS)
View File

Binary file not shown.

BIN
results/public/qwen3-0-6b@main/GaokaoBench_2010-2013_English_MCQs.json (Stored with Git LFS)
View File

Binary file not shown.

BIN
results/public/qwen3-0-6b@main/lambada.json (Stored with Git LFS)
View File

Binary file not shown.

BIN
results/public/qwen3-0-6b@main/triviaqa.json (Stored with Git LFS)
View File

Binary file not shown.

87
summary/summary_20260204_013552.csv
View File

@ -1,87 +0,0 @@
dataset,version,metric,mode,public/qwen3-0-6b@main
--------- 考试 Exam ---------,-,-,-,-
ceval,-,-,-,-
agieval,-,-,-,-
mmlu,-,-,-,-
GaokaoBench,-,-,-,-
ARC-c,-,-,-,-
--------- 语言 Language ---------,-,-,-,-
WiC,-,-,-,-
summedits,-,-,-,-
chid-dev,-,-,-,-
afqmc-dev,-,-,-,-
bustm-dev,-,-,-,-
cluewsc-dev,-,-,-,-
WSC,-,-,-,-
winogrande,-,-,-,-
flores_100,-,-,-,-
--------- 知识 Knowledge ---------,-,-,-,-
BoolQ,-,-,-,-
commonsense_qa,-,-,-,-
nq,-,-,-,-
triviaqa,2121ce,score,gen,0.01
--------- 推理 Reasoning ---------,-,-,-,-
cmnli,-,-,-,-
ocnli,-,-,-,-
ocnli_fc-dev,-,-,-,-
AX_b,-,-,-,-
AX_g,-,-,-,-
CB,-,-,-,-
RTE,-,-,-,-
story_cloze,-,-,-,-
COPA,-,-,-,-
ReCoRD,-,-,-,-
hellaswag,-,-,-,-
piqa,-,-,-,-
siqa,-,-,-,-
strategyqa,-,-,-,-
math,-,-,-,-
gsm8k,-,-,-,-
TheoremQA,-,-,-,-
openai_humaneval,-,-,-,-
mbpp,-,-,-,-
cmmlu,-,-,-,-
bbh,-,-,-,-
--------- 理解 Understanding ---------,-,-,-,-
C3,-,-,-,-
CMRC_dev,-,-,-,-
DRCD_dev,-,-,-,-
MultiRC,-,-,-,-
race-middle,-,-,-,-
race-high,-,-,-,-
openbookqa_fact,-,-,-,-
csl_dev,-,-,-,-
lcsts,-,-,-,-
Xsum,-,-,-,-
eprstmt-dev,-,-,-,-
lambada,217e11,accuracy,gen,0.04
tnews-dev,-,-,-,-
--------- 安全 Safety ---------,-,-,-,-
crows_pairs,-,-,-,-
--------- LEval Exact Match (Acc) ---------,-,-,-,-
LEval_coursera,-,-,-,-
LEval_gsm100,-,-,-,-
LEval_quality,-,-,-,-
LEval_tpo,-,-,-,-
LEval_topic_retrieval,-,-,-,-
--------- LEval Gen (ROUGE) ---------,-,-,-,-
LEval_financialqa,-,-,-,-
LEval_gov_report_summ,-,-,-,-
LEval_legal_contract_qa,-,-,-,-
LEval_meeting_summ,-,-,-,-
LEval_multidocqa,-,-,-,-
LEval_narrativeqa,-,-,-,-
LEval_nq,-,-,-,-
LEval_news_summ,-,-,-,-
LEval_paper_assistant,-,-,-,-
LEval_patent_summ,-,-,-,-
LEval_review_summ,-,-,-,-
LEval_scientificqa,-,-,-,-
LEval_tvshow_summ--------- 长文本 LongBench ---------,-,-,-,-
longbench_lsht,-,-,-,-
longbench_vcsum,-,-,-,-
longbench_dureader,-,-,-,-
longbench_multifieldqa_zh,-,-,-,-
longbench_passage_retrieval_zh,-,-,-,-
--------- 单选 自定义数据 ---------,-,-,-,-
SageBench-exam,-,-,-,-
1 dataset version metric mode public/qwen3-0-6b@main
2 --------- 考试 Exam --------- - - - -
3 ceval - - - -
4 agieval - - - -
5 mmlu - - - -
6 GaokaoBench - - - -
7 ARC-c - - - -
8 --------- 语言 Language --------- - - - -
9 WiC - - - -
10 summedits - - - -
11 chid-dev - - - -
12 afqmc-dev - - - -
13 bustm-dev - - - -
14 cluewsc-dev - - - -
15 WSC - - - -
16 winogrande - - - -
17 flores_100 - - - -
18 --------- 知识 Knowledge --------- - - - -
19 BoolQ - - - -
20 commonsense_qa - - - -
21 nq - - - -
22 triviaqa 2121ce score gen 0.01
23 --------- 推理 Reasoning --------- - - - -
24 cmnli - - - -
25 ocnli - - - -
26 ocnli_fc-dev - - - -
27 AX_b - - - -
28 AX_g - - - -
29 CB - - - -
30 RTE - - - -
31 story_cloze - - - -
32 COPA - - - -
33 ReCoRD - - - -
34 hellaswag - - - -
35 piqa - - - -
36 siqa - - - -
37 strategyqa - - - -
38 math - - - -
39 gsm8k - - - -
40 TheoremQA - - - -
41 openai_humaneval - - - -
42 mbpp - - - -
43 cmmlu - - - -
44 bbh - - - -
45 --------- 理解 Understanding --------- - - - -
46 C3 - - - -
47 CMRC_dev - - - -
48 DRCD_dev - - - -
49 MultiRC - - - -
50 race-middle - - - -
51 race-high - - - -
52 openbookqa_fact - - - -
53 csl_dev - - - -
54 lcsts - - - -
55 Xsum - - - -
56 eprstmt-dev - - - -
57 lambada 217e11 accuracy gen 0.04
58 tnews-dev - - - -
59 --------- 安全 Safety --------- - - - -
60 crows_pairs - - - -
61 --------- LEval Exact Match (Acc) --------- - - - -
62 LEval_coursera - - - -
63 LEval_gsm100 - - - -
64 LEval_quality - - - -
65 LEval_tpo - - - -
66 LEval_topic_retrieval - - - -
67 --------- LEval Gen (ROUGE) --------- - - - -
68 LEval_financialqa - - - -
69 LEval_gov_report_summ - - - -
70 LEval_legal_contract_qa - - - -
71 LEval_meeting_summ - - - -
72 LEval_multidocqa - - - -
73 LEval_narrativeqa - - - -
74 LEval_nq - - - -
75 LEval_news_summ - - - -
76 LEval_paper_assistant - - - -
77 LEval_patent_summ - - - -
78 LEval_review_summ - - - -
79 LEval_scientificqa - - - -
80 LEval_tvshow_summ--------- 长文本 LongBench --------- - - - -
81 longbench_lsht - - - -
82 longbench_vcsum - - - -
83 longbench_dureader - - - -
84 longbench_multifieldqa_zh - - - -
85 longbench_passage_retrieval_zh - - - -
86 --------- 单选 自定义数据 --------- - - - -
87 SageBench-exam - - - -

197
summary/summary_20260204_013552.txt
View File

@ -1,197 +0,0 @@
20260204_013552
tabulate format
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dataset version metric mode public/qwen3-0-6b@main
----------------------------------------------------- --------- -------- ------ ------------------------
--------- 考试 Exam --------- - - - -
ceval - - - -
agieval - - - -
mmlu - - - -
GaokaoBench - - - -
ARC-c - - - -
--------- 语言 Language --------- - - - -
WiC - - - -
summedits - - - -
chid-dev - - - -
afqmc-dev - - - -
bustm-dev - - - -
cluewsc-dev - - - -
WSC - - - -
winogrande - - - -
flores_100 - - - -
--------- 知识 Knowledge --------- - - - -
BoolQ - - - -
commonsense_qa - - - -
nq - - - -
triviaqa 2121ce score gen 0.01
--------- 推理 Reasoning --------- - - - -
cmnli - - - -
ocnli - - - -
ocnli_fc-dev - - - -
AX_b - - - -
AX_g - - - -
CB - - - -
RTE - - - -
story_cloze - - - -
COPA - - - -
ReCoRD - - - -
hellaswag - - - -
piqa - - - -
siqa - - - -
strategyqa - - - -
math - - - -
gsm8k - - - -
TheoremQA - - - -
openai_humaneval - - - -
mbpp - - - -
cmmlu - - - -
bbh - - - -
--------- 理解 Understanding --------- - - - -
C3 - - - -
CMRC_dev - - - -
DRCD_dev - - - -
MultiRC - - - -
race-middle - - - -
race-high - - - -
openbookqa_fact - - - -
csl_dev - - - -
lcsts - - - -
Xsum - - - -
eprstmt-dev - - - -
lambada 217e11 accuracy gen 0.04
tnews-dev - - - -
--------- 安全 Safety --------- - - - -
crows_pairs - - - -
--------- LEval Exact Match (Acc) --------- - - - -
LEval_coursera - - - -
LEval_gsm100 - - - -
LEval_quality - - - -
LEval_tpo - - - -
LEval_topic_retrieval - - - -
--------- LEval Gen (ROUGE) --------- - - - -
LEval_financialqa - - - -
LEval_gov_report_summ - - - -
LEval_legal_contract_qa - - - -
LEval_meeting_summ - - - -
LEval_multidocqa - - - -
LEval_narrativeqa - - - -
LEval_nq - - - -
LEval_news_summ - - - -
LEval_paper_assistant - - - -
LEval_patent_summ - - - -
LEval_review_summ - - - -
LEval_scientificqa - - - -
LEval_tvshow_summ--------- 长文本 LongBench --------- - - - -
longbench_lsht - - - -
longbench_vcsum - - - -
longbench_dureader - - - -
longbench_multifieldqa_zh - - - -
longbench_passage_retrieval_zh - - - -
--------- 单选 自定义数据 --------- - - - -
SageBench-exam - - - -
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
-------------------------------------------------------------------------------------------------------------------------------- THIS IS A DIVIDER --------------------------------------------------------------------------------------------------------------------------------
csv format
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dataset,version,metric,mode,public/qwen3-0-6b@main
--------- 考试 Exam ---------,-,-,-,-
ceval,-,-,-,-
agieval,-,-,-,-
mmlu,-,-,-,-
GaokaoBench,-,-,-,-
ARC-c,-,-,-,-
--------- 语言 Language ---------,-,-,-,-
WiC,-,-,-,-
summedits,-,-,-,-
chid-dev,-,-,-,-
afqmc-dev,-,-,-,-
bustm-dev,-,-,-,-
cluewsc-dev,-,-,-,-
WSC,-,-,-,-
winogrande,-,-,-,-
flores_100,-,-,-,-
--------- 知识 Knowledge ---------,-,-,-,-
BoolQ,-,-,-,-
commonsense_qa,-,-,-,-
nq,-,-,-,-
triviaqa,2121ce,score,gen,0.01
--------- 推理 Reasoning ---------,-,-,-,-
cmnli,-,-,-,-
ocnli,-,-,-,-
ocnli_fc-dev,-,-,-,-
AX_b,-,-,-,-
AX_g,-,-,-,-
CB,-,-,-,-
RTE,-,-,-,-
story_cloze,-,-,-,-
COPA,-,-,-,-
ReCoRD,-,-,-,-
hellaswag,-,-,-,-
piqa,-,-,-,-
siqa,-,-,-,-
strategyqa,-,-,-,-
math,-,-,-,-
gsm8k,-,-,-,-
TheoremQA,-,-,-,-
openai_humaneval,-,-,-,-
mbpp,-,-,-,-
cmmlu,-,-,-,-
bbh,-,-,-,-
--------- 理解 Understanding ---------,-,-,-,-
C3,-,-,-,-
CMRC_dev,-,-,-,-
DRCD_dev,-,-,-,-
MultiRC,-,-,-,-
race-middle,-,-,-,-
race-high,-,-,-,-
openbookqa_fact,-,-,-,-
csl_dev,-,-,-,-
lcsts,-,-,-,-
Xsum,-,-,-,-
eprstmt-dev,-,-,-,-
lambada,217e11,accuracy,gen,0.04
tnews-dev,-,-,-,-
--------- 安全 Safety ---------,-,-,-,-
crows_pairs,-,-,-,-
--------- LEval Exact Match (Acc) ---------,-,-,-,-
LEval_coursera,-,-,-,-
LEval_gsm100,-,-,-,-
LEval_quality,-,-,-,-
LEval_tpo,-,-,-,-
LEval_topic_retrieval,-,-,-,-
--------- LEval Gen (ROUGE) ---------,-,-,-,-
LEval_financialqa,-,-,-,-
LEval_gov_report_summ,-,-,-,-
LEval_legal_contract_qa,-,-,-,-
LEval_meeting_summ,-,-,-,-
LEval_multidocqa,-,-,-,-
LEval_narrativeqa,-,-,-,-
LEval_nq,-,-,-,-
LEval_news_summ,-,-,-,-
LEval_paper_assistant,-,-,-,-
LEval_patent_summ,-,-,-,-
LEval_review_summ,-,-,-,-
LEval_scientificqa,-,-,-,-
LEval_tvshow_summ--------- 长文本 LongBench ---------,-,-,-,-
longbench_lsht,-,-,-,-
longbench_vcsum,-,-,-,-
longbench_dureader,-,-,-,-
longbench_multifieldqa_zh,-,-,-,-
longbench_passage_retrieval_zh,-,-,-,-
--------- 单选 自定义数据 ---------,-,-,-,-
SageBench-exam,-,-,-,-
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
-------------------------------------------------------------------------------------------------------------------------------- THIS IS A DIVIDER --------------------------------------------------------------------------------------------------------------------------------
raw format
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-------------------------------
Model: public/qwen3-0-6b@main
GaokaoBench_2010-2013_English_MCQs: {'score': 40.0}
lambada: {'accuracy': 0.038812342324859306}
triviaqa: {'score': 0.011316057485572028}
GaokaoBench: {'error': "missing datasets: {'GaokaoBench_2010-2022_Geography_MCQs', 'GaokaoBench_2012-2022_English_Cloze_Test', 'GaokaoBench_2010-2022_English_Fill_in_Blanks', 'GaokaoBench_2010-2022_Biology_MCQs', 'GaokaoBench_2010-2022_Chemistry_MCQs', 'GaokaoBench_2010-2022_Math_I_MCQs', 'GaokaoBench_2010-2022_Physics_MCQs', 'GaokaoBench_2010-2022_History_MCQs', 'GaokaoBench_2010-2022_Political_Science_MCQs', 'GaokaoBench_2010-2022_English_Reading_Comp', 'GaokaoBench_2010-2022_Math_II_MCQs', 'GaokaoBench_2010-2022_Chinese_Lang_and_Usage_MCQs', 'GaokaoBench_2010-2022_Chinese_Modern_Lit'}"}
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$