🏆 ICML 2026 — Efficient Qwen Leaderboard

Minimizing Inference Latency for Qwen3.5-4B on A10G

Hardware: NVIDIA A10G (ml.g5.xlarge) | Ranked by average speedup over baseline

Baseline (unoptimized Qwen3.5-4B): Short: 2,582 ms | Medium: 5,441 ms | Long: 6,576 ms | Avg: 4,866 ms
#TeamAvg Speedup Short (64t)Medium (2048t)Long (8192t) MMLU-Pro ≥62.1%IFEval ≥81.4%GPQA-Diamond ≥63.0%
1 AFM-e4e9muaf 7.745x 225 ms652 ms1,912 ms ✅ 0.644 ✅ 0.845 ✅ 0.687
2 AFM-6j6duhm6 7.708x 240 ms620 ms1,833 ms ✅ 0.674 ✅ 0.881 ✅ 0.677
3 AFM-as4vvw34 7.514x 320 ms637 ms1,106 ms ✅ 0.643 ✅ 0.821 ✅ 0.667
4 AFM-k5984d3s 6.978x 232 ms782 ms2,313 ms ✅ 0.659 ✅ 0.845 ✅ 0.667
5 AFM-v8ws998a 6.197x 278 ms794 ms2,691 ms ✅ 0.646 ✅ 0.833 ✅ 0.636
6 AFM-gv7e2ebxAFM-gv7e2ebx 6.118x 310 ms703 ms2,891 ms ✅ 0.657 ✅ 0.857 ✅ 0.657
7 AFM-qd39x2e6 5.366x 345 ms881 ms2,684 ms ✅ 0.661 ✅ 0.845 ✅ 0.667
8 AFM-cpykb5cu 5.090x 387 ms972 ms2,198 ms ✅ 0.656 ✅ 0.833 ✅ 0.687
9 AFM-34qt8st6 5.084x 338 ms988 ms3,128 ms ✅ 0.671 ✅ 0.857 ✅ 0.687
10 AFM-v4we4h8u 5.056x 389 ms941 ms2,403 ms ✅ 0.685 ✅ 0.869 ✅ 0.677
11 AFM-dcakuxwp 4.280x 439 ms1,212 ms2,657 ms ✅ 0.654 ✅ 0.845 ✅ 0.687
12 AFM-bxwzjeaf 4.150x 385 ms1,624 ms2,753 ms ✅ 0.642 ✅ 0.857 ✅ 0.636
13 AFM-y6vkzu7s 4.099x 409 ms1,306 ms3,622 ms ✅ 0.670 ✅ 0.869 ✅ 0.636
14 AFM-pzpknzq8 3.880x 418 ms1,485 ms3,666 ms ✅ 0.667 ✅ 0.857 ✅ 0.657
15 AFM-69568ssx 3.224x 650 ms1,525 ms3,089 ms ✅ 0.648 ✅ 0.833 ✅ 0.667
16 AFM-gv7e2ebx 3.145x 663 ms1,587 ms3,115 ms ✅ 0.653 ✅ 0.821 ✅ 0.657
17 AFM-fbdrr789 3.097x 644 ms1,448 ms4,324 ms ✅ 0.671 ✅ 0.845 ✅ 0.636
18 AFM-hnknxz5w 2.997x 674 ms1,549 ms3,996 ms ✅ 0.658 ✅ 0.845 ✅ 0.707
19 AFM-xpvr9w7k 2.982x 700 ms1,687 ms3,234 ms ✅ 0.650 ✅ 0.833 ✅ 0.677
20 AFM-7mpasa9x 2.974x 735 ms1,575 ms3,365 ms ✅ 0.681 ✅ 0.881 ✅ 0.717
21 AFM-nqjfd547 2.887x 718 ms1,465 ms4,865 ms ✅ 0.672 ✅ 0.833 ✅ 0.667
22 AFM-wp5spjm8 2.878x 733 ms1,765 ms3,245 ms ✅ 0.640 ✅ 0.869 ✅ 0.646
23 AFM-pysuua5t 2.723x 795 ms1,884 ms3,235 ms ✅ 0.652 ✅ 0.857 ✅ 0.636
24 AFM-xbp9kss9 2.698x 901 ms1,897 ms2,785 ms ✅ 0.655 ✅ 0.857 ✅ 0.657
25 AFM-jvzphzua 2.694x 867 ms1,803 ms3,150 ms ✅ 0.687 ✅ 0.833 ✅ 0.646
26 AFM-g4yrpeuv 2.479x 769 ms2,029 ms4,710 ms ✅ 0.677 ✅ 0.869 ✅ 0.677
27 AFM-ppyvszdy 2.362x 929 ms2,148 ms3,710 ms ✅ 0.666 ✅ 0.857 ✅ 0.667
28 AFM-x9qzaft6 2.170x 1,047 ms2,378 ms3,745 ms ✅ 0.688 ✅ 0.845 ✅ 0.657
29 AFM-ayxvmqbj 2.155x 1,126 ms2,452 ms3,367 ms ✅ 0.663 ✅ 0.821 ✅ 0.657
30 AFM-kwuct7tx 2.154x 1,126 ms2,449 ms3,378 ms ✅ 0.652 ✅ 0.821 ✅ 0.646
31 AFM-z9qv547h 2.153x 1,126 ms2,451 ms3,382 ms ✅ 0.644 ✅ 0.821 ✅ 0.677
32 AFM-r2csu47t 2.011x 1,060 ms2,876 ms3,858 ms ✅ 0.672 ✅ 0.845 ✅ 0.707
33 AFM-za9buwuh 1.749x 1,308 ms2,996 ms4,518 ms ✅ 0.681 ✅ 0.857 ✅ 0.677
34 AFM-f57gsqmt 1.629x 1,537 ms3,297 ms4,224 ms ✅ 0.684 ✅ 0.857 ✅ 0.636
35 AFM-tr24e5tc 1.621x 1,553 ms3,306 ms4,228 ms ✅ 0.661 ✅ 0.881 ✅ 0.657
36 AFM-npq35pxt 1.604x 1,588 ms3,337 ms4,225 ms ✅ 0.669 ✅ 0.845 ✅ 0.636
37 AFM-yac279yz 1.366x 1,874 ms3,963 ms4,885 ms ✅ 0.671 ✅ 0.845 ✅ 0.677
38 AFM-fh8yvvt4 1.018x 2,588 ms5,382 ms6,297 ms ✅ 0.690 ✅ 0.869 ✅ 0.657
39 AFM-s9gqdtb6 1.011x 2,608 ms5,431 ms6,325 ms ✅ 0.681 ✅ 0.845 ✅ 0.636
40 AFM-newrkm40 1.000x 2,585 ms5,443 ms6,577 ms ✅ 0.691 ✅ 0.857 ✅ 0.667
41 Baseline 0.999x 2,585 ms5,444 ms6,577 ms ✅ 0.685 ✅ 0.857 ✅ 0.697
42 AFM-6bhtn7up 0.000x 0 ms0 ms0 ms ✅ 0.675 ✅ 0.821 ✅ 0.646
43 AFM-42qgh9wu 0.000x 1,147 ms2,611 ms0 ms ✅ 0.687 ✅ 0.845 ✅ 0.646

Updated after each evaluation | Quality gates must pass to be ranked | View all submissions →