Skip to content

Pull requests: open-compass/VLMEvalKit

New pull request New
30 Open 881 Closed
30 Open 881 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

[API] Add DeepOCR pipeline API provider
#1473 opened Mar 4, 2026 by leejooan Loading…
2
[Benchmark] XLRSBench: rewrite evaluate() with track_progress_rich and GPT-assisted scoring
#1472 opened Mar 4, 2026 by TianhaoLiang2000 Loading…
[Minor] sarena: explicit LPIPS model path via LMUDataRoot, use cached tokenizer path
#1471 opened Mar 4, 2026 by TianhaoLiang2000 Loading…
[Fix] physics: deduplicate preds to handle repeat output, add signal timeout handler
#1470 opened Mar 4, 2026 by TianhaoLiang2000 Loading…
[Fix] olympiadbench: add isinstance(line, int) guard in build_prompt to handle integer row index
#1469 opened Mar 4, 2026 by TianhaoLiang2000 Loading…
[Minor] mmifeval: use __name__ for logger, downgrade score_dict log to debug
#1468 opened Mar 4, 2026 by TianhaoLiang2000 Loading…
[Minor] macbench: enable interleave by default, move images before text in non-interleave mode
#1467 opened Mar 4, 2026 by TianhaoLiang2000 Loading…
[Minor] ChartMimic: Add exception guard, 600s timeout, chart_type 120s, log to debug
#1465 opened Mar 4, 2026 by TianhaoLiang2000 Loading…
[Benchmark]Add SGI-Bench 1.0 dataset
#1464 opened Mar 4, 2026 by TianhaoLiang2000 Loading…
[Minor] Add dataset NPMM-bench
#1463 opened Mar 3, 2026 by OliverLeeXZ Loading…
jt video chat v260227
#1460 opened Feb 28, 2026 by jiutiancv Loading…
[Benchmark] Add OCRBench_v2_MINI dataset
#1454 opened Feb 25, 2026 by TianhaoLiang2000 Loading…
[Fix] imagevqa.py: logicvista, mmsci_captioning
#1453 opened Feb 25, 2026 by TianhaoLiang2000 Loading…
Fix LLaVA model output issues by using official conversation templates WIP
#1424 opened Feb 2, 2026 by cdllI Loading…
3
[Benchmark] Support ScienceOlympiad Galaxy10DECaLS VRSBench
#1410 opened Jan 21, 2026 by zhouyujin Loading…
2
[Benchmark] Add AV-SpeakerBench
#1393 opened Jan 3, 2026 by plnguyen2908 Loading…
1
[Benchmark] Support SArena Benchmark
#1371 opened Dec 23, 2025 by JoeLeelyf Loading…
Add SGI-Bench-1.0
#1358 opened Dec 15, 2025 by vaporhug Loading…
6
[Benchmark] Support OmniBench
#1327 opened Nov 26, 2025 by jmlee4967 Loading…
[Benchmark] Support VP-Bench in Image MCQ WIP
#1322 opened Nov 23, 2025 by Endlinc Loading…
1
Request to add SIBench evaluation code
#1310 opened Nov 9, 2025 by song2yu Loading…
4
add support for internvideo, videollama, keye and llava-onevision-1.5 WIP
#1244 opened Sep 22, 2025 by FangXinyu-0913 Loading…
@FangXinyu-0913
1
[major] add tsv, xlsx, json fo PRED_FORMAT and csv, json for EVAL_FORMAT
#1172 opened Jul 20, 2025 by OliverLeeXZ Loading…
[Model] Add new model provider: AI/ML API
#1093 opened Jun 19, 2025 by D1m7asis Loading…
4 tasks done
3
WIP: Add Capture Dataset WIP
#974 opened May 2, 2025 by bodsul Loading…
2
ProTip! no:milestone will show everything without a milestone.