A Dataset for VQA on Document Images.

Evaluating Large Multimodal Models for Integrated Capabilities

OCRBench is a comprehensive evaluation benchmark designed to assess the OCR capabilities of Large Multimodal Models. It comprises five components: Text Recognition, SceneText-Centric VQA, Document-Oriented VQA, Key Information Extraction, and Handwritten Mathematical Expression Recognition. The benchmark includes 1000 question-answer pairs, and all the answers undergo manual verification and correction to ensure a more precise evaluation.

AI2 Diagrams (AI2D) is a dataset of over 5000 grade school science diagrams with over 150000 rich annotations, their ground truth syntactic parses, and more than 15000 corresponding multiple choice questions.

This repo contains data for the paper "BLINK: Multimodal Large Language Models Can See but Not Perceive"

An Advanced Diagnostic Suite for Entangled Language Hallucination & Visual Illusion in Large Vision-Language Models

A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI

MathVista is a consolidated Mathematical reasoning benchmark within Visual contexts.

A Benchmark for Question Answering about Charts with Visual and Logical Reasoning

A Benchmark for Visual Question Answering using World Knowledge.

A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning