LongBanana: An Expert-Verified Benchmark for Long-Context Multi-Reference Image Synthesis
arXiv cs.CL · 5 Nov 2026, 04:11 AEST
LongBanana is introduced as the first expert-designed and verified benchmark for long-context multi-reference image synthesis. It includes 90 tasks with extensive evaluation metrics to enhance agent-driven image generation.
Key facts
- The task requires retrieving dispersed visual evidence, rejecting distractors, binding entities and relations, and composing one coherent final image from multiple references.
- LongBanana contains 90 carefully designed tasks organized by a 3-by-3 taxonomy; each task includes about 32 reference images, 20 constraints, and 50 evaluation items, yielding 4,670 expert-specified atomic evaluation items in total.
- Its graph-grounded evaluation design instantiates each task with reference images and an entity-relation graph, then converts node, edge, and whole-graph obligations into checklist items for Content Fidelity, Relational Binding, and Global Constraint Satisfaction.
- Reviewed expert labels include a 15% double-labeled subset with 89.4% exact agreement and 0.83 quadratic weighted kappa.
- Together, LongBanana provides an auditable testbed for measuring dense visual evidence use and a shared target for developing stronger agent-driven image-generation systems.
Read aloud
LongBanana is introduced as the first expert-designed and verified benchmark for long-context multi-reference image synthesis. It includes ninety tasks with extensive evaluation metrics to enhance agent-driven image generation. The task requires retrieving dispersed visual evidence, rejecting distractors, binding entities and relations, and composing one coherent final image from multiple references. LongBanana contains ninety carefully designed tasks organized by a three-by-three taxonomy; each task includes about thirty two reference images, twenty constraints, and fifty evaluation items, yielding four thousand six hundred seventy expert-specified atomic evaluation items in total. Its graph-grounded evaluation design instantiates each task with reference images and an entity-relation graph, then converts node, edge, and whole-graph obligations into checklist items for Content Fidelity, Relational Binding, and Global Constraint Satisfaction. Reviewed expert labels include a fifteen percent double-labeled subset with eighty nine point four percent exact agreement and zero point eight three quadratic weighted kappa. Together, LongBanana provides an auditable testbed for measuring dense visual evidence use and a shared target for developing stronger agent-driven image-generation systems.
Sources in this cluster
- LongBanana: An Expert-Verified Benchmark for Long-Context Multi-Reference Image Synthesis
arXiv cs.CL · 5 Nov 2026, 04:11 AEST