Trivial Questions

    Definition:

    The question must require the model to demonstrate conceptual understanding of the visual data — beyond simple extraction or counting. The model should interpret the "why" or "how" of the visual.

    • Layers of reasoning: Each annotation needs at least 2 layers of logical reasoning.
    • Ontology requirements: Every VQA annotation must meet the ontology minimums — at least 2 ontologies, 3 if one of them is enumeration, and at least 1 anchor ontology.
    • Contextual questions: VQA annotations should get at the underlying meaning of the image, not surface-level details or arbitrary math. Ask two questions before submitting: does this annotation help the model understand the image better, and would a human ever actually ask this?
    • Image dependence: The prompt must depend on the image. It cannot ask about things that aren't pictured, and it cannot be answerable from general knowledge alone — the answer has to come from visual evidence in the image.
    • Performative length: Avoid wording, procedural steps, or constraints that increase instruction-parsing effort without increasing reasoning difficulty.
    • Good questions require: comparison of multiple items, finding second/third highest, numeric answers not directly on the chart, choosing the correct subplot, matching to legend labels, region-restricted reading, counting items that satisfy a condition, threshold comparisons, near-ties where the gap is still clearly visible by eye, distinguishing plotted values from labels, judging trend changes or symmetry, or using heatmaps/tables/boxplots when clearly readable.
    • Avoid: giveaways, one-step questions, questions answerable from the title alone or from a printed label or legend value, questions requiring external paper knowledge, and comparisons between two marks that look equal by eye, since that is a precision dispute rather than a reasoning question.

    ✅ Pass

    1. Requires the model to synthesize multiple visual cues.
    2. Asks for numeric answers not directly written on the chart.
    3. Requires at least two layers of logical reasoning, leading to one final question.
    4. Instructions are clear and proportionate.
    5. Answer length is no more than 5 words.
    6. Relies entirely on visual evidence within the image and cannot be answered without it.

    ❌ Fail

    1. Overly simple, number/character extraction.
    2. Question states the answer directly.
    3. No multi-step structure, or multiple separate questions instead of a sequence.
    4. Excessive indirection or inflated wording.
    5. Answer length is more than 5 words.
    6. Both inputs are printed on the image (end labels, data labels, legend values) and the only work is one subtraction or comparison between them. That is extraction, not chart reading.
    7. Asks about items not pictured, or is a general-knowledge question solvable without the image.

    Knowledge Check

    Which prompt is acceptable for the data table above?

    Which prompt is acceptable for the e-commerce dashboard above?

    Image FeasibilityPrompt Clarity