Prompt Clarity

    Definition:

    Each question should be written clearly and lack ambiguity.

    • Question is clear: Grammatically correct, free of typos, and not open to multiple valid interpretations.
    • Fine-grained precision: If the question challenges spatial reasoning, frame it to avoid precision disputes — use approximation wording with a specified rounding rule that yields one verifiable answer, or MCQs with sufficiently distinct options. Do not allow variance or tolerance ranges that accept multiple final answers. For example, “Answer with a percentage rounded to the nearest whole percent, any value within 5% is acceptable” is unacceptable because the range permits several answers. Chartography exception: if the task category says Chartography, an estimated value read from gridlines may use a stated accepted range or tolerance, and the model's wrong answer must fall outside it. All other tasks keep this rule.

    What to avoid:

    • Ambiguous questions or ties.
    • Answer variance or tolerance ranges, such as “any value within 5% is acceptable,” that allow multiple accepted final answers. Chartography exception: This prohibition does not apply when the task category says Chartography: an estimated value read from gridlines may use a clearly stated accepted range or tolerance. The model's wrong answer must fall outside that range. All other task categories retain the prohibition.
    • Exact numeric questions where the value cannot be read from the chart.
    • Overly broad trend questions with many possible answers.
    • Questions where nearby annotations or formulas could be confused with plotted values.
    • Color as a short answer — shades are named differently by different people. If the answer is a color, write the question as an MCQ so the options define the answer.
    • Stacked charts without saying which edge you mean. Write 'the top edge of the Dataset 7 band', not 'the value of Dataset 7'.
    • Undefined superlatives or change words. 'The biggest change', 'starts to decline' or 'the largest growth' must say which window, which two endpoints, and whether you mean month-over-month or first-to-last.

    ✅ Pass

    1. Clear, precise language with proper grammar; the intent is unmistakable.
    2. Spatial reasoning with clear boundaries or distinct choices. For non-Chartography tasks, approximation wording and rounding must yield one verifiable final answer, without a range of accepted values. If the task category says Chartography, an estimated value read from gridlines may instead use a clearly stated accepted range or tolerance. The question must still have one clear interpretation, and the model's wrong answer must fall outside that range.
    3. Color answers are given as MCQ options rather than free text.

    ❌ Fail

    1. Spelling errors, poor grammar, or vague language.
    2. Demands an overly precise spatial judgment that is subjective or unverifiable.
    3. Asks for a color as a short answer.
    4. A non-Chartography task allows variance or a tolerance range that accepts multiple final answers, even if a rounding rule is also given. For Chartography-category tasks, a stated accepted range or tolerance for an estimated gridline value is not a clarity failure; unclear acceptance bounds or an ambiguous question still fail.

    A valid region-restricted prompt

    "Which of the four labeled countries sits below the Economic Growth line and left of the Export Connectedness line? Answer with the three-letter country code (e.g., IND)."

    Region restrictions are fine when the boundaries are drawn lines on the chart; they fail only when the judgement is a hair's width.

    Worked examples for this criterion live in the appendix: see Bad Examples.

    Knowledge Check

    Which prompt is acceptable for the heat demand chart above?

    Trivial QuestionsAnswer Format Validity