Answer Format Validity

Where to find the full answer-format guidelines

General requirements:

• Use Short Answer by default. Aim for about 90% Short Answer and 10% MCQ across submitted annotations. The format rules below still apply to both types.
• The prompt must dictate exactly one acceptable answer format.
• The Answer field contains only the final answer — no solving steps, justification, or filler.
• There must be a single coherent final answer. Multiple separate questions with answers tacked on to each other are not acceptable. Answers in the form of a ratio or list may be fine, as long as there is clear logic and cohesion between the different parts of the answer.
• Do not allow answer variance or tolerance ranges that accept multiple final answers. For example, “Answer with a percentage rounded to the nearest whole percent, any value within 5% is acceptable” is unacceptable because the range permits several answers. Specify rounding that yields one verifiable final answer instead. Chartography exception: if the task category says Chartography, an estimated value read from gridlines may use a stated accepted range or tolerance, and the model's wrong answer must fall outside it. All other tasks keep this rule.
• True/False and Yes/No questions are not allowed.
• Color is never a short answer — write it as an MCQ instead.

Short answer:

• Short and deterministic. Text answers are 1–5 words.
• The question must leave at least four possible answers open. A choice among two panels or three legend entries is a guess, not reasoning, and is not accepted. If fewer than four answers are possible, first widen the question so it works as Short Answer. If using MCQ instead, provide at least four valid options and keep the overall 90% Short Answer / 10% MCQ target in mind.
• Every short-answer prompt must include an example answer written exactly as (e.g., X), and X must never match the Rewrite Answer. X must also be a value the question could actually produce: a country that appears on the chart, a number inside the possible range, a month on the axis.
• Use exact visible chart text for labels, methods, models, categories, panels, or legend entries.
• For numeric answers, use visible labels, ticks, gridlines, or clearly readable marks, and state the rounding rule wherever math is involved.
• If the value is not exact, ask for an approximate value with reasonable visible precision and a rounding rule that yields one verifiable final answer. Do not offer a tolerance range that accepts multiple values. Exception: Chartography-category tasks may state an accepted range for an estimated value.
• Thousands separators, dollar signs, percentage signs, and units must be requested explicitly in the example answer.
• If multiple answers are required, separate them with commas and state the required ordering.

MCQ:

• At least four options labeled A. B. C. D., each 1–5 words.
• No "All of the above" or "None of the above".
• Include an "Answer with a single letter (e.g., B)." instruction, where the example letter is one of the options but not the correct one.
• The Rewrite Answer is only the bare letter. C passes; C., C. 4, Option C, or 4 fail.