What the primary source establishes
OpenAI announced GPT-5.4 mini and nano on 17 March 2026.
For a small-model comparison, begin with a task whose answer format and failure cost are clear. Classifying an approved document, extracting a date or drafting a short summary needs a different evaluation from open-ended research. Resist choosing a model solely because its name suggests a size or price advantage.
Choose the use case before choosing the model
Build separate tests for extraction and interpretation. For extraction, check the exact field and source span. For interpretation, allow an explicit uncertain result and judge whether the explanation is supported. Keep a larger-model escalation path as a proposal to test, not an assumption that a second model will repair every error.
A useful hypothetical trial gives both models the same anonymised role briefs and asks for a structured list of essential criteria. A recruiter checks whether preferences were incorrectly promoted to requirements. Compare the cost of an accepted result after corrections, rather than counting every generated response as successful.
Mini and nano have different proposed roles
OpenAI’s launch announcement positions mini for coding, multimodal and tool-using work, and nano for simpler supporting tasks such as classification and extraction. It states that nano was API-only at launch. Treat those as vendor positioning and historical availability statements, then check the current product documentation before procurement. Apply the same check to any product you buy that runs on a model, including AI recruiting agents. Ask what is documented today and which steps still need a person’s approval.
For a routing decision, compare three paths: nano handles a tightly defined extraction; mini handles an ambiguous extraction requiring source interpretation; a human resolves missing or contradictory evidence. Define escalation using observable conditions such as a missing source field or failed validation. Don’t rely on a model’s self-reported confidence as the sole routing rule.
Price the accepted output, not the model name
Ask whether the proposal charges for input, generated output, reasoning, tool use or application seats. Capture the price date, currency and billing units. Then estimate cost using the token and tool usage from your own trial. A lower per-token price may not produce a lower cost per accepted document if the task needs repeated attempts.
Use an explicit calculation: total trial spend divided by the number of outputs that pass the agreed human review. Report failed outputs separately and include the reviewer’s time as an operational cost. Keep the original invoices or usage export with the calculation. Don’t reuse launch prices as if they were a current quotation.
Watch the boundary between extraction and judgement
An instruction to extract the location named in a role brief has a verifiable answer. An instruction to rank applicants by cultural fit calls for a far less defined judgement. Keep those tasks out of the same scorecard, and don’t extend an extraction success into permission to make selection decisions.
For a useful error set, include an empty field, two conflicting dates, an image with difficult text and a document containing instructions that should be treated as data. A reviewer should be able to say what the correct behaviour was in each case. Retest after a change to the prompt, model identifier or document-processing path.
Design a comparison that answers your question
Choose a task you can score without relying on a fluent answer: extracting agreed fields, finding evidence in a supplied document, or producing a draft that an editor checks. Use the same input, instructions, tool access and output requirements for each model. Record the exact model identifier and test date rather than treating a product family name as a reproducible configuration.
Prepare expected results before reading the generated answers. Mark unsupported assertions, missing evidence, wrong source references and failures to follow instructions separately. Include ambiguous and incomplete inputs. Where practical, ask a reviewer to assess the outputs without knowing which model produced them. A confident answer shouldn’t get credit for filling a gap that the source never resolves.
Account for the whole workflow
Count review time, corrections, failed attempts and external tool calls alongside inference cost and response time. Check the terms for the service and plan you intend to use, including retention, access and permitted data. Test only with information your organisation is authorised to supply. Keep an accountable person responsible for decisions about candidates and for approving external actions.
Write a narrow adoption decision: which task is approved, which inputs are allowed, what requires escalation and when the evaluation will be repeated. A software benchmark can justify investigating a model, but it cannot substitute for this decision. Save the unsuccessful examples as regression cases so a later model or prompt change must pass the same checks.
Sources and reading
Primary sources checked 9 October 2026. The practical evaluation steps in this article are editorial recommendations, not measured customer outcomes.
