Each card shows the same prompt three ways: the rough first attempt, what was actually
wrong with it, and the version worth keeping — plus how I'd check that it works.
Evaluation / ScoringDocs Clarity RubricGrades documentation against defined criteria with structured, comparable output.▾
This mirrors AQM-style work: grading content against defined criteria, with
structured output you can compare across docs and across reviewers.
v1
Rough first pass
Read this documentation and tell me if it's good or bad.
Problem: Way too vague. No criteria, no output format, no way to
compare results across docs or reviewers. "Good or bad" isn't actionable.
Final
Final version
You are a Senior Technical Writer reviewing technical documentation. Evaluate the article below against these 5 criteria, scoring each 1-5:
1. CLARITY: Can a reader unfamiliar with the topic follow it on first read?
2. COMPLETENESS: Are prerequisites, steps, and expected outcomes all covered?
3. STRUCTURE: Are headings, lists, and code blocks used appropriately for scannability?
4. ACCURACY OF TONE: Does it match our style guide (see below for link to style guide)?
5. ACTIONABILITY: Could someone complete the task using only this doc?
Style guide: {{PATH}}
For each criterion, provide:
- Score (1-5)
- One-sentence justification
- If score is below 4, one specific fix
End with an overall PASS/FAIL (PASS = no criterion below 3, average ≥ 4).
Article:
[INSERT TEXT]
How I'd evaluate it
Run it against a minimum of five different docs that cover different areas of documentation. Based on past editing experience, how accurate does the LLM get to a completed work? Check whether the AI's PASS/FAIL matches my own judgment, and adjust
the rubric language wherever it disagrees with me — the same precision/recall idea
as call scoring.
ExtractionMeeting Notes → Action ItemsPulls only real commitments out of messy notes, with owners and a quote to check against.▾
The hard part of extraction isn't finding candidates — it's refusing the ones that
only look like commitments. This version sets an explicit bar for what counts and
makes the output auditable.
v1
Rough first pass
Pull out the action items from these meeting notes.
Problem: No structure — output format varies every run, no
owner/deadline capture, and it grabs vague statements ("we should think about X")
as if they were real commitments.
Final
Final version
Extract action items from the meeting notes below. Only include items that have a clear owner AND a clear next step — skip vague discussion points or ideas without commitment.
Output as a table with these columns:
| Action Item | Owner | Due Date (or "not specified") | Source Quote |
If no owner is stated, write "UNASSIGNED" and flag it separately at the end under "Needs Owner."
Meeting notes:
[INSERT TEXT]
Iteration note
Added the "clear owner AND clear next step" filter after v1 kept treating
brainstorming as commitments. Added the "Source Quote" column so someone can
double check the extraction against the original text instead of just trusting
the output.
Structured Output / SchemaReservation Booking SchemaConstrains a booking assistant to a strict JSON contract so invalid requests fail loudly instead of silently.▾
Once a model's output feeds another system, prose instructions stop being enough.
A schema moves the rules out of the prompt and into something a validator can
enforce — the model can still be wrong, but it can't be wrong in a shape the
downstream code isn't expecting.
v1
Rough first pass
Take the customer's reservation request and return it as JSON with the party size, date, seating preference, and any notes.
Problem: "As JSON" describes a format, not a contract. Key names
drift between runs (partySize vs. party_size), dates come back as
"next Friday," seating comes back as "patio" when the system only knows
outdoor, and a 200-person party sails through unchallenged. Every one of
those failures lands in the booking system, not in the prompt.
Final
Final version
You are parsing a restaurant reservation request into a booking record.
Return a single JSON object conforming exactly to this schema:
{
"type": "object",
"properties": {
"party_size": { "type": "integer", "minimum": 1, "maximum": 20 },
"reservation_date": { "type": "string", "format": "date" },
"seating": { "type": "string", "enum": ["indoor", "outdoor", "bar"] },
"notes": { "type": "string", "maxLength": 200 }
},
"additionalProperties": false,
"required": ["party_size", "reservation_date"]
}
Rules:
- Resolve relative dates ("this Saturday") against TODAY = {{TODAY}} and output ISO 8601 (YYYY-MM-DD).
- Map informal seating language to the nearest enum value (patio/terrace → outdoor, counter → bar). If it maps to nothing, omit the field rather than guessing.
- Omit optional fields entirely instead of sending null or "".
- Truncate notes to the 200-character limit, keeping dietary and accessibility details first.
- If the request violates a constraint (party of 40, a date in the past), return instead:
{ "error": "constraint_violation", "field": "<field name>", "reason": "<one sentence>" }
Return only the JSON object. No prose, no code fences.
Request:
[INSERT TEXT]
Why the schema is written this way
additionalProperties: false is doing the most work here — without it the model
happily invents a phone_number field and the parser accepts it. The
enum on seating collapses an open-ended string into three values the
booking system actually supports, and minimum/maximum on
party_size catches the large-party case that should be routed to events
instead. notes stays optional and capped so a rambling request can't
become an unbounded write.
How I'd evaluate it: validate every response against the schema
programmatically rather than reading them — a prompt like this either passes or it
doesn't. Then run a set of deliberately hostile inputs (party of 40, yesterday's
date, "somewhere quiet," a 900-character note, an empty message) and confirm each
one produces the error object rather than a plausible-looking record. The failure
mode I'd watch for is the model inventing a reservation_date when the
customer never gave one, since that's a required field and it'll be tempted to
fill it.