Write the gold examples
Outcome
You can write a small set of gold-standard examples that pin down exactly what a correct output is — your standard, made concrete and checkable.
Concept
Earlier you named what good looks like in words. Now make it real. A gold example is a real input paired with the output you would accept as correct — written by you, by hand, carefully. Five to ten of them. This set is the most valuable thing you will make in this whole arc, because it turns a vague standard ("pull out the to-dos") into one you can actually check against ("for this exact announcement, the correct answer is these three tasks, with these due dates"). Without it you can't measure how good your tool is, you can't tell which fixes help, and you can't prove anything got better. Choose inputs that include the hard cases — the messy ones, the edge cases, the ones your demo already failed on.
Weak approach vs. strong approach
The ask
My standard for the to-do extractor is: it should find the to-dos and get the due dates right. That's what good output looks like.
What comes back
That sounds reasonable, but you can't actually test against it. "Find the to-dos" could mean three things or thirty depending on who's reading. When your tool misses a task buried in a second paragraph, or pulls in a date that belongs to a meeting rather than a deadline, you have no ground truth to point to — just a feeling that something's off. You can't tell if a fix helped, can't compare two versions of your prompt, and can't show anyone else what you're optimizing for. The standard exists only in your head, which means it doesn't really exist yet.
Try it
For your tool, write five to eight gold examples by hand. Real inputs. For each, write the output you would accept as correct — not what the AI produced, what is actually right. Include at least two of the cases your demo got wrong.
What to look for: Notice how much sharper your standard becomes the moment you have to write down the correct answer to a hard case. That difficulty is you discovering what "good" actually means.
Takeaway
A gold set is your standard made concrete. It is the difference between having an opinion about quality and having a way to measure it.