Check whether AI improved the whole job
Look at the result, checking time and what people learned before deciding whether to keep using AI.
Alongside People · 2 min read ·
A draft arrives in two minutes. That tells you it was quick to generate. To tell whether AI helped, look at the result, the checking and the person using it.
This guide gives you a simple comparison to try with a colleague. You can use it after drafting a handover, preparing a meeting or improving a piece of writing.
Compare the whole job
Here is an invented example. A customer brief used to take 30 minutes to prepare and 10 to check. With AI, drafting takes 5 minutes, but checking and correcting it takes 35. The whole job still takes 40 minutes.
The figures do not tell the whole story either. Perhaps the colleague now understands the customer's question better. Perhaps checking a confident but inaccurate draft is more tiring. Ask what changed.
If you keep using the tool, include setup and upkeep. Do not call a quicker first draft a saving if the work simply moved to someone else.
Notice what the person learned
Suppose the colleague wants to distinguish an agreed date from a suggestion. Ask them to explain the date in the brief and show its source. That tells you more than whether the finished paragraph sounds confident.
Paul writes about verification: software completing a task is different from the result serving its purpose. The person receiving the work can help judge its usefulness.
Try one comparison
After the next practice task, ask each other:
- What was clearer or more useful?
- What needed checking or correcting, and how much effort did that take?
- What would we keep or change next time?
A short conversation is enough. Use invented or approved material, and leave confidential information out of shared notes.
You should come away with a reason to keep, change or stop the approach. A few examples are useful learning, not proof that it will work everywhere. Using less AI can be a sensible result.
Want help reviewing an AI experiment? Tell us what you tried.