Have you ever been on the receiveing end of someone sending you a ChatGPT and Claude transcript a note saying:
“Please read. Very insightful.”?
And then you do read the transcript… only to see short, barely literate inputs from the user generating pages and pages of AI output? That you’re then expected to read with the reverence of the Dead Sea Scrolls?
It’s low key insulting when someone does this in an area related to your domain expertise. Just because an LLM can spit out words (many, many words) in response to a half baked thought, it does not mean those words are valuable or coherent.
And the expectation that someone can take 90 seconds to create something that they then expect you to spend hours reviewing and responding to? Nah, that’s not cool.
A New Era for AI Writing: And a Model to Do It Better
Sophie Alpert from Clay wrote an absolutely kick ass AI Writing Policy that adresses the above pheonomon. It should honestly be required reading for all professionals in this era. She lays out 4 key points:
- You must stand behind every idea and sentence in your docs.
- Writing is thinking.
- More time should be spent authoring a document than consuming it.
- Longer is not better.
Yes. Yes. Yes. Yes.
A thousand times YES.
There is some major nuance in these points, and Sophie gives great examples in her write up. It’s a quick read, and it’s worth a look here.
Checking Your Thinking Against an AI Output
I couldn’t agree more with the basic premise that Sophie starts with. If you’re using AI generated words, you have to be ready to stand behind those words. Every sentence. What is said, and how it’s said.
So how do you get to a place where you can confidently validate AI generated content? How do you “QA it,” for lack of a better term?
There are established patterns for QAing code, configuration, web assets, etc. But we’re in somewhat of a new world when it comes to QAing “thoughts” generated by AI.
This is my checklist of what I look for when I evaluate AI content:
- Does the piece have a clear point?
- Do I actually agree with the conclusions?
- Do I agree with what’s stressed, highlighted, or emphasized? (Particularly headlines & subheadlines. This is valuable real estate in your document, and what you put here makes a statement.)
- Are there any pieces of this thinking that contradict each other?
- Quotes, citations, names and places — are they real? Are they right?
- Check the numbers. Are stats, dates, and numerical facts accurate? (AI is still occasionally a complete idiot at math. It’s dangerous.)
- What are the counterarguments? What was not considered in this piecce?
- Is the appropriate level of certainty expressed? (“Full confidence” is AI’s default mode, but it shouldn’t be yours.)
- What could be taken out of context?
- Who is my audience, and is it written with them in mind?
- What is the logical next step or action that this content prompts, and is that clear?
- Do any images included make sense for the content?
- Do links and cited sources actually work / go to the right place?
- Are there spots where the tone, voice, or style need to be tightened up?
If there’s ANY part of the document that seems a little “off” — trust your gut. It could be a hallucination, or you could be noticing a logical inconsistency that your audience will pick up on and ask you about.
If you can’t defend a sentence or slide with your whole brain and heart, with reasons why you believe it to be true…. get it out of the document. Or spend more time thinking about what you’re actually trying to say.
Faster Writing is Usually Not Better Writing
Doing all of the above takes time. It’s supposed to.
AI speeds this up, sure. And that’s a beautiful thing.
But when you’re generating written word, slide decks & content, you’re doing more than just “making the thing.” You’re clarifying and strengthening an idea. You have to engage in thinking about the problem and understanding it to get to an actionable output.
It’s way more than just words on a page — it’s all of the thought, the processing, and the work that it took to get to final product.
That’s the work.
That’s the value.
That’s the part that’s uniquely human in the era we find ourselves in.