|
Almost Timely News: 🗞️ How AI Detection Works (2026-06-07) :: View in Browser
 |
The Big Plug
👉 My new course, GEO 201 on competitive GEO measurement, is now for sale.
Content Authenticity Statement
99% of this week’s newsletter content was generated by me, the human. You will see a small piece by Google Gemini in the opening section. Learn why this kind of disclosure is a good idea and might be required for anyone doing business in any capacity with the EU in the near future.
Watch This Newsletter On YouTube 📺
Click here for the video 📺 version of this newsletter on YouTube »
Click here for an MP3 audio 🎧 only version »
What’s On My Mind: How AI Detection Works
This week, let’s talk AI detection and how it works. My buddy Becca pinged me earlier this week with this request:
“I need your wizard secrets on AI detectors 😭 no matter how I write I get flagged! Even voice to text! HOW DO YOU DO IT?! Your posts are perfect and so well written which usually means AI would flag them but they don’t?“
After a lengthy exchange, I asked whether she’d had Claude Code try to solve the problem, and Claude told her no, it wouldn’t help her reverse engineer an AI detector.
And it was at this point where I cracked my knuckles, looked askance at the false god, and said to the Twisting Nether, “BET.”
Here’s the story of how I reverse engeineered an AI detector and what I found.
Part 1: Writing is Code
If you recall from a couple months ago, I wrote about some of the ways we can measure writing. Writing is code, at its heart. It’s not purely creative, because there are clear rules to writing. As I say in my keynotes, I can speak the words “I’m spilling the tea” and you know what they mean, even if you don’t understand the slang.
On the other hand, if I say, “Tea the spilling I’m”, you might be forgiven for thinking my cat walked on the autocorrect. Why? Because language has clear structure and rules. In North American English, that usually means (and yes, there are tons of exceptions, the English language is practically made of exceptions) subject-verb-object as a word order. In fact, there’s like a dozen or so languages on planet Earth (out of 1,377) that are object-verb-subject like Aiwoo and Urarina. The majority of languages are subject-object-verb (”I’m the tea spilling”).
So language has rules, order, and predictability. It’s code. In fact, code is writing and writing is code. When you ask someone who’s working in Python or Rust what they’re doing, more often than not they’ll say they’re writing code. Not making code, not generating code, not typing code, but writing it.
In turn, that means it can be measured and quantified. There’s an entire field about this, natural language processing (NLP), which predates generative AI by decades. Folks who have been in the AI space since the Obama years likely remember all the ups and downs of NLP during that time, from IBM Watson winning Jeopardy to the AI winter of the mid 2010s.
Part 2: What You Can Quantify
Before we can get to reverse engineering an AI detector, we have to start with reverse engineering writing. What are the different measures of writing we can objectively analyze? As I mentioned just above, NLP has been at this practice for decades. There are over 60 different measures of writing and ways to measure writing mechanistically, deterministically. (Aka there’s a concrete number at the end of the analysis)
Let’s take a look at just a handful; I’m working on a course for Trust Insights that will go into these in much greater depth, but this is a start.
Sentence length. How long are your sentences? Sometimes we write super long sentences - I do this a ton when I’m thinking out loud or when I’m dictating; those issues of this newsletter that have super long run-on sentences are a dead giveaway that I dictated the sentence rather than typed it.
Sentence length variation. Imagine you analyzed a blog post you write and you tabulated all the sentence lengths. For some of my content, my sentence length variation looks like this: 14-12-4-17-8-14. Sometimes I write super short sentences, sometimes I write very long ones.
Sentence length patterns. Those short and long sentence patterns are not random. If you read my writing, a very short sentence is usually followed by a very long one.
Why? If you look at how I write, especially in places like LinkedIn, I’m very fond of the short/long pattern. I’ll write something like this:
(Expository paragraph)
(Short emphasis)
(Long explanation)
That’s just my personal style. Other people write what folks have amusingly titled “broetry” (mostly dudes, hence the name) who write as though it was literally transcribed from William Shatner’s Captain Kirk:
You know…
…what…
…I mean!
Here’s the thing about sentence patterns - our writing style is predicated in part on how we use sentences, along with how we use line breaks, punctuations, em dashes, the works. For example, I never, ever use the em dash and I never have. Why? Not because I care about AI, but because I learned to type on a manual typewriter. An em dash is two keystrokes. An en dash is one. Instead of em dashes, I write with a space/en dash/space pattern. Again, that’s a personal style thing.
But the key takeaway here is that our writing style, as individuals, is a combination of very distinct patterns.
Part 3: How AI Detectors Work
Now, let’s talk about AI writing. All generative AI, all AI period, is probability. All of it. All AI models are probability engines and they produce their predictions as the next word, the next sentence, the next thing in order. Unguided, they produce the highest likely probability for an item in a sequence.
Let’s watch this example of KoboldCPP, which is a language model server serving up the Skyfall model, and watch it generate words. You’ll see the text on the left, and you will see the actual server making predictions on the right.