Substack's AI Detector: Same Old Story, Same Old Flaws

Yes, Substack has rolled out a new tool that, according to them, should help us tell the difference between an article penned by a real person and one spat out by artificial intelligence. Every post, note, and comment over a hundred words is now under the digital magnifying glass of this "eye." The idea is noble, like most AI-related ideas, but the execution is a bit more complicated, like most AI-related executions.

The problem is, this digital detective, fresh on the platform, seems to have the same vision problems as its predecessors. It's not the first time we've seen this show. Remember DEV.to? They also had a detector that muddled things up. And guess what? Substack hit the mark, or rather, missed the mark, in the exact same way.

The reason is simple: these tools often look for patterns and regularities that AI tends to produce, but sometimes humans can also write in a "regular" or predictable way. Substack introduced its AI text detector this week, applying it to all content over 100 words, but early tests already show it mistakenly labels human-written texts as machine-generated. It's a bit like a truffle-sniffing dog barking at a potato just because it has dirt on it.

Similar detectors, like the one tested by DEV.to, have already proven to fail in accurately distinguishing between AI-generated and human-written content. Just a small edit to an AI-generated text, even changing a few words or sentence structures, can completely bypass these systems. It's a cat-and-mouse game where the mouse always learns new tricks before the cat takes it seriously. Perhaps we should question whether it's worth investing so much in these tools, or if it's better to focus on content quality regardless of who writes it.

For readers, the promise of "guaranteed human" content might not be kept, leading to a false sense of security. For writers, however, the situation is trickier. You could spend hours crafting a brilliant piece, only to see it flagged as "bot-made" by an overly zealous algorithm. The main challenge of these tools is the generation of false positives, penalizing legitimate writers.

This is not only frustrating but could also undermine writers' trust in the platform. If an algorithm can decide your work isn't authentic, what value does your voice hold? Perhaps it's time to accept that AI is here to stay, and the real battle isn't against generated texts, but for quality and originality, regardless of who or what "helped" them come to life. In short, a big headache for everyone.