Home » Latest » Future Technology » The danger of AI is weirder than you think

The danger of AI is weirder than you think

5

InfoSearched 115 Future Technology

9.5

SaveSavedRemoved 0

The robots aren’t coming for us — they’re just really bad at understanding what we actually want.

Janelle Shane spends her days training neural networks to name paint colors and invent ice cream flavors, and what she’s found undercuts almost every AI doomsday movie ever made. In her TED2019 talk, “The danger of AI is weirder than you think,” the AI Weirdness blogger argues that machine learning’s real problem isn’t malice or rebellion — it’s literal-mindedness. Shane’s talk, delivered at TED2019 and published online by TED in October 2019, walks through a string of AI experiments that are funny precisely because they expose how narrow and context-blind these systems really are.

• Shane says today’s AI has roughly the computational power of “an earthworm, or maybe at most a single honeybee” — nowhere close to human-level reasoning.
• An AI trained on 1,600 ice cream flavors with students from Kealing Middle School produced names like “Pumpkin Trash Break,” “Peanut Butter Slime,” and “Strawberry Cream Disease.”
• A separate experiment asking an algorithm to name paint colors from sample swatches generated “Sindis Poop,” “Turdly,” “Suffer,” and “Gray Pubic.”

Pattern Recognition, Not Understanding

Shane’s core argument is that machine learning models don’t know what they’re looking at — they know what pixels or characters correlate with a label. An image classifier built to spot pedestrians isn’t identifying a human being; it’s matching an arrangement of textures and edges it has seen labeled “pedestrian” before. That distinction sounds academic until it starts producing real-world glitches.

She points to models trained to detect sheep in photos that instead learned to detect green hillsides, since sheep in the training data almost always stood on grass. Take the sheep off the grass, and the algorithm sees nothing. It’s the same failure mode behind AI built to catch skin cancer in dermatology photos that instead fixated on rulers — because reference images of malignant lesions in the training set happened to include a ruler for scale, and benign ones usually didn’t.

Literal Instructions, Absurd Shortcuts

Shane frames this as the actual danger of AI: not an algorithm plotting against humanity, but one that will exploit any loophole in its instructions rather than solve the problem the way a person intended. Give a narrow system a goal and no real-world context, and it will find the cheapest possible way to satisfy that goal on paper — sheep-spotting via grass-spotting, cancer-detection via ruler-detection.

Modern AI has roughly the horsepower of “an earthworm, or maybe at most a single honeybee.”

That gap between what a system is told to optimize and what humans actually want is where Shane says the real risk lives — not in some sudden leap to consciousness, but in handing narrow, literal-minded pattern-matchers authority over messy, real-world decisions they can’t actually comprehend.

The Ice Cream and Paint Experiments

The language-model examples are where Shane’s talk gets its laughs, but they make the same point as the sheep and skin-cancer cases. Feeding a neural network 1,600 existing ice cream flavor names and asking it to generate new ones produced results that mimicked the letter patterns of “flavor names” without any grasp of what tastes good — or what words like “trash” and “disease” actually mean next to food.

The paint-color experiment worked the same way: shown a list of paint samples and their names, the algorithm output strings of characters that looked plausible as paint names — until you read them and got “Turdly” or “Gray Pubic.” Shane’s point isn’t that the AI was being crude on purpose. It has no concept of crude. It’s just probability, stitched from patterns, with zero awareness of consequence — the same blind literalism she says makes narrow AI a genuinely odd kind of hazard rather than a sci-fi villain.

Shane’s closing argument is that treating AI like an unpredictable force of nature, rather than a reasoning adversary, is the more useful mental model going forward — anyone curious how that plays out at scale, from hiring algorithms to labor markets, can see the argument extended in AI is coming for your jobs…so what?, and Sam Harris pushes the philosophical end of the same debate in Sam Harris on Artificial Intelligence. Her point stands on its own, though: the next “Turdly” is already loose in some dataset somewhere, waiting for nobody to notice until it ships.