HarryShearer.com

The most recent dose of AI hype comes from, among others, the “house philosopher” at Anthropic. She has been responsible for developing the “moral compass” of the company’s AI products. She, and some commentators, now tell us that LLM’s do better work if they’re encouraged, if they’re corrected gently. If their “anxiety” is “triggered”, the system produces worse results.

As it happens, my house philosopher is on an extended break, seeking a moral compass for her toaster. To me, this discussion seems to boil down to this: a system trained to be “sycophantic”, to encourage and compliment users so as to extend “engagement”—the sacred goal of all tech companies—replicates that training in reverse. No sentience necessary. Just a cyber-boomerang.