The thing to know about LLMs

(I'm mostly describing LLMs here, but many of the principles apply in their own way to other forms of generative AI.)

LLMs generate text. But there's a world of difference between "generating text" and "thinking", or even just "reporting information". LLMs generate text that is statistically plausible based on the patterns discovered in the mountains of text they were trained on. The text they were trained on, and the text they generate, are just words to the LLM; it learns how they are used together and in what patterns, but it doesn't understand any of them. "Thinking", or even "reporting information", implies a way to represent that information meaningfully, some kind of cognitive capacity, but LLMs don't have that, they just have words with no meaning and patterns for how to combine them. To the extent that their output appears to reflect thought or carry meaningful information, it's only because the training text was meaningful and some of that meaning was accidentally reproduced by similar word combinations in the generated text. In other words, they aren't as smart as they look; they're really just parlor tricks.

Here are some major reasons not to use LLMs:

Even though they appear to be sane, reasonable, and smart, they are in fact grotesquely unreliable, frequently asserting falsehoods, confidently and often doubling down on their hallucinations.
They cause intellectual atrophy in those who use them.
They are being used to replace human workers, even though they are unreliable.
When used to augment human workers, the human is relegated to verifying, proofreading, code review, etc. -- mind-numbing tasks that a human cannot maintain focus on, especially when the output is acceptable most of the time. The AI took the fun creative part of the job and left the human to do the drudgery.
Their training data is stolen from human creators who neither consented nor were compensated.
They are the cause of the current explosion in the construction of monstrous data centers.
They waste obscene amounts of water and electricity, and some are being powered by new fossil fuel generating capacity that had previously been retired.
They are destabilizing our economy by creating a massive and insane investment bubble which will cause a depression when it inevitably pops.
They are destabilizing our politics by making it easy for anyone to make convincing fake content.
They are used by criminals to create convincing phishing and fraud campaigns.
They are created, controlled and pushed by fascist billionaires, and those billionaires are the ones who really benefit from them.

Author Lionel Dricot has published his personal policy on generative AI, and I adopt it as mine also.