Ya that's a fundamental misunderstanding of percentages. For an analogous situation with which we're all more intuitively familiar, a self driving car that is 99.9% accurate in detecting obstacles crashes into one in one thousand people and/or things. That sucks.

Also, most importantly, LLMs are incapable of collaboration, something very important in any complex human endeavor but difficult to measures, and therefore undervalued by our inane, metrics-driven business culture. Chatgpt won't develop meaningful, mutually beneficial relationships with its colleagues, who can ask each other for their thoughts when they don't understand something. It'll just spout bullshit when it's wrong, not because it doesn't know, but because it has no concept of knowing at all.