This morning, I sat down to write the latest edition of the Poka-post – a (hopefully) weekly blog promoting our core mission of error-proofing and reliability. Struggling for inspiration, I did what many of us now instinctively do when at an impasse and turned to my good friend ChatGPT. Chat, for short. The prompt was simple: provide a list of factual, historical anecdotes that illustrate the “make no mistakes” philosophy that underpins Poka-Yoke. Chat whirred away for a moment, before informing me that my prompt was “excellent” and rattling off a few examples. One in particular caught my attention.
It was a story of how, during WW2, Boeing engineers discovered that ground crews were accidentally swapping identical electrical connectors — leading to catastrophic control reversals. The fix wasn’t training; it was keying the connectors so they could only fit one way. Brilliant. What a perfect example of our company mission. There’s just one problem with it: It’s not true.
At least, not in the way presented. Wars are messy. It is possible, arguably probable, that something akin to what ChatGPT described did occur during the frantic design cycles that existed throughout the conflict. The problem is how an AI chatbot chooses to present this information. When I asked Chat for its source, it told me my question was excellent and then preceded to let me know that it didn’t have one. Great.
A quick google search (remember those?) revealed lots of examples of similar tunes, but none that played exactly like the song ChatGPT was singing. When I challenged it further, it told me I was excellent, then confessed that it had amalgamated different engineering folk stories into one killer anecdote, entirely of its own creation.
So why am I telling you this? Well, Poka-Yoke means error-proofing, and to us it means designing manufacturing solutions that make no mistake. ChatGPT, and many of the other large language models, often present themselves as part of this broader philosophy. As highly advanced, highly accurate tools, able to manufacture highly accurate solutions. But ChatGPT is designed to be convincing, not correct. It is important to remember that, while it may be a useful tool, it remains fallible. If you really want to make no mistakes, you must challenge it and question it the same way you would any other source of truth. Even if those sources don’t let you know how excellent you are.