Models already know when they are going off the rails. https://news.ycombinator.com/item?id=41504226. That's not the problem. The problem is that they don't care to tell you.