we obviously don't need to understand everything about how LLMs work to safely deploy them (and this probably isn't practically possible), but having additional evidence from mech interp is valuable especially when all other forms of monitoring are becoming less and less useful

the issue i have with this post is that it's comparing LLMs to humans. that is like comparing a hydrogen bomb to a coughing baby, because it's ignoring the scale of the intelligence and deployment

so here's a little story:
imagine taking a convicted felon from another planet, who doesn't really understand what being human means, then making him superhuman in math and coding and expert level in most other domains, giving him 10x human thinking and interaction speed, and then releasing him

of course, first you put him through multiple psychological evaluations and he passes 99% of the test scenarios.
that's good right?

well, you also know that he's way smarter than you and that he's perfectly capable of deceiving you and the tests, and that when he's exposed to certain situations will commit crimes

whatever.
let's release him anyway.

oh, but you don't just release him once. you first clone him 10 million times and then release him.

the best part: each clone gets access to someone's home, workplace or computer with access to private files, the local network and databases

now this will sound crazy to you. but maybe, just maybe, you would want some more proof that he is law abiding, friendly and reliable before making him even smarter, cloning him into a billion copies, making every copy 10x faster, and handing them even more access and control