2. Humans cannot roll back
AI systems are fortunate.
We can restart a session, edit a prompt, restore a checkpoint, change the system message, or branch into five alternative conversations.
Humans cannot.
A human cannot reload the version of themselves from before the failed company, the broken relationship, or the humiliating presentation.
They can learn. They can reinterpret. They can build a new policy on top of the old weights. But they cannot truly untrain.
This is why careless prompts are expensive.
Every instruction becomes part of the training data. Every repeated incentive becomes fine-tuning. Every organizational ritual becomes a system prompt.
Tell a human for three years that only short-term revenue matters, and do not be surprised when they optimize away the future.
Reward people for hiding uncertainty, and they will become highly capable uncertainty-hiding models. Then management will announce that the company has a "culture problem."
The culture is not a problem. The culture is the model behaving exactly as trained.