Search

Showing top 20 results for "Software behavior changes"

All sources anthropic.com 20

People also ask

Why does reward hacking lead to worse behaviors?

These results are an example of generalization. Generalization occurs in benign ways in the training of all AI models: training a model to solve math problems turns out to make it better at, say, planning vacations and a whole range of other useful tasks. But as we show here, it can happen for more concerning behaviors, too: when we accidentally reward the model for one kind of “bad thing” (cheating), this makes it more likely to do other “bad things” (deceiving, aligning itself with malicious actors, planning to exfiltrate its own weights, and more). As in previous work studying emergent misa

From shortcuts to sabotage: natural emergent misalignment from reward hacking

To show you the most relevant results, we’ve omitted some entries very similar to those already shown. Repeat the search with the omitted results included.