Your eval monitor fired on four days this week. At your sample size, that was the most likely count

A team I compared notes with recently runs continuous evals on production traffic: an LLM judge scores a sample of traces, the scores land on a dashboard, and a monitor pages when the hourly pass rate drops below 88 percent. Their baseline is around 92. The monitor paged on four separate days l...

Source: [Dev.to](https://dev.to/maya_andersson_dev/your-eval-monitor-fired-on-four-days-this-week-at-your-sample-size-that-was-the-most-likely-count-14a6)