The first number I track when evaluating a free model route is not the public benchmark delta. It is the semantic failure count in a 100-case golden set. A public benchmark can be flat while your application's specific tool calls, JSON schemas, and failure states move.
Source: [Dev.to](https://dev.to/kongkong1/measure-a-free-model-route-with-a-golden-set-not-a-leaderboard-g5d)