I built an eval harness to prove an LLM worked. It proved the opposite.

I set out to have a language model classify integration failures. I built an evaluation harness to prove it worked. The harness proved it wasn't worth using.

Source: [Dev.to](https://dev.to/fissible/i-built-an-eval-harness-to-prove-an-llm-worked-it-proved-the-opposite-327m)