How to evaluate an LLM agent: evals, golden sets, and LLM-as-judge

Short answer You can't unit-test an LLM to correctness, because the same input can take a different path on the next run. Evals are the test suite for probabilistic systems: a scored, repeatable check of whether the system reached an acceptable outcome across runs, not whether it returned one ex...

Source: [Dev.to](https://dev.to/aiarch_wibo/how-to-evaluate-an-llm-agent-evals-golden-sets-and-llm-as-judge-1e2g)