An exact-match cache misses "how do I reverse a list in Python" when it has already answered "python list reverse". A semantic cache doesn't: it embeds the prompt, finds the closest one it has seen, and replays that answer instead of calling the model. Fewer API calls, lower latency, same answers.
Source: [Dev.to](https://dev.to/tonytonycoder11/kmemo-a-semantic-cache-for-llm-calls-that-refuses-to-serve-you-the-wrong-answer-54h7)