Stochastic KV Routing: Enabling Adaptive Depth-Wise Cache Sharing
1 points, 0 comments on Hacker News
🏷️ #routing
3 headlinesModel routing is a fix for AI overspending, a problem for OpenAI and Anthropic
3 points, 0 comments on Hacker News
Trace-Based Adaptive Cost-Efficient Routing
2 points, 1 comments on Hacker News