Efficient and Lossless Moe Diffusion LLM Inference with I/O-Aware Expert Offload
1 points, 1 comments on Hacker News
🏷️ #inference
8 headlinesAx-engine: Native Apple Silicon ML inference runtime with a fast Rust core
2 points, 0 comments on Hacker News
GKE Inference Gateway delivers up to 92% faster AI responses
1 points, 0 comments on Hacker News
A system programmer's guide to LLM inference
2 points, 0 comments on Hacker News
Accelerating Feed-Forward Networks for Disaggregated AI Inference
1 points, 0 comments on Hacker News
Saving Money on Inference
3 points, 1 comments on Hacker News
Decentralized AI Inference Marketplace
1 points, 1 comments on Hacker News
Why LLM Inference Needs a New Kind of Router
7 points, 0 comments on Hacker News