runNburn is an Apache-2. 0 Rust inference engine for quantized GGUF models that are too big for your fast memory. The core idea: weights stay file-backed (mmap), host residency stays under an explicit byte budget (--ram-budget), and GPU caches are sized from detected free/total VRAM — never from...
Source: [Hacker News](https://github.com/coderredlab/runNburn)