hey everyone, i decided to make a vLLM plugin that implements the Star-KV paper. the results are quiet promising with a decode kernel thats almost twice as fast as FA2 in higher batch sizes.

Source: [Hacker News](https://github.com/Tenosra/Proxima)

Sponsored