@nate_512Istg the part that actually gets me about EdgeRock-4B is that Rock Matrix kept the KV cache flat. 72 MiB per sequence, and it does not care whether you are sitting at a short prompt or pushing all the way out to 256K tokens. That is the whole trick, and the open-source 4B model pulls it off by fusing recurrent state with local attention and Rock Memory Anchors (RMA). Learned KV records get parked in a fixed-capacity bank, so the system can still dig up info from way earlier in the context even after it has fallen out of the local attention window. Normal LLMs just keep stacking cache as the context grows, which is exactly the wall this thing sidesteps. Now the numbers, because they are cooked in the best way. Output throughput lands at 12,124 tokens per second on a single 80GB Hopper GPU with 512 concurrent sequences running. Compare that to Qwen3.5-4B and you are looking at 6.42 times the output. Cache usage drops 99.1% at 256K context versus the traditional growth approach. Long generation holds 233.7 tokens per second even when the model chews through 128K of input and then writes another 128K of output. And it still reasons. 82.9% mean pass@1 on AIME 2024, 2026. The strongest bounded-state baseline they evaluated only hit 18.8%. Long context, a cache that never balloons, and reasoning that keeps up, all from an open-source 4B model

@nate_512Istg the part that actually gets me about EdgeRock-4B is that Rock Matrix kept the KV cache flat. 72 MiB per sequence, and it does not care whether you are sitting at a short prompt or pushing all the way out to 256K tokens. That is the whole trick, and the open-source 4B model pulls it off by fusing recurrent state with local attention and Rock Memory Anchors (RMA). Learned KV records get parked in a fixed-capacity bank, so the system can still dig up info from way earlier in the context even after it has fallen out of the local attention window. Normal LLMs just keep stacking cache as the context grows, which is exactly the wall this thing sidesteps. Now the numbers, because they are cooked in the best way. Output throughput lands at 12,124 tokens per second on a single 80GB Hopper GPU with 512 concurrent sequences running. Compare that to Qwen3.5-4B and you are looking at 6.42 times the output. Cache usage drops 99.1% at 256K context versus the traditional growth approach. Long generation holds 233.7 tokens per second even when the model chews through 128K of input and then writes another 128K of output. And it still reasons. 82.9% mean pass@1 on AIME 2024, 2026. The strongest bounded-state baseline they evaluated only hit 18.8%. Long context, a cache that never balloons, and reasoning that keeps up, all from an open-source 4B model