Nate@nate_512

Seguir

Inworld Inference Gemma 4 26B latency

The 95th percentile is where this gets wild. Inworld's Gemma 4 26B hosting starts streaming in 0.31 seconds while the OpenRouter provider median sits at 2.42. That's 7.8x, and it's the kind of tail latency that decides whether an AI tutor or a game character feels alive or feels broken. A two second freeze kills the illusion, so pulling it down to a third of a second is a real shift. Median tells the same story from the other end: 0.17 seconds on Inworld against 0.61 for OpenRouter, which lands at 3.6x faster. Both numbers measure the wait until tokens actually start coming back, not total generation. Inworld Inference is the LLM inference API behind all this, built for devs shipping real-time consumer AI apps. Gemma isn't riding solo either. GLM, DeepSeek, and MiMo are hosted alongside it, and the whole lineup is reachable through an OpenAI compatible API. Istg the latency race is cooked in the best way

1h

Nate@nate_512

Seguir

Inworld Inference Gemma 4 26B latency

The 95th percentile is where this gets wild. Inworld's Gemma 4 26B hosting starts streaming in 0.31 seconds while the OpenRouter provider median sits at 2.42. That's 7.8x, and it's the kind of tail latency that decides whether an AI tutor or a game character feels alive or feels broken. A two second freeze kills the illusion, so pulling it down to a third of a second is a real shift. Median tells the same story from the other end: 0.17 seconds on Inworld against 0.61 for OpenRouter, which lands at 3.6x faster. Both numbers measure the wait until tokens actually start coming back, not total generation. Inworld Inference is the LLM inference API behind all this, built for devs shipping real-time consumer AI apps. Gemma isn't riding solo either. GLM, DeepSeek, and MiMo are hosted alongside it, and the whole lineup is reachable through an OpenAI compatible API. Istg the latency race is cooked in the best way

1h