Kog Targets 10x Faster LLM Inference by September on Existing GPUs
Updated
Updated · TechCrunch · Aug 14
Kog Targets 10x Faster LLM Inference by September on Existing GPUs
1 articles · Updated · TechCrunch · Aug 14
Summary
September is Kog’s target to show a major LLM running at 10x speed on standard datacenter GPUs, a milestone the French startup says is crucial for proving its software-only approach and raising a Series A.
200 business leads followed Kog’s May tech preview, as enterprises looked for relief from AI workflow delays and inference costs on hardware they already own, including Nvidia H200 and AMD MI300X systems.
3,000 tokens per second was Kog’s headline demo result, but it came from the open-sourced 2 billion-parameter Laneformer 2B model, leaving the company to prove the same gains can extend to much larger LLMs.
11 people make up Kog’s team, and its low-level optimization work can take weeks or months per GPU, limiting how many chips it can support even as it argues newer GPUs still have untapped memory-bandwidth potential.
France-backed Kog is pitching that deeper software tuning can stretch conventional GPUs further, joining a broader push to speed inference without custom chips as Europe seeks more AI hardware and model sovereignty.