LLM inference in pure C++
The pioneer of local GGUF inference: runs quantized models everywhere from laptops to servers.
Georgi Gerganov's C/C++ inference engine that put LLMs on laptops: GGUF quantized models on CPU/Metal/CUDA. The substrate beneath Ollama and LM Studio
Who it's for
Developers running LLM inference locally on consumer hardware with quantised GGUF models.
Official links
Keep exploring
Concepts in the glossary
1 in catalog
llama.cpp implements
1 in catalog