vLLM
High-throughput open-source LLM inference with PagedAttention
vLLM Project (UC Berkeley / community) · Infrastructure Launched Jun 2023
Quick Answer
vLLM is high-throughput open-source llm inference with pagedattention, made by vLLM Project (UC Berkeley / community). It is free and open source. Key uses: Self-hosted LLM serving, High-throughput inference, Production LLM deployment.
Last updated:
- Users
- 30K+ GitHub stars; deployed at major AI labs and enterprises
- Valuation
- $1B (seed, a16z, 2025)
What is vLLM?
vLLM is an open-source, high-throughput and memory-efficient inference engine for large language models, built by UC Berkeley. Its PagedAttention algorithm manages GPU memory like an OS manages RAM, enabling 2-4× more throughput than standard HuggingFace inference. Provides an OpenAI-compatible server for drop-in deployment.
What's new in vLLM in 2026?
vLLM 0.6 (2024) adds speculative decoding, lora serving, prefix caching; 30K+ GitHub stars; vLLM Production Stack for Kubernetes; used by major AI companies for model serving; vLLM team raised $100M seed at $1B valuation (Andreessen Horowitz, 2025)
How much does vLLM cost?
Open-Source
$0
- Apache 2.0 license
- All features
- Community support
- Commercial use
What can you do with vLLM?
- Self-hosted LLM serving
- High-throughput inference
- Production LLM deployment
- Multi-GPU serving
- Cost-efficient inference
Pros
- 2-4× throughput vs HuggingFace
- OpenAI-compatible API
- Continuous batching
- Multi-GPU support
- All major open models
Cons
- Requires ML expertise
- GPU hardware needed
- No GUI
- Setup complexity
Specs
Underlying Models
- Llama 4
- DeepSeek R1
- Mistral
- Gemma
- Qwen
- 200+ models
Platforms
- Linux (CUDA/ROCm)
- AWS
- GCP
- Azure
- On-premise
Integrations
- Hugging Face
- LangChain
- LlamaIndex
- Kubernetes
- Ray
- Docker
Ideal For
- MLOps engineers
- Platform teams
- High-volume inference
- GPU cluster operators
Tags
- Open-Source
- LLM Inference
- High-Throughput
- Self-Hosted
- PagedAttention
- GPU
What are the best alternatives to vLLM?
Compare vLLM Head-to-Head
3
Frequently Asked Questions about vLLM
How much does vLLM cost?
vLLM pricing: Open-Source at $0.
Is vLLM free?
Yes, vLLM is fully free and open source.
What is vLLM used for?
vLLM is used for: Self-hosted LLM serving, High-throughput inference, Production LLM deployment, Multi-GPU serving, Cost-efficient inference.
What are the best alternatives to vLLM?
Top alternatives to vLLM include Replicate, Groq API, Ollama.
What platforms does vLLM support?
vLLM is available on: Linux (CUDA/ROCm), AWS, GCP, Azure, On-premise.