Atlas: Open Source Inference Engine (Rust + CUDA)

Source Excerpt

Overview

Atlas is a pure Rust + CUDA inference engine that claims 3.1x faster performance than vLLM with a dramatically smaller footprint (~2.5GB vs vLLM's 20+GB).

Key Specifications

• Performance: 111.4 tok/s vs vLLM's 37.5 tok/s on Qwen3.5-35B [Unverified]
• Image size: ~2.5GB (vs vLLM's 20+GB) [Unverified]
• Cold start: <2 minutes [Unverified]
• Dependencies: No Python, no PyTorch
• Model support: Qwen, Gemma, Nemotron, Mistral, MiniMax
• Features: MTP speculative decoding

Why It Matters

vLLM is the de facto standard for Large Language Model serving. A genuine alternative with 3.1x speedup and 8x smaller image size addresses real pain points: deployment complexity, resource constraints, and cold start times. The Rust-only approach eliminates the Python/PyTorch dependency chain that makes vLLM deployments heavy.

The Qwen3.5-35B benchmark is directly relevant to our current model family.

Actionability

HIGH — Direct replacement candidate for current inference stack. The benchmarks are compelling and the model support includes Qwen. Should be evaluated in our environment.

Status

Open Source, available at atlasinference.io.

Sources

• https://atlasinference.io/
• @AtlasInference, May 6, 2026