Best open-weight LLMs for the RTX 4090 (24GB)

With ~24 GB available for model weights, 1 open-weight model runs locally on a RTX 4090 (24GB) — each at the best quantization that fits. The 4 that need more memory are listed below with the cheapest provider to rent them instead. VRAM figures are estimates.Updated May 2026.

Different machine? Pick another rig or enter your VRAM →

VRAM ≈ parameters × bytes-per-param × 1.2 overhead, at the best available quantization — see the methodology docs. Estimates, not a guarantee.

Other hardware