Used GPU Buying Guide for AI

Used datacenter GPUs offer exceptional value for AI workloads. Cards that cost $10,000+ new can be found for $500-2,000, providing professional-grade features at consumer prices.

Key advantages: Large VRAM, ECC memory, designed for 24/7 operation, NVLink support on higher-end models.

Price: $300-500. The most affordable 24GB option.

Pros: Huge VRAM for the price, reliable, widely available.

Cons: Pascal architecture (2016), no FP16 acceleration, requires cooling solution.

Best for: Inference workloads, running large models on a budget.

Cooling: Needs aftermarket cooler - Noctua NH-L9a or similar works well.

Price: $800-1,500 depending on variant.

Pros: Tensor Cores, HBM2 memory, NVLink support, still competitive performance.

Cons: Higher power draw (300W), requires NVLink bridge for multi-GPU.

Best for: Training and inference, serious homelab builds.

Tip: 32GB SXM2 variant offers best value but needs compatible server board.

Price: $6,000-12,000 for 40GB, $12,000-18,000 for 80GB.

Pros: Modern architecture, excellent performance, FP64 support.

Cons: Still expensive, power hungry (400W), limited availability.

Best for: Production workloads, when you need A100 features at a discount.

Tip: PCIe variants are easier to deploy than SXM4.

Mining cards: Generally fine for AI, but check for physical damage.

Datacenter pulls: Usually well-maintained, but verify functionality.

Cooling: Most datacenter cards are passively cooled - budget for aftermarket solutions.

Compatibility: Check PCIe slot requirements and power connectors.

Warranty: Most used cards have no warranty - buy from reputable sellers.

Testing: Run stress tests and verify VRAM with tools like gpu-burn.

eBay: Largest selection, use buyer protection.

r/hardwareswap: Good deals, community reputation system.

Chinese sellers (AliExpress): Cheapest prices, longer shipping, some risk.

Enterprise liquidators: Best for bulk purchases.

Local data centers: Sometimes sell directly during upgrades.