Fit the model

  • Choose
  • Level 36 of 41
  • Fabric
  • Rack

Difficulty 2 of 5

A decode step reads 140 GB of weights. Each H100 holds 80 GB. Pick the design that finishes within 30 ms. The times include memory reads and data transfers.

Play level

Options

  • Tensor parallel
  • Pipeline
  • CPU offload
A full batch keeps each GPU busyArrow keys change the GPUs. Tap or Space changes the batch

Goals and limits

p99 latency
under 30 ms
Monthly cost
under 10,000 USD/mo

Concepts

How players did

Player results load from the server.

All levels