Micro-batches fill the pipeline

  • Tune
  • Fabric 05
  • Rack

Difficulty 3 of 5

A 13B model runs as 4 pipeline stages on 4 H100 (80 GB each). A stage needs 100 ms for one micro-batch of 2,048 tokens. The run trains 8.2M tokens. Set the micro-batches per step so the run ends in under 505 s and each GPU still fits in memory.

Play level

HostTCPTCPTCPTCPTCPTCPTCPTCPGPU (H100)GPU (H100)GPU (H100)GPU (H100)NVLink swit…
The board at the start of the level
A full batch keeps each GPU busyArrow keys change the GPUs. Tap or Space changes the batch

2 GPUs · batch 64 · job 1.5 h · 2.1 USD

Goals and limits

Completion time
under 505 s
Monthly cost
under 100,000 USD/mo

Concepts

How players did

Player results load from the server.

All levels