Mixed Precision Training
A training method combining half-precision (FP16/BF16) and single-precision (FP32) to speed up GPU operations.
Mixed Precision Training uses half-precision formats for weight projections and gradient updates, switching to single-precision for numerical stability in sensitive calculations. This reduces training times and memory footprints.
Historical figures and technical concepts for informational purposes only. Not technical, professional, legal, or financial advice. Sources: Official Documentation.