Softmax Saturation
A mathematical state where extremely large input logits result in near-zero gradients during training.
Softmax Saturation occurs when values passed to a softmax layer are excessively large, causing output probabilities to cluster near 0 or 1. This flattens the gradient curve, stalling backpropagation unless scaled by factor roots.
Historical figures and technical concepts for informational purposes only. Not technical, professional, legal, or financial advice. Sources: Official Documentation.