NormalFloat 4 (NF4)
An information-theoretically optimal quantile quantization data type designed to compress parameters.
NF4 is a specialized 4-bit quantization format that maps model parameters to a normal distribution. It provides better preservation of model quality than standard 4-bit integer quantization, making it a key component of QLoRA.
Historical figures and technical concepts for informational purposes only. Not technical, professional, legal, or financial advice. Sources: Official Documentation.