Feed-Forward Network (FFN)
A fully-connected neural network layer processing token vectors independently after attention calculations.
The FFN in a transformer consists of two linear transformations separated by a non-linear activation. It operates on each token coordinate independently, projecting features to expand network capacity.
Historical figures and technical concepts for informational purposes only. Not technical, professional, legal, or financial advice. Sources: Official Documentation.