Quantisation: Why Low-Precision Weights Keep Full-Precision Accuracy A modern language model stores tens of billions of weights, each one a 32-bit floating point number. Cut every weight to 4 bits and you shrink the model eightfold, halve the memory bandwidth, and run it on hardware that could n...
Source: [Dev.to](https://dev.to/umeshtharukaofficial/quantisation-why-low-precision-weights-keep-full-precision-accuracy-26d)