Investigation about mixed precision in NVIDIA (APEX.AMP)


Review NN

Implement neural network

Three tasks applying mixed precision in NN

Pasted image 20221018144838.png

Implement in PyTorch

Training Neural Networks with Mixed Precision

How to choose factor

Training Mixed Precision User Guide#^hcxfmn1qod

Learn FP16 and Tensor cores in NVIDIA GPUs

Learn SASS Semantics -- (FP16) Half instructions store pattern

Use GPUFPX to create tight constraints.