Edward Pyne, Lillian Pentecost, Udit Gupta, Gu-Yeon Wei, David Brooks
Harvard University
Technical Talk: Quantifying the Impact of Data Encoding on DNN Fault Tolerance
Abstract:
Characterizing DNN fault tolerance enables codesign of models and accelerators for improved power consumption and performance. In this work, we quantify the fault tolerance of DNNs to (1) demonstrate that data encoding format impacts fault tolerance by up to 10x and (2) quantify the variation in measured fault tolerance across DNNs trained to convergence with identical training hyperparameters. This variance is reduced with alternate weight encodings. Both results have impacts on making robust design decisions to maintain model accuracy. These studies are enabled by an updated open-source fault injection framework for DNNs designed to quantify the impact of faults on both weight and activation values in both training and inference.