Number Systems for Deep Neural Network Architectures

Book excerpt: This book provides readers a comprehensive introduction to alternative number systems for more efficient representations of Deep Neural Network (DNN) data. Various number systems (conventional/unconventional) exploited for DNNs are discussed, including Floating Point (FP), Fixed Point (FXP), Logarithmic Number System (LNS), Residue Number System (RNS), Block Floating Point Number System (BFP), Dynamic Fixed-Point Number System (DFXP) and Posit Number System (PNS). The authors explore the impact of these number systems on the performance and hardware design of DNNs, highlighting the challenges associated with each number system and various solutions that are proposed for addressing them.