Modeling Data Reuse in Deep Neural Networks by Taking Data-Types into Cognizance

Jha, Nandan Kumar and Mittal, Sparsh (2021) Modeling Data Reuse in Deep Neural Networks by Taking Data-Types into Cognizance. IEEE Transactions on Computers, 70 (9). pp. 1526-1538. ISSN 0018-9340

[img] Text
IEEE_Transactions.pdf - Published Version
Available under License Creative Commons Attribution.

Download (1MB)


In recent years, researchers have focused on reducing the model size and number of computations (measured as 'multiply-accumulate' or MAC operations) of DNNs. The energy consumption of a DNN depends on both the number of MAC operations and the energy efficiency of each MAC operation. The former can be estimated at design time; however, the latter depends on the intricate data reuse patterns and underlying hardware architecture. Hence, estimating it at design time is challenging. This article shows that the conventional approach to estimate the data reuse, viz. arithmetic intensity, does not always correctly estimate the degree of data reuse in DNNs since it gives equal importance to all the data types. We propose a novel model, termed 'data type aware weighted arithmetic intensity' ($DI$DI), which accounts for the unequal importance of different data types in DNNs. We evaluate our model on 25 state-of-the-art DNNs on two GPUs. We show that our model accurately models data-reuse for all possible data reuse patterns for different types of convolution and different types of layers. We show that our model is a better indicator of the energy efficiency of DNNs. We also show its generality using the central limit theorem. © 1968-2012 IEEE.

[error in script]
IITH Creators:
IITH CreatorsORCiD
Item Type: Article
Additional Information: This work was supported by Semiconductor Research Corporation under the project 2020-IR-2972.
Uncontrolled Keywords: arithmetic intensity; Deep neural networks (DNNs); energy-efficiency; roofline model
Subjects: Computer science
Divisions: Department of Computer Science & Engineering
Depositing User: . LibTrainee 2021
Date Deposited: 08 Sep 2022 15:07
Last Modified: 08 Sep 2022 15:07
Publisher URL:
OA policy:
Related URLs:

    Actions (login required)

    View Item View Item
    Statistics for RAIITH ePrint 10491 Statistics for this ePrint Item