Delay differential neural networks

Anumasa, S. and Srijith, P.K. (2021) Delay differential neural networks. In: 6th International Conference on Machine Learning Technologies, 23 April 2021 through 25 April 2021, Virtual, Online.

[img] Text
Delay_Differential_Neural_Networks.pdf

Download (713kB)

Abstract

Neural ordinary differential equations (NODEs) treat computation of intermediate feature vectors as trajectories of ordinary differential equation parameterized by a neural network. In this paper, we propose a novel model, delay differential neural networks (DDNN), inspired by delay differential equations (DDEs). The proposed model considers the derivative of the hidden feature vector as a function of the current feature vector and past feature vectors (history) unlike only the current feature vector in the case of NODE. The function is modelled as a neural network and consequently, it leads to continuous depth alternatives to recent ResNet variants. For training DDNNs, we discuss a memory-efficient adjoint method for computing gradients and back-propagate through the network. DDNN improves the data efficiency of NODE by further reducing the number of parameters without affecting the generalization performance. Experiments conducted on real-world image classification datasets such as cifar10 and cifar100 to show the effectiveness of the proposed model. © 2021 ACM.

[error in script]
IITH Creators:
IITH CreatorsORCiD
Srijith, P Khttps://orcid.org/0000-0002-2820-0835
Item Type: Conference or Workshop Item (Paper)
Additional Information: ISBN:978-145038940-2
Uncontrolled Keywords: Adjoint method, Deep learning, Delay differential equations
Subjects: Computer science
Computer science > Wireless Networks
Divisions: Department of Computer Science & Engineering
Depositing User: Mrs Haseena VKKM
Date Deposited: 28 Jun 2022 09:04
Last Modified: 28 Jun 2022 09:04
URI: http://raiith.iith.ac.in/id/eprint/9367
Publisher URL: https://doi.org/10.1145/3468891.3468908
Related URLs:

Actions (login required)

View Item View Item
Statistics for RAIITH ePrint 9367 Statistics for this ePrint Item