Activating frequencies

Activating frequencies: Exploring non-linearities in the Fourier domain

Uijens, Wouter (TU Delft Electrical Engineering, Mathematics and Computer Science; TU Delft Pattern Recognition and Bioinformatics)

van Gemert, Jan (mentor)
Reinders, Marcel (graduation committee)
Hildebrandt, Klaus (graduation committee)

Delft University of Technology

2018-04-13

Convolutional Neural Networks (CNNs) are achieving state of the art performance in computer vision. One downside of CNNs is their computational complexity. One way to make CNNs more computational efficient is by implementing their convolutions in the frequency domain, using Fast Fourier Transforms (FFTs). This has as a consequence that most computational time in modern CNNs is spent in those FFTs.
If all components of the CNN algorithm could be implemented in the frequency domain, it would no longer be necessary to go back at all to the spatial domain. Most components of the CNN do have alternatives that can be implemented in the frequency domain, however one crucial component doesn’t: the activation function. This thesis is the first published study into activation functions in the frequency domain.
In this thesis several potential candidates for an activation function that works directly in the frequency domain are studied. Furthermore, some theoretical contributions on this subject are made.

Machine Learning
Fourier Transform
Activation Function

http://resolver.tudelft.nl/uuid:b6dfdac6-691d-44a4-bacb-645df5cfdeaf

Student theses

master thesis

© 2018 Wouter Uijens