Evaluating and Improving Large-Scale Machine Learning Frameworks

Evaluating and Improving Large-Scale Machine Learning Frameworks

Graur, Dan (TU Delft Electrical Engineering, Mathematics and Computer Science)

Rellermeyer, Jan S. (mentor)
Alonso, Gustavo (mentor)
Epema, Dick (graduation committee)

Delft University of Technology

2019-09-10

Given the increasing popularity of Machine Learning, and the ever increasing need to solve larger and more complex learning challenges, it is unsurprising that numerous distributed learning strategies have been brought forward in recent years, along with many large scale Machine Learning frameworks. It is however unclear how well these strategies perform across different cluster and batch sizes, or what their hardware demands are, as there is little research in the public domain on this matter. Identifying the weaknesses and limitations of the parameter update strategies is, however, essential towards increasing the efficiency of large scale Machine Learning and making it commonplace. This thesis seeks to find the answers to these aforementioned issues, and provide evidence of the strategies’ limitations and the root causes behind them. To make the study possible, the thesis looks into particular implementations of the strategies within the TensorFlow and Caffe2 frameworks.

Machine Learning
Deep Learning
Tensorflow
Caffe2
Neural Networks
Scalability
Bottleneck Identification
Backpropagation
Large Scale
Distributed Systems
Clusters
Nodes
Limitations
Performance
Hardware
ResNet
Classification
Images
Parameter Update

http://resolver.tudelft.nl/uuid:ea9d655d-eabd-4409-9d06-0e10dd7124ef

Student theses

master thesis

© 2019 Dan Graur