Job containerization and orchestration to reduce TTC and operational costs

Title

Job containerization and orchestration to reduce TTC and operational costs

 Author

Barendse, Simon (TU Delft Electrical Engineering, Mathematics and Computer Science)
Spitzen, Jorick (TU Delft Electrical Engineering, Mathematics and Computer Science)

Contributor

Chen, Lydia Y. (graduation committee) ORCID 0000-0002-4228-6735

 Degree granting institution

Delft University of Technology

 Date

2019-03-25

 Abstract

Predictive analytics has become a common practice among retailers. Veneficus provides insights and assists clients in making decisions based on facts. The analyses required for this are often computationally intensive and take a lot of time to run. In order to provide a scalable, efficient and cost effective method for performing these analyses, the company must employ the latest techniques in cloud computing like Docker and Kubernetes. This project shows the process of setting up a autoscaling compute cluster, as well as an interface that data scientists with little to no knowledge of Kubernetes or other orchestration frameworks can use. By using the scale of a cloud based cluster and Greedy scheduling algorithms, we show that it is possible to perform large scale analytics in a way that Time to Completion is constant in the number of jobs being executed.

 To reference this document use:

http://resolver.tudelft.nl/uuid:f17df6eb-4138-4053-8b2e-b3b88fe0fd84

 Part of collection

Student theses

 Document type

bachelor thesis

 Rights

© 2019 Simon Barendse, Jorick Spitzen
Files
PDF Job_containerization_and_ ... _costs.pdf 2.15 MB
Close viewer