A Resiliency-First Approach to Distributed DAG Computations

Title

A Resiliency-First Approach to Distributed DAG Computations

 Author

Leliveld, Dorus (TU Delft Electrical Engineering, Mathematics and Computer Science; TU Delft Quantum & Computer Engineering)

Contributor

Hofstee, Peter (mentor)

Degree granting institution

Delft University of Technology

 Programme

Computer Engineering

 Date

2017-07-05

 Abstract

A framework is introduced for computations with transformations on immutable data. Inspiration is taken from Apache Spark, however the model of computation is generalized from an emphasis on narrow and wide dependencies, to an arbitrary set of transformations that form a directed acyclic graph (DAG). A distributed scheduling algorithm is developed with resiliency mechanisms that can account for stopping failure. Furthermore some properties of the system are derived. Finally future work is discussed showing there is fertile ground for further research and development to extend this work.

 Subject

Distributed Computing
Resiliency
Distributed Scheduling
Load Balancing

 To reference this document use:

http://resolver.tudelft.nl/uuid:566b7b54-b40b-45fa-85c2-debaaa8098d4

 Part of collection

Student theses

 Document type

master thesis

 Rights

© 2017 Dorus Leliveld
Files
PDF ResilientFirstApproachDag ... tation.pdf 188.92 KB
Close viewer