Por favor, use este identificador para citar o enlazar este ítem:
http://hdl.handle.net/10609/147615
Title: | Study of the Feasibility of Serverless Access Transparency for Python Multiprocessing Applications |
Author: | Finol, Gerard Arjona, Aitor |
Tutor: | Garcia Lopez, Pedro |
Abstract: | Access transparency means that both local and remote resources are accessed using identical operations. Transparency simpli es the complexity of programming a distributed system because the system is perceived as a whole rather than a collection of independent components. With access transparency, we can treat disaggregated compute, storage, and memory resources as if they were a single monolithic machine. This would considerably simplify the creation and execution of parallel applications in the Cloud in a scalable manner. In this work, we evaluate the feasibility of access transparency over state-of-the-art Cloud disaggregated resources. We propose an alternative implementation of Python's multiprocessing API that transparently runs distributed processes on serverless functions and that leverages disaggregated in-memory storage to maintain the shared state of processes consistent and mediate their communication. To evaluate transparency, we have used four parallel stateful applications intended to be executed locally (Uber Research's Evolution Strategies, Baselines-AI's Proximal Policy Optimization, Pandaral·lel's dataframe and ScikitLearn's Hyperparameter tuning), and, without changing the code, we have scaled them with serverless technology. We compare execution time and scalability of the same application running over disaggregated resources using our library, with the single-machine Python libraries in a large VM. Despite the higher latency and lower throughput of communication, we achieve comparable results and we observe that the applications can continue to scale beyond VM limited resources leading to a better speedup and parallelism. |
Keywords: | transparency disaggregation serverless |
Document type: | info:eu-repo/semantics/masterThesis |
Issue Date: | 29-Jun-2021 |
Publication license: | http://creativecommons.org/licenses/by/3.0/es/ |
Appears in Collections: | Bachelor thesis, research projects, etc. |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
gfinol_aarjonapeFMDP0621report.pdf | Report of FMDP | 4,6 MB | Adobe PDF | View/Open |
Comparte:
Este ítem está sujeto a una licencia Creative Commons Licencia Creative Commons