Please use this identifier to cite or link to this item:
http://hdl.handle.net/10609/83667
Title: | A staging area for in-memory computing |
Author: | Santamaria Mateu, Pol |
Tutor: | Rodero, Ivan |
Abstract: | An in-memory staging area provides fast access to different applications. This research is based on evaluating the benefits of a distributed in-memory staging area applied to the field of Big data. With this purpose, a prototype is designed and proposed to verify the idea. Then, a working version comprised of the in-memory software Alluxio and the processing engine Apache Spark is deployed and evaluated. In particular, the work demonstrates the increase in performance resulting from updating the data in the in-memory staging instead of allocating space for new objects. The evaluation is conducted by running an analytic with Spark over a continuously changing dataset stored in Alluxio. The experiments reported a throughput increase of 10x when compared to storing information in a regular parallel filesystem, and an increase of 3x compared to the official deployment methodology. By updating the dataset, the Alluxio in-memory capacity stays constant at a low level compared to current deployments where its capacity decreases linearly, resulting in lower performance. |
Keywords: | in-memory database big data Alluxio |
Document type: | info:eu-repo/semantics/masterThesis |
Issue Date: | 15-Jul-2018 |
Publication license: | http://creativecommons.org/licenses/by-nc-nd/3.0/es/ |
Appears in Collections: | Trabajos finales de carrera, trabajos de investigación, etc. |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
psantamariamTFM0618memory.pdf | Memoria del TFM | 2,82 MB | Adobe PDF | View/Open |
Share:
This item is licensed under a Creative Commons License