ERA

Download the full-sized PDF of Cooper: Expedite Batch Data Dissemination in Computer Clusters with Coded Permutation GossipsDownload the full-sized PDF

Analytics

Share

Permanent link (DOI): https://doi.org/10.7939/R3M95F

Download

Export to: EndNote  |  Zotero  |  Mendeley

Communities

This file is in the following communities:

Graduate Studies and Research, Faculty of

Collections

This file is in the following collections:

Theses and Dissertations

Cooper: Expedite Batch Data Dissemination in Computer Clusters with Coded Permutation Gossips Open Access

Descriptions

Other title
Subject/Keyword
cloud computing
data dissemination
coded permutation gossips
Type of item
Thesis
Degree grantor
University of Alberta
Author or creator
Liu, Yan
Supervisor and department
Niu, Di (Department of Electrical and Computer Engineering)
Examining committee member and department
Khabbazian, Majid (Department of Electrical and Computer Engineering)
Ardakani, Masoud (Department of Electrical and Computer Engineering)
Department
Department of Electrical and Computer Engineering
Specialization
computer engineering
Date accepted
2015-01-05T09:50:12Z
Graduation date
2015-06
Degree
Master of Science
Degree level
Master's
Abstract
Data transfers happen frequently in server clusters for software and application deployment, and in parallel computing clusters to transmit parameters in batches among servers between computation stages. This thesis presents Cooper, an optimized prototype system to speedup multi-batch data transfers among a cluster of servers, leveraging a theoretically proven optimal algorithm called "permutation gossip" which employs randomly permuted node connections to best utilize bandwidth and random linear code to maximize the useful information transmitted. By chunking the file into a proper number of blocks, we present a pipelining technique to parallelize the coding operation and network transfer on the process level, realizing the theoretically promised benefits of random linear codes. More importantly, for batch-based or multiple transfers, we propose priority-based scheduling algorithms to overlap the transfers of different batches, which further reduce the transfer finish time of each batch, while only delaying the first batch for a constant time. We present an asynchronous and distributed prototype implementation of Cooper and deploy it on Amazon EC2 for evaluation. Based on results from real experiments, we show that Cooper can significantly speedup data transfers and reduce redundant transmissions in server clusters as compared to state-of-the-art content distribution tools, including BitTorrent and an optimized random-block transfer strategy based on buffer negotiation in a wide range of practical settings.
Language
English
DOI
doi:10.7939/R3M95F
Rights
Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.
Citation for previous publication

File Details

Date Uploaded
Date Modified
2015-06-15T07:03:29.000+00:00
Audit Status
Audits have not yet been run on this file.
Characterization
File format: pdf (PDF/A)
Mime type: application/pdf
File size: 852067
Last modified: 2015:10:21 22:26:27-06:00
Filename: Liu_Yan_201501_MSc.pdf
Original checksum: b41d01e7f3d33df1e4bbbbd2eab3dd9d
Activity of users you follow
User Activity Date