Entropy4Cloud: Using Entropy-Based Complexity To Optimize Cloud Service Resource Management Computational Intelligence for Cloud Computing

Huikai Chen, Frank Wang, Na Helian

Research output: Contribution to journalArticlepeer-review

9 Citations (Scopus)
78 Downloads (Pure)

Abstract

In cloud service resource management system, complexity limits the system’s ability to better satisfy the application’s QoS requirements, e.g. cost budget, average response time and reliability. Numerousness, diversity, variety, uncertainty, etc. are some of the complexity factors which lead to the variation between expected plan and actual running performance of cloud applications. In this paper, after defining the complexity clearly, we identify the origin of complexity in cloud service resource management system through the study of ”Local Activity Principle”. In order to manage complexity, an Entropy-based methodology is presented to use which covers identifying, measuring, analysing and controlling (avoid and reduce) of complexity. Finally, we implement such idea in a popular cloud engine, Apache Spark, for running Analysis as a Service (AaaS). Experiments demonstrate that the new, Entropy-based resource management approach can significantly improve the performance of Spark applications. Compare with the Fair Scheduler in Apache Spark, our proposed Entropy Scheduler is able to reduce overall cost by 23%, improve the average service response time by 15% - 20% and minimized the standard deviation of service response time by 30% - 45%.
Original languageEnglish
Pages (from-to)13-24
Number of pages12
JournalIEEE Transactions on Emerging Topics in Computational Intelligence
Volume2
Issue number1
DOIs
Publication statusPublished - 23 Jan 2018

Fingerprint

Dive into the research topics of 'Entropy4Cloud: Using Entropy-Based Complexity To Optimize Cloud Service Resource Management Computational Intelligence for Cloud Computing'. Together they form a unique fingerprint.

Cite this