University of Hertfordshire

By the same authors


View graph of relations
Original languageEnglish
Number of pages12
Pages (from-to)13-24
JournalIEEE Transactions on Emerging Topics in Computational Intelligence
Journal publication date23 Jan 2018
StatePublished - 23 Jan 2018


In cloud service resource management system, complexity limits the system’s ability to better satisfy the application’s QoS requirements, e.g. cost budget, average response time and reliability. Numerousness, diversity, variety, uncertainty, etc. are some of the complexity factors which lead to the variation between expected plan and actual running performance of cloud applications. In this paper, after defining the complexity clearly, we identify the origin of complexity in cloud service resource management system through the study of ”Local Activity Principle”. In order to manage complexity, an Entropy-based methodology is presented to use which covers identifying, measuring, analysing and controlling (avoid and reduce) of complexity. Finally, we implement such idea in a popular cloud engine, Apache Spark, for running Analysis as a Service (AaaS). Experiments demonstrate that the new, Entropy-based resource management approach can significantly improve the performance of Spark applications. Compare with the Fair Scheduler in Apache Spark, our proposed Entropy Scheduler is able to reduce overall cost by 23%, improve the average service response time by 15% - 20% and minimized the standard deviation of service response time by 30% - 45%.


© 2017 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission.

ID: 12067325