Providing Geo-Elasticity in Geographically Distributed Clouds
To appear in ACM Transactions on Internet Technology (TOIT'17)
Geographically distributed cloud platforms are well suited for serving a geographically diverse user base. However traditional cloud provisioning mechanisms that make local scaling decisions are not adequate for delivering best possible performance for modern web applications that observe both temporal and spatial workload fluctuations. In this paper, we propose GeoScale, a system that provides geo-elasticity by combining model-driven proactive and agile reactive provisioning approaches. GeoScale can dynamically provision server capacity at any location based on workload dynamics. We conduct a detailed evaluation of GeoScale on Amazon’s geo-distributed cloud, and show up to 40% improvement in the 95th percentile response time when compared to traditional elasticity techniques.
On the Feasibility of Cloud-Based SDN Controllers for Residential Networks
2017 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN'17)
Residential networks are home to increasingly diverse devices, including embedded devices that are part of the Internet of Things phenomenon, leading to new management and security challenges. However, current residential solutions that rely on customer premises equipment (CPE), which often remains deployed in homes for years without updates or maintenance, are not evolving to keep up with these emerging demands. Recently, researchers have proposed to outsource the tasks of managing and securing residential networks to cloud-based security services by leveraging software-defined networking (SDN). However, the use of cloud-based infrastructure may have performance implications.
In this paper, we measure the performance impact and perception of a residential SDN using a cloud-based controller through two measurement studies. First, we recruit 270 residential users located across the United States to measure residential latency to cloud providers. Our measurements suggest the cloud controller architecture provides 90% of end-users with acceptable performance with judiciously selected public cloud locations. When evaluating web page loading times of popular domains, which are particularly latency-sensitive, we found an increase of a few seconds at the median. However, optimizations could reduce this overhead for top websites in practice.
Towards Efficient Deep Inference for Mobile Applications
Mobile applications are benefiting significantly from the advancement in deep learning,
e.g. providing new features. Given a trained deep learning model, applications usually
need to perform a series of matrix operations based on the input data,
in order to infer possible output values. Because of model computation
complexity and increased model sizes, those trained models are usually hosted
in the cloud. When mobile apps need to utilize those models,
they will have to send input data over the network.
While cloud-based deep learning can provide reasonable response time for mobile apps,
it also restricts the use case scenarios, e.g. mobile apps need to have access to network.
With mobile specific deep learning optimizations, it is now possible to employ device-based inference.
However, because mobile hardware, e.g. GPU and memory size, can be very different and limited when
compared to desktop counterpart, it is important to understand the feasibility of this new device-based
deep learning inference architecture. In this paper,
we empirically evaluate the inference efficiency of three Convolutional Neural Networks using a
benchmark Android application we developed. Based on our application-driven analysis,
we have identified several performance bottlenecks for mobile applications powered by
on-device deep learning inference.
Performance and Cost Considerations for Providing Geo-Elasticity in Database Clouds
To appear in Transactions on Autonomous and Adaptive Systems (TAAS'17)
Online applications that serve global workload have become a norm and those applications are experiencing
not only temporal but also spatial workload variations. In addition, more applications are hosting their
backend tiers separately for benefits such as ease of management. To provision for such applications,
traditional elasticity approaches that only consider temporal workload dynamics and assume well-provisioned backends are insufficient.
Instead, in this paper, we propose a new type of provisioning mechanisms---geo-elasticity,
by utilizing distributed clouds with different locations.
Centered this idea, we build a system called DBScale that tracks geographic variations
in the workload to dynamically provision database replicas at different cloud locations across the globe.
Our geo-elastic provisioning approach comprises a regression-based model that infers database query workload
from spatially distributed front-end workload, a two-node open queueing network model that estimates the capacity of
databases serving both CPU and I/O-intensive query workloads, and greedy algorithms for selecting best cloud locations
based on latency and cost. We implement a prototype of our DBScale system on Amazon EC2’s distributed cloud.
Our experiments with our prototype show up to a 66% improvement in response time when compared to local elasticity approaches.
Latency-aware Virtual Desktops Optimization in Distributed Clouds
Multimedia Systems (MMSJ'17)
Distributed clouds offer a choice of data center
locations for providers to host their applications. In this paper
we consider distributed clouds that host virtual desktops
which are then accessed by users through remote desktop
protocols. Virtual desktops have different levels of latency-sensitivity,
primarily determined by the actual applications
running and affected by the end users’ locations. In the scenario
of mobile users, even switching between 3G and WiFi
networks affects the latency sensitivity. We design VMShadow,
a system to automatically optimize the location and performance
of latency-sensitive VMs in the cloud. VMShadow
performs black-box fingerprinting of a VM’s network traffic
to infer the latency-sensitivity and employs both ILP and
greedy heuristic based algorithms to move highly latency-sensitive
VMs to cloud sites that are closer to their end users.
VMShadow employs a WAN-based live migration and a new
network connection migration protocol to ensure that the
VM migration and subsequent changes to the VM’s network
address are transparent to end-users. We implement a prototype
of VMShadow in a nested hypervisor and demonstrate
its effectiveness for optimizing the performance of
VM-based desktops in the cloud. Our experiments on a private
as well as the public EC2 cloud show that VMShadow
is able to discriminate between latency-sensitive and insensitive
desktop VMs and judiciously moves only those that
will benefit the most from the migration. For desktop VMs
with video activity, VMShadow improves VNC’s refresh rate
by 90% by migrating virtual desktop to the closer location.
Transcontinental remote desktop migrations only take about
4 minutes and our connection migration proxy imposes 13µs
overhead per packet.
Managing Risk in a Derivative IaaS Cloud
IEEE Transactions on Parallel and Distributed Systems (TPDS'17)
Infrastructure-as-a-Service (IaaS) cloud platforms rent computing
resources with different cost and availability tradeoffs. For example,
users may acquire virtual machines (VMs) in the spot market that are
cheap, but can be unilaterally terminated by the cloud operator. Because
of this revocation risk, spot servers have been conventionally used for
delay and risk tolerant batch jobs. In this paper, we develop risk mitigation
policies which allow even interactive applications to run on spot servers.
Our System, SpotCheck is a derivative cloud platform, and provides
the illusion of an IaaS platform that offers always-available VMs on
demand for a cost near that of spot servers, and supports unmodified
applications. SpotCheck’s design combines virtualization-based mechanisms
for fault-tolerance, and bidding and server selection policies for
managing the risk and cost. We implement SpotCheck on EC2 and show
that it i) provides nested VMs with 99.9989% availability, ii) achieves
nearly 5× cost savings compared to using on-demand VMs, and iii)
eliminates any risk of losing VM state.