The enStratus Blog

Deploying a Redundant Cloud Infrastructure

If the latest events in the AWS EC2 cloud east region have taught cloud architects and administrators anything, it is that the requirement for redundancy and disaster recovery cannot be overstated. enStratus enables cloud architects to span architecture across different regions in the same cloud and even across clouds altogether.

Since the AWS East region is the most widely used cloud region, many companies found themselves in a situation where their entire infrastructure was provisioned in that region. Even with redundancy across availability zones, the outage was devastating enough to bring down multiple availability zones, taking with it large portions of applications running there.

 

Using Multiple Regions

enStratus automation features offer an elegant and intuitive process to plan for eventualities such as region failure in a cloud. Using the deployment manager, architects can add another region as needed or balance their application across regions as a matter of procedure. 

When the EC2 East region outage hit, many companies found themselves scrambling to bring up applications in the EC2 West or other region.

The prospect of trying to move data that was "locked" in the East region was very problematic for administrators because in many cases backups were stored in S3 in that same region. enStratus allows for disaster recovery storage in multiple regions and/or multiple clouds. 

 

Backup Management

enStratus disaster recovery options automate the process of exporting backups off-site. What off-site means to the architect is open to a number of configurable options
including private storage such as swift or walrus.

Backups can be encrypted and moved at predetermined intervals, ensuring data is secure in transit and there when you need it.

Also, the enStratus application is hosted outside the cloud by enStratus or on-premise by our customers to ensure cloud governance and reliability.

This past week was a trying week for cloud workers, but tools like enStratus exist to plan for and maintain continuity of operations independent of the cloud used.

Check out our Webinar that we hosted May 4th to learn more: How To Deliver Cloud Infrastructure Redundancy.

 

 

What Is Autoscaling...And Why Does It Matter?

‘Autoscaling,’ as the name implies, is the ability to have an application dynamically take advantage of additional compute resources as required and then release those resources when they are not required. 

Some examples include:

• A consumer facing web application might “scale out” (i.e. add extra servers) the web tier (and maybe even the local balancing tier) of an application architecture during periods of high traffic and then scale back for quieter periods.
• An analytical application might scale out the servers performing in calculation process – using 20 machines for 1 hour, rather than 1 machine for 20 hours.

This concept of ‘autoscaling’ has become the poster child for cloud adoption because it actively illustrates the power and flexibility offered by cloud infrastructure. At an individual application level, a good part of the business case for cloud rests on only paying for the compute resource actually required.

Without autoscaling, an application deployed in the cloud is no more effective in its utilization of resources than in a traditional, physical, or virtualized environment and will have the same ‘over provisioning’ required to meet peak demand.

With autoscaling, an application can reduce its consumption to match actual demand and increase efficiency dramatically, particularly for an application with a high variance between peak and off-peak demand.

The only public or private cloud platform that provides any application autoscaling function is AWS ….and that is a command line tool.

If you want to autoscale an application, you need a cloud management tool like enStratus that understands the application architecture and can automatically request additional resources and connect them into the target architecture according to your defined scaling rules.

For example, enStratus allows you to scale the different tiers of an application based on basic metrics such as CPU utilization, or on custom business function such as transaction per second or concurrent active users. Additionally, you can define autoscaling across regions or even across clouds; so for example, you could scale your web tier up to 10 instances within your cloud.com private cloud, but then add further web instance, in EC2, while maintaining the database tier within your private cloud.

And enStratus can autoscale within any of the public or private clouds that we support -- Amazon Web Services, AT&T Synaptic Storage, Cloud.com, CloudSigma, EMC Atmos, Eucalyptus, Google Storage, GoGrid, OpenStack, Rackspace, ReliaCloud, ServerExpress, Terremark, VMware and Windows Azure.

We have found autoscaling adds value to a surprising number of application deployments, not just the highly volatile applications that are so often discussed. A little bit of attention can often reduce your cloud bill by a significant sum.

Automated EBS RAID and Encryption with enStratus

We spend so much time talking about the governance features of enStratus that we often fail to talk about some of the simple operational advantages that enStratus provides. Perhaps the most overlooked operational feature in enStratus is the ability to automatically attach, format, and mount RAID volumes through the enStratus console with the option to have those volumes automatically encrypted.

When you launch an instance in enStratus in the Amazon cloud or any other cloud with an EBS-like concept, enStratus includes a tab that lets you define volumes to be created and attached upon launch. In defining those volumes, you can indicate that you would like to arrange multiple volumes into a RAID and optionally encrypt the file system built on the mounted volume. enStratus takes care of provisioning the block volumes, attaching them to the launched instances, constructing the RAID (if desired), encrypting the file system (if desired), formatting the file system, and mounting the volume(s).

You can customize the software RAID and file system encryption by extending the enStratus scripts on your machine image. By default, we use mdadm to handle software RAID and luks for file system encryption on Unix. We don’t currently provide default options on Windows, but you can extend the scripts to use whatever software RAID and encryption you’d like.

Configuring the RAID and encryption is just part of the story. enStratus manages your encryption keys and takes care of making consistent volume snapshots across your RAID.

On the encryption front, enStratus will generate a unique encryption key to use for encrypting your file system based on the number of bits required by the encryption system you are using. The key is then stored securely outside of the cloud in our credential management system. It only makes brief, well-orchestrated appearances in the cloud when your volumes need to be mounted.

enStratus tracks which volumes/RAID arrays were encrypted with which keys. It can therefore make snapshots of the encrypted volumes, create new volumes, and attach the new volumes automatically to other instances without you having to remember which volumes were encrypted with which keys. All you need are the proper access rights to the instance and volumes in question.

Finally, when it comes time to make backups of a RAID, enStratus enables you to snapshot all volumes attached to a given virtual machine together. In the Servers screen, you can elect to snapshot the server. enStratus will tell then agent on that server to lock all services running on it, lock its file systems, and then enStratus will execute a snapshot of all volumes. The resulting snapshots enable you to reconstruct the RAID in a consistent fashion at a later date.

Best Practice: The only reason to use RAID in the cloud is to get a performance boost in your disk I/O. You should thus only ever use RAID0. Other RAID levels are useless in the cloud. They provide greater resiliency over a RAID0, but erase all speed benefits.

Managing Access in the Cloud

A manager recently asked me about managing user access to his cloud infrastructure. User access in the cloud presents an interesting challenge because of the nature of cloud accounts and cloud management platforms.

I generally divide the access problem into two separate parts. The first is access to the cloud account itself, which typically requires providing the cloud account credentials to each user requiring access. The second is access to individually running servers, which requires a system administrator to create accounts for each user. A third layer is added by the enStratus cloud management platform which I will discuss in a moment.

Each of these access levels carries with it security implications. In the first case, it is unwise to spread credentials of any kind among users even if they are trusted because of the tendency for shared  credentials to be shared even further than the trusted circle.

enStratus removes the requirement for sharing cloud credentials by introducing a third layer of user management. As the cloud management platform credentials are held outside of the cloud in an encrypted database.  Using enStratus, administrators can focus on access rights and permissions without having to worry about losing control of account credentials.

User managment in enStratus is role-based and familiar to managers and system administrators. Securely adding remote access to individual servers to developers or administrators can be done quickly and reliably using the enStratus console. I grant remote access to a single user on a single server in this manner on a routine basis.

Integration into existing LDAP and Active Directory offerings enables seamless integration of user management in cloud infrastructure into traditional datacenters. When a user account is removed in Active Directory, for example, any corresponding cloud user accounts are removed.

To fully appreciate and leverage the power of user managment in enStratus I typically further divide users into groups and then grant them access permissions to specific cloud resources using the access rights enStratus assigns to cloud resources.

For example, the manager who asked me about managing users wanted to split his infrastructure into two parts, one for development and one for production, just like his company used in their own datacenter. I created two groups and permissioned the roles to allow personnel in the development group access only to servers in their group, while production remained accessible only to a select subset of developers.

This concept can be expanded to create entire virtual departments analogous to traditional datacenter components. In another account I set up a database group, an applications group, and a security group. In this particular scenario, the security group was responsible for vetting servers and then making them available to the applications and database teams.

Access and user management in the cloud can be secure and role-based using enStratus. By keeping credentials securely outside of the cloud and using groups and roles, managing access can be a successful endeavor.

Using Labels to Prevent VM Sprawl

Last month's update introduced the ability to label servers with a special color. enStratus assigns no meaning to this color. Instead, you can label a server with a color and place any meaning you like on that color.  

From our perspective, one of the most compelling uses of labels is to help control VM sprawl.

VM sprawl happens when people indiscriminately start up virtual machines in the cloud but hesitate to shut them down. Basically, it's much easier to start virtual machines in the cloud than it is to shut them down. Starting a VM is never something destructive; shutting one down is. People consequently hesitate to terminate a cloud instance unless they know for certain that its termination won't have any negative consequences.

enStratus color labels can help you color code your cloud servers so that it's easier to tell what servers can be shut down.

For example, you might establish a process that any unlabeled servers in the cloud can be terminated by any administrator 48 hours after launch. If you want a server to live longer than 48 hours without someone else shutting it down, you must assign it a label. Some labels may represent production systems, while others may represent test or R&D. These colors not only make it easier to kill off servers that were meant for temporary lives, it also makes the process of reviewing what survives simpler.

In the December update, we will expand the presence of labels to machine images and provide the ability to set a label when you launch an instance.