Getting Started

Initiating Amazon Web Services (AWS) appears simple enough: Create an account, spin up a few compute instances and let the magic and speed of AWS take care of the rest. Moving past this initial phase, however, is not necessarily straightforward. Even if the primary use for AWS is for non­-production use—whether that be for testing or research and development—once embedded into an organization, challenges will arise around billing, scaling limits and resource control. Segregation of infrastructure, resources and services is necessary for AWS to be integrated for technical, financial and business process purposes.

The bottom line is: A single account for any size organization is not going to work. Virtual resources, developers and products will eventually get muddled together, causing an administrative burden to track and maintain. Further, one department’s decisions could negatively affect another or, worse, bring down a live production application.

Fortunately, there are tools to use and best practices to follow to build a supportable, scalable environment.

Dealing with soft limits

Soft resource limits are seldom advertised by AWS, but they are documented. While it may feel like you can infinitely scale resources at any given moment, limits exist on every new account that can be restricting to larger enterprises. The limits are soft and can be increased through opening a ticket with AWS support staff, but even then the limit does not go away. It is merely replaced with another agreed upon soft limit that can potentially be hit again in the future.

Imagine the consequences of a development and production environment sharing the same account. The development team launches new EC2 instances which brings the account right up to the soft limit for that region. Soon after, a production application experiences a spike in demand and attempts to auto-scale by launching more EC2 instances. The production application will not be able to create the new instances and customers will begin to receive errors and possible service interruptions.

In another scenario, an emergency fix needs to be applied. To test the fix, a new Virtual Private Cloud (VPC) that mirrors production must be created. When doing so, creation fails, due to the VPC soft limit, which starts at five per region. An existing VPC, possibly not related to the project having the issue, would need to be destroyed or the fix may need to be tested in a more expensive or remote region.

The soft limit issue does not go away by managing multiple accounts, but it does allow for isolation of resource spikes. By using accounts dedicated to a specific purpose, there is also a clear distinction of caps placed on development, production or product teams, as opposed to a single organization account.

Simplified billing

Within an AWS account, resources can be tagged and those tags will appear for each line item in the detailed billing report . Unfortunately, not all AWS resources support tagging. Some notable omissions include Simple Queue Service (SQS), Simple Notification Service (SNS) and DynamoDB.

In a single account solution, it would be very difficult to relate non-tagged resource costs to projects. Secondary metadata stores, naming conventions and other out-of-band solutions would be necessary.

By splitting functional and business use cases across accounts, it is feasible to associate each account to a billing or project number. The master account will be able to report on each linked account, as well as totals for all accounts.

Support tools for multiple accounts

Multiple accounts keep products, departments and environments running smoothly. AWS realizes this and not only supports customers with multiple accounts, it also provides a few tools to help with billing and management.
Consolidated billing. AWS supports multiple accounts through the consolidated billing feature. One account acts as the master and consumes all the billing information from other linked accounts. Billing for each linked account can be analyzed individually and volume discounts will be aggregated and applied across all linked accounts. It is not recommended to create any resources in the master billing account. Once a process is in place for linking accounts to a master billing account, architects and engineers can begin to create accounts for departments, products and production environments. There is no hard and fast rule to the number of accounts; each should be created based on the need of the application and resource access. One solution may involve creating two dedicated accounts for a particular department. That department could then perform on non-production development (dev, test and QA) in one account and use the second for production.

Cross-Account Access. AWS recently made available Cross-Account Access for the AWS Console. Cross-Account Access is based on the long established use of Identity and Access Management (IAM) roles that are also used for federated single-sign-on and EC2 instance profiles. Once a user gains access to the console in one account, they are allowed to switch to other pre-approved accounts at will. There is no need to sign out of the first account or try to manage multiple browsers.

Cross-account Services. IAM Roles, combined with Policies and Amazon Resource Names (ARNs), allow people, code and resources to manage and communicate across accounts within an AWS. This pattern becomes more robust when combined with native AWS services, such as DynamoDB, SNS, SQS and S3. Information can naturally flow to and from accounts without sharing access keys or creating relationship accounts.

Manual Account Creation

A downside to running multiple accounts is the manual process of creating an account with AWS. The process requires completing a CAPTCHA, entering a secret code via a telephone call and a credit card. The credit card can be removed once the account is linked with the master billing account. While not ideal, account creation, even in large organizations, is an occasional need that usually presents itself at the beginning of a
new product design or onboarding a new department to AWS.

Final Thoughts

The creation and use of multiple accounts on AWS is a supported and logical design pattern that will help any size organization create, deploy and support a variety of internal and external cloud products. Tracking the business case, and who is ultimately responsible for respective linked accounts, will require time and resources. However, this approach will ultimately make it easier to manage AWS resources, when compared to doing the same for a single account and having too many hands in the pot. The best part is that additional accounts add no additional cost. If you have development and production resources in the same account, it is time to split those environments and start learning how to handle multiple accounts today.

Kevin is a member of the CTO Architecture Team, which is responsible for researching and evaluating innovative technologies and processes. He has a strong, extensive background in DevOps specifically in developing tools for Operations Support Systems (OSS) within the communications and IT infrastructure industry.

Before arriving at Sungard Availability Services, Kevin spent 13 years at AT&T (by way of AT&T’s acquisition of USi) rising through the engineering ranks to become a Senior Technical Architect for the Hosting Division’s Tools Team.

Kevin holds a B.A. in Economics from the University of Maryland and a Masters in Technology Management from University of Maryland University College.


Comments

3
  • Andy Anderson

    Kevin,
    I have hit those soft limits once I started toying with cloudformations and then hitting it again with auto-scaling groups. The problem with the limits is not that they exist, it is that I hit them on off-hours when I am experimenting or making progress.

    One thing I bump into with all service providers is that there isn’t a way to notify you when you forget to shut something off that isn’t being used. For instance, have you ever created a dynamic IP in AWS and then forgotten your not using it? They charge you for that. It would be great to see, in a tool, realtime account charges. Pay-as-you-go should be accompanied by View-my-bill-as-I-go. Maybe this exists, but I haven’t seen a solution yet. At a minimum I would like to see what I am “not” using – Idle resources.

    Andy

    Posted on