At the beginning of December, we sent several of our Amazon Web Services (AWS) certified developers and solutions architects to AWS’s annual conference re:Invent. During the five day event, AWS introduced a long list of new features, new services and enrichment of existing services.
The key themes we saw throughout the conference surrounded serverless architecture, voice enablement and a new focus on "compute at the edge." Database solutions continue to be improved with new offerings like Amazon Athena and enhancements to Amazon Aurora and Redshift.
Below are some of the announcements that our developers consider to be the highlights of this year’s show:
Amazon Lightsail is the AWS answer to new user complaints that Amazon Elastic Compute Cloud (EC2) is still far too complex with too steep of a learning curve. Additionally, some customers discovered they could run Google Compute Engine (GCE) at a lower cost than EC2. Lightsail addresses both the complexity and pricing issue.
Anyone who has used EC2 for more than a few years knows that it has come a long way in reliability and in ease of use. In the early days, there was a steep learning curve to understanding how to define, deploy, and manage EC2 instances, and it was not uncommon for them to crash, freeze up or otherwise need to be urgently relaunched or redeployed. Prior to Amazon Elastic Block Store (EBS) volumes, you even lost all your data on the ephemeral virtual hard drives anytime an EC2 instance was no longer running. These challenges forced good planning and good habits, but it nevertheless made it harder for a new user to efficiently get up and running with EC2 instances.
EC2 has become significantly more reliable and easier to get up and running, to the detriment of some of the good habits and best practices that users were previously forced to implement, but it still requires a fair amount of knowledge. For example, users need to understand a Virtual Private Cloud (VPC), public vs. private subnets, routing tables, security groups, and DNS resolution. A software development team just trying to get a prototype up and running might not fully understand these infrastructure concepts that perhaps their IT colleagues took care of in their on-premises paradigm.
Meanwhile, Google and Microsoft made it fairly simple to get up and running quickly with a Virtual Private Server (VPS) in Google Cloud or Microsoft Azure.
Enter Amazon Lightsail. Lightsail is a preconfigured, easy-to-deploy VPS that starts at just $5 USD per month in the US East (Northern Virginia) Region (currently the only supported region for Lightsail as of this writing). Now anyone can get up and running quickly without having to first learn about VPCs, subnets, and other networking topics. Lightsail is expected to become a popular offering because of its pricing and ease of use. More information about Lightsail is available at https://amazonlightsail.com/
One year ago at re:Invent 2015, AWS QuickSight was announced. QuickSight makes it easy for business users to analyze and summarize data. Initially the offering was restricted to a small audience with interested customers finding themselves on a waiting list. It was finally made generally available (GA) shortly before re:Invent 2016.
Over the past year, there has been significant interest and buzz around QuickSight. While AWS historically focuses on the needs of developers, QuickSight allows less technical end users, for example line of business (LOB) managers, to use the tool – even from a smartphone – to analyze and visualize everything from Excel spreadsheets to complex relational databases.
Below is a sample screenshot from QuickSight running on an iPhone 7 Plus:
Notably, QuickSight can work with on premises databases as well as with data in Amazon Simple Storage Service (S3). It of course also supports Amazon Redshift, Amazon RDS, and Amazon Aurora.
QuickSight is expected to rapidly become popular now that it is GA. Full details are available at https://quicksight.aws/
The most significant announcement regarding Amazon Aurora is that it is now also PostgreSQL compatible. Additionally, both Amazon Aurora and RDS for PostgreSQL are now included in the AWS HIPAA compliance program, allowing customers needing HIPAA compliance to take advantage of the managed service of either Aurora or RDS.
A key argument the Aurora team makes when encouraging customers to choose Aurora over a NoSQL solution is the available talent of MySQL experts. MySQL certified developers and DBAs can be up and running quickly with Aurora when taking advantage of its MySQL compatibility, whereas there is a smaller talent pool of resources knowledgeable in NoSQL solutions, according to AWS. Additionally, some Aurora customers have found a 40% cost savings when migrating away from a NoSQL solution to Amazon Aurora, according to the Aurora team.
Migrating databases is of course not trivial, so careful consideration should be given up front whether an application should leverage an ACID compliant relational database or instead use a NoSQL database solution. Pricing, available talent, and vendor lock-in are also important considerations.
An interesting scenario mentioned by AWS was existing MySQL customers converting their multiple MySQL shards into a single Amazon Aurora database. Application-level sharding is a popular scalability strategy in MySQL when needing to scale out and reduce individual table sizes. For example, some MySQL tables will have performance difficulties beyond hundreds of millions of records in a single table, and sharding is a popular strategy for reducing this table size. A telephone directory of the United States, for instance, could be broken down into geographic shards, with each shard containing a subset of the data (e.g., Northern California shard, Midwest shard, etc.). Some customers are finding that with Amazon Aurora, they can consolidate their shards, which reduces cost and complexity while making large queries much simpler.
Compute at the Edge
Another key theme at re:Invent was Compute at the Edge. AWS has quietly been hinting about this for the past year, and re:Invent saw three important announcements in this space: AWS Greengrass, Lambda@Edge, and Snowball Edge.
AWS Greengrass puts compute, and specifically AWS Lambda, on local devices so they can function independently of the Cloud. A popular use case is a home automation solution, where perhaps your car pulling in the driveway causes your house doors to unlock, your indoor holiday lights to turn on, your kitchen lights to turn on, music to start playing, and indoor surveillance cameras to turn off. The challenge prior to Greengrass is that often this logic requires Internet connectivity and fast responsiveness from code running in the Cloud. A proximity sensor or door sensor would communicate to a local hub which would in turn find out what to do from code running in the Cloud. The code in the Cloud would then send a push command back down to the hub and/or to additional devices within the home. If there were an Internet outage, Cloud provider incident, or other connectivity problem – perhaps even just extended latency – desired actions might be delayed minutes or hours. We want our lights turned on immediately, not hours later. Bringing the code and compute down to the local devices eliminates the Cloud and the Internet as potential single points of failure (SPOF).
Greengrass allows developers to use their existing AWS Lambda skills to also implement code locally. More information about Greengrass is available at https://aws.amazon.com/greengrass/
A popular announcement at 2015 re:Invent was Snowball, an AWS appliance that allows for simple transfer of terabytes of data from an on-premises datacenter to Amazon S3. Snowball Edge, announced last week, not only doubles the capacity of a single device from 50 TB to 100 TB, but it also adds local clustering ability and local compute power in order to more intelligently process and transfer data. Details on Snowball Edge are here: https://aws.amazon.com/snowball-edge/
It is also worth noting that in October 2016, AWS added the ability to transfer directly from HDFS to Snowball, a feature that was missing at the original launch of Snowball. Prior to October, customers had to do an intermediary step of getting the data out of HDFS first and then to Snowball second. This commonly requested feature makes migrating HDFS data to AWS significantly faster and easier. Details on this announcement are here: https://aws.amazon.com/blogs/aws/snowball-hdfs-import/
Amazon Athena is one of the more interesting serverless solutions, since it introduces a “pay per query” model. Athena uses standard SQL to query data directly from S3. Anyone with SQL skills can immediately start analyzing data files with no infrastructure required. It is not expected to be as fast as server-based solutions, but it provides an interesting alternative when ad-hoc questions need to be answered without time or budget to set up an infrastructure first. More details are here: https://aws.amazon.com/athena/
Full List of Announcements
The full list of announcements from AWS re:Invent 2016 are available at this web site: https://aws.amazon.com/new/reinvent/
Videos of re:Invent sessions are available here: https://www.youtube.com/user/AmazonWebServices
The Cloud is a strategic part of everything we do. Every year it becomes easier to use and configure, easier to access and more powerful. This year’s re:Invent announcements represent the continuation of this trend. At Dev9, we believe that every industry can benefit in some way by using Cloud services. We are dedicated Cloud experts supporting our current and future clients on their Cloud journey.
Dev9 is proud to be a member of the Amazon Web Services Partner network. We work with organizations to develop custom software solutions, migrate from legacy systems to the Cloud, and modernize existing applications. Find out more about our Cloud services here, or contact us if you’d like to talk about your specific Cloud needs.