Shell scripting with AWS Lambda

One of the newest pieces of the AWS toolkit is the Lambda compute engine.  Lambda provides an ability to deploy small bits of code that run independently as functions.  AWS only charges for the time that these snippets are running based on the resources requested to run the code.  This allows for extremely granular use of compute resources.

Previously, billing for general-purpose compute power was only in increments of one hour using EC2.  This was true even for managed aspects of EC2: Elastic Beanstalk, RDS, Elastic Load Balancing, etc.  This is not to say that there were no services that charged under a different model.  Many other non-compute specific services such as S3, SQS, or Kinesis are based on a per usage model.  The 100 millisecond-pricing model introduced with Lambda provides something that feels a great deal like per usage pricing for general compute.

Since Lambda functions are small and charged only when in use, Lambda encourages a model where development items can be deployed as many single functions rather than as more monolithic single server applications.  This is the Unix command line philosophy applied to the cloud.    It encourages the developers to focus on purpose built tools and interaction between components.  Just as when you build a shell script expecting it to interact with other shell scripts to achieve a larger task.

This parallel with shell scripting is also interesting with regards to administration of your AWS cloud infrastructure as a whole.   The Lambda Node functions contain the JavaScript AWS API by default.  Building functions to perform cloud maintenance, provisioning, or other scripting is now easily performed within the cloud itself.  I realize that you could always deploy an EC2 instance that contained scripts for this purpose in any language needed.  This is a very heavyweight approach for a simple activity that is likely to only run for a few minutes.  You incur not only the cost of the instance but all of the development work to get the instance setup and ready to run scripts.  Lambda does this all for you. 

It is better to produce a series of scripts to manage specific aspects your AWS infrastructure.  This is how shell scripts are used on Linux systems to manage aspects of the running machine.  A Lambda script can perform a single activity or when needed can be chained together to perform a series of actions.  Lambda also comes with a couple of different external triggers: S3 and SQS.  This allows your Lambda scripts to respond to actions occurring with other AWS or external applications that interface with these tools.

This is all supported by the fact that Lambda is secured using IAM roles.  This reduces the use of AWS credentials and allows you to pinpoint the AWS products, or even specific instances within the product, that the script has permission to access.  This security helps minimize the chance that a buggy script causes any issues outside of its allowed domain.  It is again similar to shell scripting permissions to reduce possible script exposure.

All of this points to Lambda as a great tool for managing AWS infrastructure in addition to other compute tasks that you may want to use it for.  In the next article in this series, I’ll  use Lambda to stop existing EC2 instances that are not in use. Stay tuned.