A Simple Approach to Choosing AWS Compute for Enterprise Workloads

Image by Katerina Limpitsouni from https://undraw.co/

How do you weigh up the options to choose between EC2, Lambda, ECS, Fargate, EKS, Batch and numerous alternatives? This choice can seem really hard. It’s not made any easier by our typical decision-making habits. Often, we debate our opinions on technology options and force through choices that are more based on our existing biases and recent successes than looking at the facts and using a simple guiding methodology.

This post details factors to consider and factors that can be avoided when choosing compute services. Simplicity is the guiding principle driving the decision-making process. It will:

  1. Reduce time spent making…

Converting formats on demand using S3 Object Lambda

Fetching and transforming data from S3 using AWS Lambda is one of the most common serverless patterns. The new S3 Object Lambda feature allows a Lambda to be invoked on demand as part of the lifecycle of S3 GetObject. This opens up a new set of possibilities. Objects can be transformed, filtered and generated on the fly without adding higher level access points like API Gateways. I’ll give a full code example of this below. First, let’s summarise the main points of S3 Object Lambda.

  1. It works for GetObject only. …

There are two primary methods to trigger Lambda when an object is added to an S3 bucket — S3 Notifications and EventBridge. Using S3 notifications is more typical but there is one big drawback for me. For this reason, let’s look at EventBridge as an alternative.

S3 Notifications and CloudTrail/EventBridge are differing approaches to triggering functions on S3 Object events

An S3 notification is part of its NotificationConfiguration, a property of the bucket itself. When using CloudFormation, this generally means that the notification must be created or modified as part of the bucket resource itself. In a serverless application, you can imagine having some sort of shared bucket with separately deployable serverless services triggered…

EDIT: If you’re here for the solution, skip straight to the “Solving the Problem with Secrets Manager” section!

Using an Infrastruture-as-Code approach for a Serverless application is a given. The Serverless Framework gives you the tooling to define you AWS resources using CloudFormation definitions, giving you benefits like stacks, versioning and rollback support.

When it comes to implementing a continuous build, integration and deployment pipeline for Serverless, there are quite a few options. We generally go with an AWS-native approach for these things. Since we first started adopting CodeBuild and CodePipeline, the services have matured quite a bit. Critically, their…

Judging by the packed rooms, Machine Learning and Serverless are the two topics grabbing most of the attention at this year’s AWS Summit London. Julien Simon — Principal Evangelist, AI & ML at AWS, gave a comprehensive rundown of the machine learning capabilities of AWS.

While the adoption of machine learning technology has really taken off in the last five years, Julien reminded us that Amazon have been working in this space since their foundations in online book retail. Even in 1995, personalized content was front and center on Amazon.com. It was recognized early that personalizing content and targeted recommendations…

Microservice Containers with Increased Productivity

Using Docker brings plenty of benefits to microservice projects. Continuous deployment, immutable deployment and consistency between development and production environments eliminate lots of the problems you might find using other approaches.

I have seen quite a few approaches to microservice deployment using Docker, primarily with Node.js. Most of them have a separate code repository per microservice, each with its own dependencies and build tooling. If you’re scaling to tens or hundreds of microservices in a large, established organisation, this might work out. If you’re working as part of a small team in a startup that wants to move fast, isolating…

Eoin Shanaghy

CTO @fourtheorem https://fourTheorem.com; Co-Author of AI as a Service https://www.manning.com/books/ai-as-a-service

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store