Compute Architecture

1. EC2 Compute Architecture & Infrastructure Sizing

Amazon Elastic Compute Cloud (EC2) remains the foundation of AWS compute. Architecting for EC2 requires matching workload profiles to the correct instance families, balancing placement strategies, and optimizing automated elasticity.

Instance Families & Workload Mapping

Selecting an EC2 instance involves evaluating the core hardware metrics (vCPU, Memory, Local Storage, Network, and I/O performance):

• General Purpose (M, T, C-family derivatives): Balanced ratios of CPU and memory. Ideal for backend application servers, small-to-medium databases, and build fleets. Note: T-instances rely on a burst credit model for CPU credits.

• Compute Optimized (C-family): High ratio of vCPUs to memory. Tailored for high-performance web servers, scientific modeling, batch processing, and media transcoding.

• Memory Optimized (R, X, U-family): High memory-to-vCPU footprint. Crucial for open-source high-performance caching layers (e.g., Redis, Memcached), in-memory databases (SAP HANA), and massive data analytics processing.

• Accelerated Computing (P, G, Trn, Inf-family): Hardware offloading using GPUs or specialized ASICs (AWS Trainium, AWS Inferentia). Necessary for LLM fine-tuning, generative AI inference, and complex graphics processing.

• Storage Optimized (I, D, H-family): Low-latency, local NVMe storage or high sequential throughput. Engineered for distributed databases (Cassandra, MongoDB) and data warehousing.

Placement Groups

Placement groups allow you to influence the physical distribution of EC2 instances within the AWS data center fabric to meet networking or resiliency goals:

Amazon EC2 Auto Scaling

Modern Auto Scaling relies heavily on Target Tracking Policies (e.g., maintaining an average CPU utilization of 60%) and Predictive Scaling, which uses machine learning to forecast traffic cycles and pre-provision EC2 fleets ahead of time.

• Capacity Providers & Mixed Instance Policies: Production ASGs should use mixed instance policies, blending On-Demand (for baseline predictability) with Spot Instances (for cost reduction up to 90%, utilizing 2-minute interruption notices) across multiple instance types to protect availability.

2. Containerized Infrastructure: ECS, EKS, & Fargate

AWS categorizes its container ecosystem into Orchestrators (ECS, EKS) and Compute Planes (EC2, Fargate).

Amazon ECS vs. Amazon EKS

• Amazon Elastic Container Service (ECS): AWS’s proprietary, highly integrated orchestrator. It uses an AWS-native mental model (Tasks instead of Pods, Services, and Task Definitions) and features seamless integration with IAM, Application Load Balancers (ALBs), and AWS Service Connect for service mesh capabilities. Highly recommended for AWS-first, lean development squads.

• Amazon Elastic Kubernetes Service (EKS): CNCF-compliant managed Kubernetes service. It provides the full standard Kubernetes API, allowing multi-cloud portability, declarative GitOps pipelines, and integration with the wider cloud-native ecosystem (Helm, ArgoCD, Karpenter). Ideal for massive scale and hybrid cloud infrastructure.

AWS Fargate: The Serverless Compute Alternative

Fargate removes the operational overhead of managing underlying virtual machines. Instead of provisioning EC2 instances for your container clusters, you declare resource footprints (vCPU and Memory allocation) at the Task/Pod level.

• Trade-off Profile: Fargate offers an unbeatable operational model (zero patching, instant scaling isolation per task) but carries a pricing premium at sustained high volumes compared to perfectly packed EC2 groups utilizing Spot pricing. Fargate is ideal for bursty workloads, microservices under lower load, and batch jobs.

3. Serverless Architecture: AWS Lambda Deep Dive

AWS Lambda permits event-driven execution with automated, fine-grained scalability. However, high-throughput production environments require deep management of execution loops and compute overhead.

Lifecycle of an Event: Cold Starts vs. Warm Starts

When an event triggers a Lambda function, AWS creates or reuses an Execution Environment built on Firecracker microVM technology.

[ Cold Start Path ] -> [ Download Code ] -> [ Initialize Runtime (INIT Phase) ] -> [ Run Handler (INVOKE Phase) ]
[ Warm Start Path ] --------------------------------------------------------------> [ Run Handler (INVOKE Phase) ]

• Cold Start: Occurs if no idle environment is ready. AWS must provision resources, download code/container images, initialize the runtime, and execute your global scope initialization code (the INIT Phase).

• Warm Start: Occurs if an existing execution environment is kept warm from a recent invocation. It skips the INIT phase and routes directly to the handler function, resulting in sub-10ms response times.

• The Lambda Cost Update: AWS bills for the full duration of the INIT Phase during a cold start for managed ZIP-based runtimes. Optimizing initialization logic is now a financial priority as well as a performance requirement.

Key Optimization Strategies

1. Architecture Choice: Migrate from x86 to ARM64 (AWS Graviton). It provides a native 15–40% optimization in cold-start speeds and a 20% lower price point per GB-second.

2. AWS Lambda SnapStart: Essential for managed runtimes like Java or .NET. SnapStart initializes the function, takes an encrypted snapshot of the memory/disk state, and caches it. On cold starts, it restores from the snapshot, reducing startup latencies from seconds to sub-200 milliseconds.

3. Code Optimization: Maintain heavy initializations (database connection pools, loading AWS SDK clients) outside the handler function block so they execute once during INIT and persist across warm starts.

Concurrency and Scaling Mechanics

• Account Limits: Accounts have a baseline concurrent execution limit per region (typically 1,000, burstable depending on the region).

• Reserved Concurrency: Guarantees a dedicated pool of concurrent executions for a specific function while acting as a ceiling to prevent an unmanaged loop from consuming the entire region’s concurrency pool.

• Provisioned Concurrency: Pre-initializes a set number of execution environments. This completely eliminates cold starts for critical user-facing APIs, though it incurs steady-state hourly charges.

4. Hybrid Compute Topologies

To meet strict residency regulations, sub-millisecond local network requirements, or specific edge topologies, AWS extends its compute plane out of regional data centers.

+-----------------------------------------------------------------------------------+
|                            AWS Core Region (Compute & Storage)                     |
+-----------------------------------------------------------------------------------+
       |                                      |                               |
       v                                      v                               v
+--------------------------+      +-----------------------+      +--------------------------+
|       AWS Outposts       |      |    AWS Local Zones    |      |      AWS Wavelength      |
| (On-Premises Data Center)|      | (Metropolitan Edges)  |      | (5G Telco Carrier Nodes) |
+--------------------------+      +-----------------------+      +--------------------------+

• AWS Outposts: Fully managed, physical rack enclosures delivered directly to an on-premises data center or co-location facility. Outposts run native AWS services (EC2, EBS, ECS, S3) locally, seamlessly connecting back to the regional control plane.

• AWS Local Zones: Compute extensions located close to major metropolitan or industrial centers. Instead of shipping hardware to your data center, you deploy infrastructure into an AWS-run local zone to achieve single-digit millisecond latency for end-users in that specific geography.

• AWS Wavelength: Embeds AWS compute and storage hardware within the data centers of major 5G telecommunication providers. Applications route directly from mobile devices to Wavelength zones, bypassing the public internet to enable real-time processing for connected cars, industrial IoT, and AR/VR stream rendering.

5. Automated Orchestration & Batch Processing

High-throughput distributed workflows require separating data preparation, ingestion, compute heavy lifting, and final state validation into resilient pipelines.

AWS Batch

AWS Batch automates the provisioning of compute resources for batch computing workloads across the full spectrum of scale.

• Mechanics: Developers define a containerized Job, submit it to a Job Queue, and AWS Batch schedules it on an optimally configured compute environment (EC2 or Fargate).

• Architecture Advantage: It handles complex multi-node parallel jobs and manages Spot instance termination signals natively, automatically re-queueing interrupted jobs without developer intervention.

AWS Step Functions

AWS Step Functions is a serverless visual orchestration layer that allows you to chain various AWS resources (Lambda, AWS Batch, ECS, DynamoDB) into resilient distributed workflows using state machines.

[Start] -> [Lambda: Validate Data] -> [Choice State]
                                          |
                        +-----------------+-----------------+
                        | (Data Size < 10MB)                | (Data Size >= 10MB)
                        v                                   v
             [Lambda: Fast Process]                [AWS Batch: Heavy Compute]
                        |                                   |
                        +-----------------+-----------------+
                                          v
                              [DynamoDB: Save State] -> [End]

• State Machine Patterns: Offers Standard Workflows (long-running up to 1 year, durable, audit-logged execution history) and Express Workflows (high-throughput, sub-5-minute execution times designed for high-volume IoT ingestion or quick API aggregation).

• Error Resiliency: Step Functions removes structural error-handling boilerplate code from applications by managing retries, exponential backoffs, and catch routines natively at the infrastructure definition layer.