AI/ML Architecture

AI/ML Architecture

1. Amazon SageMaker End-to-End Pipeline

SageMaker Pipelines is a native, serverless workflow orchestrator purpose-built for ML. It ensures reproducibility, governance, and rapid deployment by treating your entire ML lifecycle as a continuous, automated Directed Acyclic Graph (DAG).

Core Lifecycle Steps:

  • Data Prep (ProcessingStep): Ingests raw data from Amazon S3. Runs scripts via native containers (e.g., SKLearn, PySpark) to clean, transform, and partition data into training, validation, and test sets.

  • Feature Store Integration: Consumed and engineered features can be written directly to the SageMaker Feature Store for low-latency online serving or offline training runs.

  • Model Training (TrainingStep): Provisions managed compute instances (CPUs/GPUs), pulls deep learning or custom Docker containers, streams data via FastFile/Pipe mode, and stores trained artifacts (model.tar.gz) back in S3.

  • Evaluation (ConditionStep): Evaluates the model against a holdout test set. If the performance metric (e.g., $R^2 \ge 0.85$ or $F_1 \text{ score} \ge 0.90$) meets your threshold, it triggers the promotion branch.

  • Model Registry (RegisterModel): Registers the qualified model version into the centralized SageMaker Model Registry, cataloging its structural metadata, baseline constraints, and deployment readiness status (Pending Manual Approval).

2. Real-Time Inference vs. Batch Inference

Choosing the right serving strategy depends on your application’s volume, latency tolerance, and cost structure.

Feature Real-Time Inference Batch Inference (Batch Transform)
Latency Milliseconds ($<100\text{ms}$ ideal) Minutes to Hours (High throughput)
Trigger Immediate API calls (HTTP POST) Scheduled, Event-driven (S3 upload), or On-demand
Data Payload Single record or tiny micro-batches Massive datasets (Gigabytes to Terabytes)
Cost Profile Continuous billing for provisioned endpoints (or scaling down with Serverless Inference) Ephemeral compute; you pay only for the minutes the batch cluster runs
AWS Tooling SageMaker Real-Time/Asynchronous Endpoints SageMaker Batch Transform

3. Vector Databases on AWS (For RAG & GenAI)

Vector databases store and query high-dimensional embeddings generated by LLMs or embedding models. AWS provides three primary patterns depending on your scale and operational choices:

Amazon OpenSearch Service (Vector Engine)

  • Best For: Teams that want a unified platform for enterprise keyword search, log analytics, and semantic vector search.

  • Tech Stack: Uses k-NN (k-nearest neighbors) plugin leveraging HNSW (Hierarchical Navigable Small World) or IVF (Inverted File) indexes.

Amazon Aurora / RDS PostgreSQL with pgvector

  • Best For: Relational architectures where you want to keep operational data and vector embeddings in the exact same database.

  • Tech Stack: A standard PostgreSQL extension supporting distance metrics like Cosine, L2, and Inner Product. Highly scalable up to tens of millions of vectors without infrastructure overhead.

Amazon Bedrock Knowledge Bases (Managed Option)

  • Best For: Fully serverless, zero-ops Retrieval-Augmented Generation (RAG).

  • Tech Stack: Automatically syncs an S3 bucket, chunks text, generates embeddings via Amazon Titan/Cohere, and handles the vector read/writes behind a single API endpoint.

4. AI Integration with Lambda & API Gateway

To expose machine learning or generative AI capabilities to web applications, AWS utilizes a serverless microservices pattern.

[ Client / Web App ]
         │ (HTTPS POST /predict)
         ▼
[ Amazon API Gateway ]  <── (Throttling, IAM Auth, API Keys)
         │
         ▼
[ AWS Lambda ]          <── (Payload validation & orchestration)
         ├── Boto3 Invoke ──> [ Amazon Bedrock / SageMaker Endpoint ]
         └── SQL Query      ──> [ RDS pgvector (Context Retrieval) ]

Architectural Flow:

  1. Ingress: The client sends an HTTP POST request containing raw text/data to Amazon API Gateway. API Gateway handles request throttling, JWT/IAM authentication, and CORS.

  2. Compute Orchestration: API Gateway triggers an AWS Lambda function asynchronously or synchronously. Lambda acts as the lightweight “glue”:

    • It parses the client payload.

    • (Optional) It calls a vector database to fetch relevant prompt context.

  3. Inference: Using the AWS SDK (boto3), Lambda invokes either an Amazon Bedrock foundation model or an active SageMaker Endpoint.

  4. Egress: Lambda formats the inference output and routes it back through API Gateway to the end user.

5. MLOps on AWS

MLOps bridges the gap between model development and reliable IT production. AWS organizes structural MLOps around a multi-account blueprint.

  • Multi-Account Isolation: Production environments are split into segregated AWS accounts: Dev (Experimentation/Training) $\rightarrow$ Test/Staging (Integration & Shadow Deployment) $\rightarrow$ Prod (Isolated Inference).

  • CI/CD CodePipelines: Changes pushed to a Git repository (AWS CodeCommit, GitHub) automatically trigger a pipeline execution. This builds containers via AWS CodeBuild and updates SageMaker resources via AWS CloudFormation or Terraform.

  • Data & Model Drift Monitoring: SageMaker Model Monitor continuously intercepts data passing through real-time endpoints. It compares live inference features against the baseline training schema stored in S3, auto-triggering alerts via Amazon CloudWatch and SNS if a data drift (e.g., demographic shifts) or concept drift occurs.

  • Automated Retraining Loops: When CloudWatch captures a performance degradation metric or an explicit drift threshold breach, an event triggers an AWS Step Function or SageMaker Pipeline to run a fresh training job with the latest data, closing the automated MLOps flywheel.