Module 1 — Orientation & Architecture

Module 1 — Orientation & Architecture

1. What Spring AI Is and Isn’t

The single most important framing, and the one most people get wrong: Spring AI is a portability and integration layer, not a model, not a model-training framework, and not a hosted service. It does for AI model providers what JDBC did for databases or what Spring Data does for persistence — a consistent programming abstraction over a fragmented ecosystem of underlying providers, each with their own SDK, request/response shape, and quirks.

What Spring AI actually provides, concretely:

  • A portable API across chat, embedding, image, audio, and moderation models — write code against ChatModel/ChatClient, swap the underlying provider (OpenAI → Anthropic → Bedrock → Ollama) with a dependency and config change, not a rewrite.
  • Spring Boot auto-configuration and starters for every supported provider and vector store — the same “convention over configuration” philosophy as every other Spring Boot starter you’ve used.
  • Higher-level patterns as reusable middleware (Advisors) for things every real AI application needs — memory, RAG, tool calling — so you’re not reimplementing the same scaffolding in every project.
  • First-class MCP (Model Context Protocol) support and an emerging agentic toolkit (Module 8-9 territory) — this is genuinely cutting-edge ecosystem work, not just a thin wrapper.

What Spring AI explicitly is not:

  • Not a model itself — it doesn’t run inference; it calls out to OpenAI, Anthropic, Bedrock, Ollama, etc. Spring AI adds zero model capability of its own — it adds consistency and Spring-idiomatic ergonomics around capability that already exists elsewhere.
  • Not a replacement for understanding the underlying provider’s behavior — portability has limits; model-specific features (e.g., Anthropic’s extended thinking, OpenAI’s specific function-calling quirks) still require provider-specific configuration, and the abstraction can leak when you need a feature only one provider supports.
  • Not a silver bullet against prompt engineering or evaluation discipline — Spring AI gives you the plumbing (Module 10’s evaluation utilities exist precisely because the framework doesn’t make hallucination go away).

Talking point: “I think of Spring AI the same way I think of Spring Data — Spring Data doesn’t make your SQL queries correct or your schema well-designed, it makes the repository pattern consistent across whatever database you’re using. Spring AI doesn’t make a model produce better answers, it makes the integration code consistent across whatever model you’re using. The architecture decisions that actually determine output quality — prompt design, RAG retrieval quality, evaluation — still live entirely with the developer.”

2. Spring AI in the Broader Java AI Landscape

vs. LangChain4j — the most common comparison question, and worth answering with nuance rather than a tribal “ours is better”:

Dimension Spring AI LangChain4j
Origin/philosophy Built by the Spring team, deeply idiomatic to Spring Boot conventions (auto-config, starters, @ConfigurationProperties) A Java port of the original Python LangChain’s concepts and philosophy, framework-agnostic at its core
Integration with existing Spring apps Native — DI, auto-configuration, observability (Micrometer) all just work the way every other Spring module works Usable inside Spring, but you’re integrating an external library’s idioms rather than getting Spring-native ergonomics out of the box
Ecosystem maturity (provider/vector-store breadth) Very broad provider and vector store support, growing fast, but younger overall as a project Generally considered to have started earlier and historically had broader chain/agent abstractions inherited from the Python ecosystem
Abstraction philosophy Leaner, more “Spring Boot starter” feeling — Advisors as composable middleware rather than deeply nested chain abstractions More chain/agent-oriented abstractions, closer to LangChain’s original (sometimes criticized for being over-abstracted) design
2026 trajectory Currently pushing hard into MCP-native support and an opinionated agentic stack (Agent Harness, Judge framework, Agent Clients) Also evolving toward agent support, but the comparison point worth tracking is how each handles the same MCP/agent-protocol wave, since both are converging on similar territory

Talking point: “If a team is already deep in the Spring ecosystem — Spring Boot, Spring Data, Spring Security — Spring AI is the lower-friction choice because every other piece of infrastructure (DI, config, observability, testing conventions) already matches. LangChain4j is a reasonable choice if a team wants closer conceptual parity with a Python LangChain codebase they’re porting, or if they need an abstraction LangChain4j has matured further than Spring AI has at a given point in time. I’d actually prototype both for a genuinely greenfield decision rather than assume one is categorically better — they’re solving the same problem with different philosophies, not different problems.”

vs. calling provider SDKs directly (e.g., the official openai-java SDK, Anthropic’s Java SDK):

Direct SDK Spring AI
Portability None — locked to that provider’s request/response shapes Swap providers with a starter dependency + config change
Boilerplate You write your own retry, observability, memory, RAG scaffolding from scratch Advisors and starters give you this out of the box
Control/access to newest features Immediate — no abstraction lag waiting for Spring AI to wrap a new provider feature Possible lag between a provider shipping a feature and Spring AI exposing it portably (though provider-specific options are usually still accessible as an escape hatch)
When it’s the right call Single-provider commitment, need bleeding-edge provider-specific features immediately, or a very thin/simple integration that doesn’t justify a framework Multi-provider flexibility matters, building a real application with memory/RAG/tools, want Spring-native observability and testing

An important nuance worth surfacing directly: Spring AI’s OpenAI module wraps the official openai-java SDK under the hood (a recent architectural change — the old spring-ai-openai-sdk module was merged in) — so “Spring AI vs. direct SDK” isn’t always an either/or in practice; Spring AI is increasingly composing the official SDKs underneath its own abstraction rather than reimplementing HTTP clients itself. That’s a meaningfully different engineering decision than rolling your own client, and it’s worth knowing this detail because it signals the project is maturing toward “thin, well-integrated wrapper over best-in-class SDKs” rather than “competing reimplementation.”

Talking point: “The direct-SDK-vs-framework decision is really a build-vs-portability tradeoff, same as it always is in software architecture — and it’s worth noting Spring AI itself is increasingly composing official provider SDKs rather than reimplementing them, so adopting Spring AI doesn’t mean abandoning the official SDK’s correctness — it means getting a consistent abstraction on top of it.”

3. Core Architecture: ChatModel, ChatClient, Advisors, Auto-Configuration/Starters

This is the architectural skeleton everything else in the track builds on — worth understanding as layers, each building on the one below.

┌─────────────────────────────────────────────────────────┐
│                     ChatClient (fluent API)              │  ← what most app code calls
│         .prompt() / .advisors() / .call() / .stream()    │
├─────────────────────────────────────────────────────────┤
│                       Advisors                            │  ← composable middleware
│   Memory │ RAG (QuestionAnswerAdvisor) │ Tool calling     │
├─────────────────────────────────────────────────────────┤
│                       ChatModel                            │  ← portable, low-level abstraction
│   OpenAiChatModel │ AnthropicChatModel │ BedrockChatModel  │
├─────────────────────────────────────────────────────────┤
│              Auto-Configuration / Starters                 │  ← Spring Boot wiring
│   spring-ai-starter-model-openai, -anthropic, etc.         │
└─────────────────────────────────────────────────────────┘

ChatModel — the lowest-level portable interface. Every provider implements it (OpenAiChatModel, AnthropicChatModel, etc.). It’s deliberately minimal — call(Prompt) in, ChatResponse out (or a Flux<ChatResponse> for streaming). This is the layer that makes provider-swapping possible, because application code that depends on ChatModel rather than a concrete provider class never needs to change when the provider changes.

@Autowired
private ChatModel chatModel;  // could be backed by OpenAI, Anthropic, Bedrock, etc.

ChatResponse response = chatModel.call(
    new Prompt("Summarize this in two sentences: ...")
);

ChatClient — the layer almost all real application code should actually use. It’s a fluent, builder-style API deliberately modeled after WebClient/RestClient — same idiomatic feel, lower learning curve for anyone who’s used Spring’s modern HTTP clients.

@Autowired
private ChatClient chatClient;   // auto-configured, wraps a ChatModel

String result = chatClient.prompt()
    .system("You are a helpful Java architecture assistant.")
    .user("Explain the difference between a calculated column and a measure.")
    .call()
    .content();

// Streaming
Flux<String> stream = chatClient.prompt()
    .user("Stream a long explanation of CQRS.")
    .stream()
    .content();

Advisors — the extensibility/middleware layer, and arguably Spring AI’s most architecturally distinctive contribution. An Advisor intercepts the request before it reaches the model and/or the response before it reaches the caller — the same conceptual shape as a Spring Security filter chain or a servlet filter, just applied to AI interactions.

String result = chatClient.prompt()
    .advisors(
        MessageChatMemoryAdvisor.builder(chatMemory).build(),     // adds conversation memory
        QuestionAnswerAdvisor.builder(vectorStore).build()         // adds RAG retrieval
    )
    .user("What does our refund policy say about late returns?")
    .call()
    .content();

Talking point on why Advisors matter architecturally: “The Advisors API is what stops every AI feature from turning into a tangle of manually-wired retrieval calls and memory lookups scattered through application code. It’s the same architectural instinct as the AOP/proxy pattern from the Java/Spring track — cross-cutting concerns (memory, RAG, logging, tool calling) get composed declaratively around the core model call, instead of every developer hand-rolling the same plumbing inline. And just like AOP proxies, advisor ordering matters — there was a real bug fixed recently where memory advisors needed to be explicitly placed outside the tool-calling advisor in the chain to behave correctly, which is exactly the kind of proxy-ordering subtlety that shows up across Spring’s middleware patterns generally.” (Direct, genuine callback to the AOP self-invocation/ordering theme that recurred across your whole Java/Spring track.)

Auto-Configuration / Starters — the Spring Boot convention-over-configuration layer. Adding spring-ai-starter-model-anthropic to your build and setting spring.ai.anthropic.api-key in properties is enough to get a fully wired ChatModel/ChatClient bean — no manual @Bean configuration needed, the same pattern as any other Spring Boot starter (spring-boot-starter-data-jpa, spring-boot-starter-security, etc.).

spring.ai.anthropic.api-key=${ANTHROPIC_API_KEY}
spring.ai.anthropic.chat.options.model=claude-sonnet-4-6

4. Versioning Landscape — What to Target Right Now

This is a genuinely important practical module, because Spring AI’s release cadence in 2026 has been unusually fast and milestone releases have carried real breaking changes — version judgment matters more here than in most mature Spring projects.

Track Status What it means practically
1.0.x (currently 1.0.6) Stable, production-supported The safe target for production systems — receives maintenance fixes and CVE patches, no breaking API changes expected
1.1.x (currently 1.1.5/1.1.6) Stable-ish, newer features backported A reasonable production target if you need features not yet in 1.0.x but want to avoid 2.0’s milestone instability — still receives security patches
2.0.0-Mx (currently around M8) Milestone/pre-release — breaking changes between milestones Experimentation, prototyping, and tracking the direction the framework is heading — not yet recommended for production given the explicit breaking-change history across milestones

Concrete breaking changes worth knowing by name, because they illustrate why 2.0.0-Mx isn’t yet a safe production target:

  • MCP annotation package renames and transport artifact relocations — code written against an earlier 2.0 milestone’s MCP annotations may not compile against a later milestone without changes.
  • Jackson 2 → Jackson 3 migration — a foundational serialization library change, which is the kind of dependency shift that can ripple into application code if you’re doing any custom JSON handling alongside Spring AI.
  • Removal of conversation history from ToolContext — a behavioral change affecting any tool implementation that previously relied on that context being present.
  • ToolCallAdvisor renamed to ToolCallingAdvisor (with backward-compatible shims provided, which is the right way to handle a rename, but still worth knowing if you’re reading code/tutorials written against the old name).
  • API key now required even for cookie/session-based auth flows in one milestone (a regression later flagged and worth checking is actually fixed in the milestone you’re using) — a good concrete example of why milestone releases need to be evaluated individually, not assumed stable just because the version number went up.

Talking point: “My rule of thumb with Spring AI right now: production systems target 1.0.x or 1.1.x, and I treat 2.0.0-Mx the way I’d treat any pre-GA milestone in any framework — useful for learning where the architecture is heading and for low-stakes prototypes, but I wouldn’t put it behind a customer-facing system without accepting that the next milestone might require real migration work. This is a faster-moving, more agentic-AI-adjacent project than most mature Spring modules, so I check the release notes before every upgrade rather than assuming semantic-versioning-style safety between milestone bumps — milestones explicitly don’t carry that guarantee.”

A genuinely good practice worth naming: Spring AI’s release notes consistently flag CVE fixes alongside feature work (multiple CVEs patched across 1.0.x/1.1.x/2.0.0-Mx through 2026) — treating dependency upgrades as a security practice, not just a feature-adoption decision, is itself a senior-level habit worth stating explicitly in an interview (this connects directly to Module 11 in the table of contents).

5. Maven/Gradle Setup via start.spring.io, BOM Management

start.spring.io is the recommended entry point — select Spring Boot version, then the specific Model provider(s) and Vector Store provider(s) needed, and it generates a project with the correct starters and BOM already wired. This mirrors exactly how you’d bootstrap any other Spring Boot project, and it’s the officially recommended path specifically because it avoids manual BOM/version mismatches.

Bill of Materials (BOM) — why it matters here specifically: Spring AI ships its own BOM (spring-ai-bom) that pins compatible versions of all the starters, core modules, and provider integrations together. Without the BOM, manually specifying individual module versions risks pulling mismatched versions of spring-ai-core, a provider starter, and a vector store starter that weren’t tested together — exactly the same dependency-hell problem the Spring Boot parent BOM solves for the broader Spring ecosystem, just scoped to the AI modules.

Maven setup:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-bom</artifactId>
            <version>1.0.6</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-starter-model-anthropic</artifactId>
        <!-- no explicit version needed — managed by the BOM -->
    </dependency>
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-starter-vector-store-pgvector</artifactId>
    </dependency>
</dependencies>

Gradle setup (Kotlin DSL):

dependencyManagement {
    imports {
        mavenBom("org.springframework.ai:spring-ai-bom:1.0.6")
    }
}

dependencies {
    implementation("org.springframework.ai:spring-ai-starter-model-anthropic")
    implementation("org.springframework.ai:spring-ai-starter-vector-store-pgvector")
}

A practical gotcha worth flagging explicitly (and a real one from the release history): some starters have had incorrect or missing transitive dependency declarations — e.g., the PGVector vector store starter not working without explicitly also adding spring-boot-starter-jdbc, and the Google GenAI model starter incorrectly declaring a dependency on the embedding module. Talking point: “Even with the BOM doing its job on version alignment, I still expect to occasionally need an explicit transitive dependency the starter’s metadata didn’t fully declare — this is a young, fast-moving set of starters, and a few have shipped with incomplete dependency declarations. I treat ‘starter doesn’t work out of the box’ as worth checking the GitHub issues/release notes before assuming I’ve misconfigured something myself.” This is a genuinely useful, currently-accurate piece of practical guidance, not just generic Maven advice.