Connect Spring Boot to LLMs with Spring AI

Quick Start

This Byte shows how a Spring Boot application connects to an LLM using Spring AI. We begin with the essential pieces—ChatClient, prompts, configuration and dependency injection—so the lesson works even if this is your first Byte in the handbook.

Meet the Scenario

Meena wants PaisaWise's Java-based backend to power a simple internal tool: a customer-support assistant that summarizes long complaint threads. Karthik asks: "What connects our Java service to the language model?" Divya replies: "Spring AI's ChatClient—a Spring-native interface for preparing a prompt, calling the model and reading its response."

Core Concept

An LLM call needs three basic steps: prepare a prompt, send it to a model and read the response. Spring AI provides this workflow through a Spring Bean—an object Spring creates and manages for the application. Once the bean exists, dependency injection supplies it to your service and a fluent method chain performs the call.

Think of it like ordering at a coffee shop with a "build your own" ordering ritual: .prompt() starts your order, .user(...) is what you actually want to say, and .call() is handing the order to the counter and waiting for the result — Spring AI's ChatClient uses exactly this same fluent, step-by-step ordering style.

How It Works Under the Hood

Step 1: Spring Boot auto-configures a ChatClient.Builder bean once you add the Spring AI starter dependency for your chosen provider (OpenAI, Azure, Amazon Bedrock, Google, and others are all supported the same way).

Step 2: Inject it into your own service, just like any other Spring bean:

01@Service02public class ComplaintSummaryService {0304    private final ChatClient chatClient;0506    public ComplaintSummaryService(ChatClient.Builder builder) {07        this.chatClient = builder.build();08    }0910    public String summarize(String complaintThread) {11        return chatClient.prompt()12                .user("Summarize this customer complaint in 2 sentences: " + complaintThread)13                .call()14                .content();15    }16}

Step 3: Expose it through a familiar REST controller (exactly like Byte 1's AccountController):

01@RestController02@RequestMapping("/api/support")03public class SupportController {0405    private final ComplaintSummaryService summaryService;0607    public SupportController(ComplaintSummaryService summaryService) {08        this.summaryService = summaryService;09    }1011    @PostMapping("/summarize")12    public String summarize(@RequestBody String complaintThread) {13        return summaryService.summarize(complaintThread);14    }15}

Notice the pattern is identical to Byte 1 — a @Service doing the work, a @RestController exposing it, dependency injection wiring both together. Spring AI didn't require learning a new architectural style; it fit into the one Java developers already know.

chatClient.prompt()
  .system("Summarise support complaints factually")
  .user(complaint)
  .options(temperature(0.2))
  .call()
  .entity(ComplaintSummary.class);

Try It Yourself (Small Snippet)

Switching model providers is a configuration change, not a code rewrite. In application.properties:

01spring.ai.openai.api-key=${OPENAI_API_KEY}02spring.ai.openai.chat.options.model=gpt-4o-mini

Try mentally rewriting this for a different provider (say, Amazon Bedrock) — notice that ComplaintSummaryService's Java code above wouldn't need to change at all; only this configuration would.

Real Company Angle

Spring AI is useful when a team wants model access, structured responses, advisors and tool calling through conventions that fit existing Spring applications. Evaluate it through maintainability, provider portability, tests and operational fit.

Common Mistakes

1. Hardcoding API keys directly in code — always externalize them via application.properties or environment variables, exactly as you would for a database password.
2. Creating a new ChatClient for every request — build it once, usually through the injected builder, and reuse the managed bean.
3. Forgetting that switching providers is meant to be a config change — if changing providers requires touching business logic code, the abstraction isn't being used correctly.
4. Not handling the case where the LLM call fails or times out — a production support tool needs a fallback message, not an unhandled exception reaching the customer.

Persona Wrap-Up

Karthik is pleasantly surprised: "This feels almost identical to our AccountService pattern from Byte 1 — just with a ChatClient instead of a database call." Meena tests the summarizer: "This actually saves our support team real time on long complaint threads." Divya adds: "That's the whole point of Spring AI — it didn't ask Java developers to learn a new paradigm, it extended the one they already trust."

Compare & Contrast

Mini Practice Task

Design (on paper) a new ComplianceQAService with one method, answerQuestion(String question), that uses ChatClient to answer general compliance questions. Write the method signature and the one-line chatClient.prompt()...call().content() chain you'd use — no need to run it.

Key Takeaways

• Spring AI brings LLM calling into Java using the same Spring Bean and dependency injection patterns Java developers already know from Spring Boot.
• The core object is ChatClient, used with a fluent chain: .prompt().user(...).call().content().
• Switching model providers (OpenAI, Azure, Bedrock, Google, and others) is meant to be a configuration change, not a code rewrite.
• Spring AI provides a Spring-native abstraction for model calls and related AI application patterns.
• Keeping AI calls inside the familiar Spring ecosystem reduces both security risk and developer ramp-up time for enterprises.

FAQ / Knowledge Check

Q1: What is the Java/Spring AI equivalent of LangChain's ChatOpenAI? Spring AI's ChatClient, typically created via an auto-configured ChatClient.Builder bean.

Q2: Does switching from OpenAI to another model provider require rewriting your service code? No — it should only require changing the Spring Boot starter dependency and configuration properties, not the business logic.

Q3: Why shouldn't you create a new ChatClient for every single request? It is unnecessary and harder to manage. Build it once, typically at startup through dependency injection, and reuse it across requests.

Knowledge Check:

1. What are the three steps in the fluent ChatClient call chain shown in this byte?
2. True/False: Switching LLM providers in Spring AI usually requires rewriting your @Service class.
3. What percentage of Java developers building AI apps report using Spring AI?

(Answers: 1. .prompt(), .user(...), .call() (followed by .content() to get the text); 2. False — it's meant to be a configuration change, not a code change; 3. Spring-native configuration and application integration)

Next Byte: RAG in Spring Boot—use Spring AI vector-store support to answer from approved documents. New to RAG? Start with What Is RAG?, then return here for the Java implementation.

Interactive Knowledge Check

References and Further Reading

• Spring AI Reference
• Spring AI ChatClient
• Spring Boot Reference