Spring AI Interview Questions (2026): ChatClient, RAG, Tool Calling and MCP

These Spring AI interview questions cover what Java teams are actually asked in 2026: ChatClient versus ChatModel, advisors, chat memory, structured output, RAG with a vector store, tool calling and MCP. Every answer is written against Spring AI 2.0, the line built for Spring Boot 4 and Spring Framework 7, so you will not be caught out by APIs that were removed in the 1.x to 2.0 upgrade.

The page is for Java and Spring Boot developers with one to eight years of experience who are interviewing for roles that now include "build an AI feature" in the job description. Each question starts with a short, direct answer you can say out loud, then adds the detail, code and traps that separate a prepared candidate from someone who skimmed a tutorial.

There are 39 questions, grouped from basics to production scenarios. Where it matters, a callout tells you what the interviewer is really checking.

In this guide

• Spring AI basics: ChatClient, ChatModel, prompts and structured output
• Advisors, chat memory and RAG
• Spring AI tool calling and MCP
• Advanced: observability, cost, testing and safety
• Scenario-based Spring AI interview questions
• FAQ

Spring AI basics: ChatClient, ChatModel, prompts and structured output

What is Spring AI?

Spring AI is the Spring project for building AI features in Java. It gives you portable APIs for chat, embeddings, images and audio, plus the pieces an AI feature needs around the model: prompt templates, conversation memory, vector stores, retrieval augmented generation (RAG), tool calling, MCP, structured output and observability.

The value is the same as the rest of Spring: you write against an abstraction (ChatClient, EmbeddingModel, VectorStore), pick a provider with a starter and a few properties, and get auto-configuration, Micrometer metrics and testability for free. Swapping OpenAI for Anthropic, Amazon Bedrock or a local Ollama model is mostly a dependency and configuration change, not a rewrite.

Which Spring AI version should you know for interviews in 2026?

Know Spring AI 2.0. It went GA in June 2026, the current stable release is 2.0.2 (October 2026), and it is designed for Spring Boot 4.0/4.1 and Spring Framework 7. The 1.x line (1.0 and 1.1) targets Spring Boot 3 and is what you will meet in older codebases.

The 2.0 changes worth naming in an interview: ChatClient is now clearly the main API; the tool-calling loop moved out of each ChatModel into a ToolCallingAdvisor; model options are immutable and built with builders; the .options segment disappeared from property keys (spring.ai.openai.chat.model); JSON handling moved to Jackson 3; and the MCP annotations (@McpTool, @McpResource, @McpPrompt) are now part of Spring AI itself. The official upgrade notes list every rename.

What the interviewer is checking: whether your knowledge is current. Saying "I'd set internalToolExecutionEnabled to false" or "use FunctionCallback" tells them you learned from a 1.x tutorial; both were removed in 2.0.

What is the difference between ChatClient and ChatModel?

ChatModel is the low-level, provider-specific interface: you give it a Prompt and get a ChatResponse. ChatClient is the fluent, high-level API on top of it that adds default system prompts, advisors (memory, RAG, logging), tools, templates and conversion to Java types. Application code should use ChatClient.

A good way to put it: ChatModel is like a JDBC driver, ChatClient is like JdbcClient. In 2.0 the gap grew, because the tool-calling loop now lives in an advisor inside ChatClient. Calling a ChatModel directly with tools gives you the model's tool-call request back; nothing runs the tool for you.

The diagram shows how the pieces fit: your code talks to ChatClient, a chain of advisors enriches the request, and the ChatModel for your provider makes the HTTP call.

@RestController
class AssistantController {

    private final ChatClient chatClient;

    AssistantController(ChatClient.Builder builder) {
        this.chatClient = builder
                .defaultSystem("You are a concise assistant for a Java training site.")
                .build();
    }

    @GetMapping("/ask")
    String ask(@RequestParam String q) {
        return chatClient.prompt()
                .user(q)
                .call()
                .content();
    }
}

How do you add Spring AI to a Spring Boot project?

Import the spring-ai-bom in dependencyManagement, add the starter for your model provider (for example spring-ai-starter-model-openai or spring-ai-starter-model-ollama), and set the API key and model in application.yml. Auto-configuration then gives you a prototype-scoped ChatClient.Builder to inject.

spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        model: gpt-5-mini

Never hard-code the key; read it from the environment or a secret store. If more than one model starter is on the classpath, spring.ai.model.chat selects which chat auto-configuration is active (for example openai or ollama), and none disables it.

How do you switch from OpenAI to a local Ollama model?

Replace the OpenAI starter with spring-ai-starter-model-ollama, point spring.ai.ollama.base-url at the Ollama server (default http://localhost:11434) and set spring.ai.ollama.chat.model. Code written against ChatClient does not change.

What does change: output quality, latency, context window and tool-calling ability vary a lot between models, so re-run your evaluation tests after a switch. Set spring.ai.ollama.init.pull-model-strategy to when_missing if you want the app to pull the model at startup instead of failing.

What are messages and roles in a Spring AI prompt?

A Prompt is a list of messages plus options. Each message has a role: system (instructions and persona), user (the human input), assistant (earlier model replies) and tool (a tool's result sent back to the model). Spring AI models these as SystemMessage, UserMessage, AssistantMessage and ToolResponseMessage.

The model has no memory between calls. Anything it should "remember", including earlier turns of the conversation, must be in the messages you send. That single fact explains why chat memory, RAG and token cost all work the way they do.

How do prompt templates work in Spring AI?

Templates are rendered by StTemplateRenderer (StringTemplate) by default, with {name} placeholders filled through .param(). If the prompt itself contains braces, such as JSON examples, switch the delimiters to something like < and >.

String answer = chatClient.prompt()
        .user(u -> u.text("List 3 interview topics for a {level} {stack} developer")
                .param("level", "senior")
                .param("stack", "Spring Boot"))
        .call()
        .content();

Keep system prompts in .st files under src/main/resources and load them as a Resource; they are easier to review and version than long Java strings. NoOpTemplateRenderer turns templating off when you want to send text exactly as written.

What is the difference between call() and stream()?

call() waits for the whole answer and returns it as String (content()), ChatResponse (with metadata such as token usage) or a Java type (entity()). stream() returns a Reactor Flux of partial content as the model generates it, which you can push to a browser with Server-Sent Events.

A detail people miss: call() does not hit the model by itself. The request runs when you call a terminal method such as content(), chatResponse() or entity(). Use streaming for chat UIs where time-to-first-token matters; use call() for back-end jobs, classification and anything you need to parse.

How do you get structured output from an LLM in Spring AI?

Call .entity(SomeRecord.class) instead of .content(). Spring AI derives a JSON schema from the record, tells the model to answer in that format, and maps the JSON back with BeanOutputConverter. Use a ParameterizedTypeReference for generic types such as List<Ticket>.

enum Priority { LOW, MEDIUM, HIGH }

record TicketTriage(String category, Priority priority, String summary) {}

TicketTriage triage = chatClient.prompt()
        .system("Classify the customer email for the support queue.")
        .user(emailBody)
        .call()
        .entity(TicketTriage.class);

The prompt-based approach is best effort: a model can still wrap JSON in prose or code fences. "How do you make structured output reliable?" in the Advanced section covers the switches that fix this.

What are embeddings, and how does similarity search work?

An embedding is a vector of floats that represents the meaning of a text; texts with similar meaning get vectors that point in similar directions. Spring AI exposes this through EmbeddingModel.embed(String), which returns a float[]. A vector store finds the stored chunks whose vectors are closest to the question's vector, usually by cosine similarity.

This plain-Java program shows the idea with toy four-dimensional vectors (real models return hundreds or thousands of dimensions). It ranks chunks by cosine similarity, then applies a top-k limit and a similarity threshold, exactly the two knobs you set on a SearchRequest.

import java.util.Comparator;
import java.util.List;

public class SimilarityDemo {

    record Chunk(String text, float[] vector) {}

    record Hit(String text, double score) {}

    // cosine similarity = dot(a, b) / (|a| * |b|)
    static double cosine(float[] a, float[] b) {
        double dot = 0, na = 0, nb = 0;
        for (int i = 0; i < a.length; i++) {
            dot += a[i] * b[i];
            na += a[i] * a[i];
            nb += b[i] * b[i];
        }
        return dot / (Math.sqrt(na) * Math.sqrt(nb));
    }

    public static void main(String[] args) {
        // Toy 4-dimensional "embeddings". Real models return hundreds or thousands of dimensions.
        List<Chunk> store = List.of(
                new Chunk("Refunds are processed within 5 working days", new float[] {0.9f, 0.1f, 0.0f, 0.2f}),
                new Chunk("You can return an unopened item within 30 days", new float[] {0.7f, 0.3f, 0.1f, 0.1f}),
                new Chunk("Our office is closed on public holidays", new float[] {0.1f, 0.0f, 0.9f, 0.3f}),
                new Chunk("Shipping to Europe takes 7-10 days", new float[] {0.2f, 0.8f, 0.1f, 0.4f}));

        float[] question = {0.85f, 0.2f, 0.05f, 0.15f}; // "How long does a refund take?"
        int topK = 2;
        double threshold = 0.75;

        List<Hit> hits = store.stream()
                .map(c -> new Hit(c.text(), cosine(question, c.vector())))
                .sorted(Comparator.comparingDouble(Hit::score).reversed())
                .toList();

        hits.forEach(h -> System.out.printf("%.3f  %s%n", h.score(), h.text()));
        System.out.println("--- topK=" + topK + ", threshold=" + threshold + " ---");
        hits.stream()
                .filter(h -> h.score() >= threshold)
                .limit(topK)
                .forEach(h -> System.out.println("context: " + h.text()));
    }
}

Output (run on JDK 25):

0.990  Refunds are processed within 5 working days
0.982  You can return an unopened item within 30 days
0.483  Shipping to Europe takes 7-10 days
0.207  Our office is closed on public holidays
--- topK=2, threshold=0.75 ---
context: Refunds are processed within 5 working days
context: You can return an unopened item within 30 days

Notice the second hit: it is about returns, not refunds, yet it scores almost as high. Similar is not the same as relevant, which is why RAG answers need evaluation and not just a high threshold.

Advisors, chat memory and RAG

What is an advisor in Spring AI?

An advisor is an interceptor around a ChatClient call. It can change the request before the model sees it (add history, add retrieved documents, block sensitive input) and change or inspect the response afterwards (save it to memory, log it, validate it). Memory, RAG, tool calling, logging and structured-output validation are all advisors.

Built-in ones you should be able to name: MessageChatMemoryAdvisor, VectorStoreChatMemoryAdvisor, QuestionAnswerAdvisor, RetrievalAugmentationAdvisor, ToolCallingAdvisor, SafeGuardAdvisor, SimpleLoggerAdvisor and StructuredOutputValidationAdvisor. If you know servlet filters or Spring AOP, the model is the same: a chain where each link calls the next one.

How does advisor order work, and why does it matter?

Each advisor has a getOrder() value. On the way in, the lowest order runs first; on the way out, the order reverses, like nested layers of an onion. Order decides what each advisor can see: memory must wrap the tool loop, and anything ordered after ToolCallingAdvisor runs again on every tool round trip.

The defaults in 2.0 are deliberate. Memory advisors sit at HIGHEST_PRECEDENCE + 200, the auto-registered ToolCallingAdvisor at HIGHEST_PRECEDENCE + 300, and QuestionAnswerAdvisor and SimpleLoggerAdvisor default to order 0. So the logger sees every intermediate model call in a tool loop, and memory stores only the final exchange. If you want retrieval to happen once per question rather than once per round trip, give the RAG advisor an order below the tool advisor's. The tool advisor's order is configurable with spring.ai.chat.client.tool-calling.advisor-order.

What the interviewer is checking: that you can reason about a pipeline, not just configure one. "Why is my logger printing three requests for one question?" is a real support ticket, and the answer is advisor order.

How do you write a custom advisor?

Implement CallAdvisor (and StreamAdvisor if you stream), do your work around chain.nextCall(request), and return a name and an order. Register it with defaultAdvisors(...) on the builder or .advisors(...) per request.

public class TokenUsageAdvisor implements CallAdvisor {

    private static final Logger log = LoggerFactory.getLogger(TokenUsageAdvisor.class);

    @Override
    public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain chain) {
        long start = System.nanoTime();
        ChatClientResponse response = chain.nextCall(request);
        ChatResponse chatResponse = response.chatResponse();
        if (chatResponse != null) {
            Usage usage = chatResponse.getMetadata().getUsage();
            log.info("tokens in={} out={} in {} ms", usage.getPromptTokens(), usage.getCompletionTokens(),
                    (System.nanoTime() - start) / 1_000_000);
        }
        return response;
    }

    @Override
    public String getName() {
        return "TokenUsageAdvisor";
    }

    @Override
    public int getOrder() {
        return 0; // inside the tool loop: one log line per model round trip
    }
}

For simple before/after logic, BaseAdvisor lets you implement just before(...) and after(...). An advisor can also call the rest of the chain more than once, through chain.copy(this).nextCall(...); that is how the tool loop and the structured-output retry work.

How does chat memory work in Spring AI?

A ChatMemory bean stores recent messages per conversation, and MessageChatMemoryAdvisor loads them into each request and saves the new exchange afterwards. The default MessageWindowChatMemory keeps the last 20 messages. You must pass a conversation id with every call.

@Bean
ChatClient supportClient(ChatClient.Builder builder, ChatMemory chatMemory) {
    return builder
            .defaultAdvisors(MessageChatMemoryAdvisor.builder(chatMemory).build())
            .build();
}

String reply = supportClient.prompt()
        .user(message)
        .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, userId + ":" + sessionId))
        .call()
        .content();

Storage is pluggable through ChatMemoryRepository: in-memory by default, or JDBC, Redis, MongoDB, Cassandra or Neo4j. For JDBC, add spring-ai-starter-model-chat-memory-repository-jdbc; set spring.ai.chat.memory.repository.jdbc.initialize-schema=never if Liquibase or Flyway owns your schema.

What the interviewer is checking: the 2.0 change. There is no default conversation id any more; leaving it out throws IllegalArgumentException. In 1.x a missing id silently shared one "default" conversation between users, a classic data leak.

What is the difference between chat memory and chat history?

Chat memory is the slice of the conversation you send to the model so it keeps context. Chat history is the full record of everything said. Spring AI's ChatMemory is built for the first job only; it trims old messages by design. Store the full history yourself, for example in a normal table through Spring Data, if you need audit, analytics or "show my past chats".

What is a VectorStore in Spring AI?

VectorStore is the portable interface for storing documents with their embeddings and searching them by similarity. You call add(List<Document>) to store chunks (Spring AI embeds them with the configured EmbeddingModel) and similaritySearch(SearchRequest) to query with topK, similarityThreshold and a metadata filter.

Implementations include PostgreSQL with pgvector, Redis, Elasticsearch, OpenSearch, MongoDB Atlas, Milvus, Qdrant, Chroma, Weaviate, Pinecone and a SimpleVectorStore for tests. In a Spring Boot shop already on PostgreSQL, pgvector is the usual first choice because there is no new database to run. The full build is in our Spring AI RAG tutorial with PGVector.

What is RAG, and how do you implement it in Spring AI?

Retrieval augmented generation means: find the chunks of your own documents most related to the question, put them in the prompt, and tell the model to answer only from them. In Spring AI the shortest path is QuestionAnswerAdvisor, which runs the similarity search and adds the results to the user message for you.

var rag = QuestionAnswerAdvisor.builder(vectorStore)
        .searchRequest(SearchRequest.builder().topK(4).similarityThreshold(0.6).build())
        .build();

String answer = chatClient.prompt()
        .advisors(rag)
        .user("How long do refunds take?")
        .call()
        .content();

RAG has two halves. Ingestion runs offline: read documents, split them into chunks, embed and store. Querying runs per request: embed the question, search, augment the prompt, generate. Most quality problems come from the ingestion half (bad chunking, missing metadata), not from the model.

What is the difference between QuestionAnswerAdvisor and RetrievalAugmentationAdvisor?

QuestionAnswerAdvisor (artifact spring-ai-vector-store-advisor) is a ready-made, single-step RAG: search the vector store, stuff the results into a prompt template. RetrievalAugmentationAdvisor (artifact spring-ai-rag) is modular: you plug in a DocumentRetriever, optional query transformers, a query expander, a document joiner and a query augmenter.

Advisor rag = RetrievalAugmentationAdvisor.builder()
        .queryTransformers(RewriteQueryTransformer.builder()
                .chatClientBuilder(chatClientBuilder)
                .build())
        .documentRetriever(VectorStoreDocumentRetriever.builder()
                .vectorStore(vectorStore)
                .similarityThreshold(0.5)
                .topK(5)
                .build())
        .build();

Reach for the modular version when users ask follow-up questions ("and for Europe?") that need rewriting with CompressionQueryTransformer, when content is in another language (TranslationQueryTransformer), or when one query is not enough (MultiQueryExpander).

What happens when the retriever finds nothing relevant?

With RetrievalAugmentationAdvisor, the default ContextualQueryAugmenter does not allow empty context: the model is told not to answer the question. Set allowEmptyContext(true) if the model may fall back to its general knowledge. QuestionAnswerAdvisor's default template tells the model to answer from the context and say so when the answer is not there.

For a support or compliance bot, refusing is the right default. A confident answer that is not in your documents is worse than "I don't know".

How do you filter RAG results by tenant or document type?

Store metadata on each Document at ingestion (tenant, source, language, version) and filter at query time. Spring AI has a portable filter syntax, such as tenant == 'acme' && type in ['faq','policy'], which each vector store translates to its own query language.

String answer = chatClient.prompt()
        .advisors(rag)
        .advisors(a -> a.param(QuestionAnswerAdvisor.FILTER_EXPRESSION, "tenant == '" + tenantId + "'"))
        .user(question)
        .call()
        .content();

Build the filter from the authenticated user on the server, never from request input, or one tenant can read another's documents. FilterExpressionBuilder gives you a type-safe alternative to the string form.

Spring AI tool calling and MCP

What is tool calling in Spring AI?

Tool calling (also called function calling) lets the model ask your application to run a method, such as looking up an order or checking stock, and then use the result in its answer. The model never runs code. It returns a structured request with a tool name and JSON arguments; Spring AI runs the matching Java method and sends the result back.

@Component
public class OrderTools {

    private final OrderService orders;

    public OrderTools(OrderService orders) {
        this.orders = orders;
    }

    @Tool(description = "Get the status and expected delivery date of an order by its numeric id")
    public OrderStatus orderStatus(@ToolParam(description = "The order id, for example 42") long id) {
        return orders.status(id);
    }
}

String answer = chatClient.prompt()
        .user("Where is order 42?")
        .tools(orderTools)
        .call()
        .content();

The description is the most important line: it is the only thing the model reads when deciding whether to call the tool. Say what it does, when to use it and what the arguments mean.

Walk me through the tool-calling loop.

Spring AI sends the messages plus the JSON schema of every registered tool. If the model replies with a tool call, ToolCallingAdvisor hands it to ToolCallingManager, which runs the method, appends the result to the conversation and calls the model again. This repeats until the reply has no tool calls, and that final text is what content() returns.

Two consequences interviewers like to probe: one user question can cost several model calls (and their tokens), and the model can call several tools in one reply. ToolCallingManager.builder() lets you cap calls per tool and in total with maxCallsPerTool and maxTotalToolCalls. Since 2.0.2 there are default caps (40 calls per tool, 150 in total), also settable under spring.ai.tools.limits.*.

What changed in Spring AI tool calling in 2.0?

The loop moved from every ChatModel into a single ToolCallingAdvisor that ChatClient auto-registers in its advisor chain. ChatModel.call() now returns the model's raw tool-call request instead of running tools, internalToolExecutionEnabled was removed, the old FunctionCallback API is gone (use ToolCallback and FunctionToolCallback), and ToolCallAdvisor was renamed ToolCallingAdvisor.

To drive the loop yourself, turn off auto-registration per request with AdvisorParams.toolCallingAdvisorAutoRegister(false) or globally with spring.ai.chat.client.tool-calling.enabled=false, then call toolCallingManager.executeToolCalls(prompt, chatResponse) in your own loop, for example to ask a human to approve a risky action first.

What is ToolContext, and why not just pass everything as tool arguments?

ToolContext carries data from your code to the tool without sending it to the model, such as the tenant id or the authenticated user. Anything in a tool argument is chosen by the model and can be manipulated through the prompt; anything in the tool context is set by your server.

@Tool(description = "List the current user's open orders")
public List<OrderSummary> myOpenOrders(ToolContext context) {
    String customerId = (String) context.getContext().get("customerId");
    return orders.openOrdersFor(customerId);
}

chatClient.prompt()
        .user("What have I ordered recently?")
        .tools(orderTools)
        .toolContext(Map.of("customerId", currentUser.id()))
        .call()
        .content();

What the interviewer is checking: security instinct. A tool such as getOrders(customerId) where the model fills in the customer id lets any user ask for anyone's orders. Identity belongs in ToolContext.

What does returnDirect do on @Tool?

With returnDirect = true, the tool's result goes straight back to the caller instead of being sent to the model for another round. Use it when the tool output is already the final answer, for example a generated report link, and you want to save a model call or stop the model from rephrasing exact data. If the model calls several tools in one round, the result is returned directly only when all of them have returnDirect set.

How does Spring AI handle exceptions thrown by tools?

The exception is wrapped in a ToolExecutionException and passed to a ToolExecutionExceptionProcessor. The default sends a RuntimeException's message back to the model so it can recover or explain, and rethrows checked exceptions and errors. Set spring.ai.tools.throw-exception-on-error=true to throw everything to your code instead.

Be careful with what the message says: it goes to the model and may end up in the answer. "Order not found" is fine; a stack trace with a SQL statement is not.

What is MCP, and how does Spring AI support it?

MCP (Model Context Protocol) is an open protocol for exposing tools, resources and prompts to any AI application through a standard JSON-RPC interface. Spring AI ships both sides: MCP server starters that turn annotated Spring beans into MCP tools, and MCP client starters that connect to servers and hand their tools to ChatClient. For the protocol itself, read What is MCP?

// MCP server (spring-ai-starter-mcp-server-webmvc)
@Component
public class CatalogTools {

    @McpTool(name = "find_course", description = "Search the course catalog by keyword")
    public List<Course> findCourse(@McpToolParam(description = "Keyword such as kafka", required = true) String keyword) {
        return catalog.search(keyword);
    }
}

# MCP client (spring-ai-starter-mcp-client) in another app
spring:
  ai:
    mcp:
      client:
        streamable-http:
          connections:
            catalog:
              url: http://localhost:8081

The client auto-configuration exposes the remote tools as a ToolCallbackProvider (SyncMcpToolCallbackProvider for the default sync client), which you pass to .tools(provider) like any other tool source; the older .toolCallbacks(...) methods still work but are deprecated in 2.0. Since 2.0, Streamable HTTP is the default HTTP transport, SSE is deprecated, and stdio is still there for local, process-launched servers. We build a full server step by step in Build an MCP server in Java with Spring AI.

When would you use an @Tool method and when an MCP tool?

Use @Tool for logic that belongs to the application that calls the model: it is a plain method call in the same JVM, with no network hop. Use MCP when the same tools should serve several AI clients (your app, Claude Code, an IDE), when another team owns them, or when they are written in another language.

The two mix freely: both become ToolCallback objects, so one ChatClient call can use local @Tool methods and remote MCP tools in the same loop.

What the interviewer is checking: that you do not reach for MCP by reflex. An MCP server adds a process, a network call, authentication and versioning. If only one app uses the tool, a method is simpler and faster.

How do you handle an app with dozens of tools?

Every tool definition is sent on every request, so 50 tools cost tokens and confuse the model. Group tools so each ChatClient only gets what its use case needs, or use ToolSearchToolCallingAdvisor (new in 2.0): it indexes the full tool set and gives the model a single search tool to discover relevant tools on demand. Add spring-ai-starter-tool-search-advisor and enable it with spring.ai.chat.client.tool-search-advisor.enabled=true; it then replaces the default ToolCallingAdvisor.

Advanced: observability, cost, testing and safety

How do you monitor Spring AI in production?

Spring AI records Micrometer observations for ChatClient calls, advisors, model calls, tool calls and vector store operations. With Actuator and a Prometheus or OpenTelemetry backend you get latency timers (gen_ai.client.operation) and a token counter (gen_ai.client.token.usage, tagged input, output and total) per model, plus traces showing each step.

spring:
  ai:
    chat:
      client:
        observations:
          log-prompt: false      # true only while debugging locally
          log-completion: false
      observations:              # the ChatModel-level equivalents
        log-prompt: false
        log-completion: false

Prompt and completion logging is off by default for a reason: prompts often contain personal data. Turn it on only in development, and alert on token usage per endpoint so a prompt change that doubles cost is caught the same day.

How do you control token cost in a Spring AI application?

Send less and generate less. Keep system prompts short, cap memory with MessageWindowChatMemory.builder().maxMessages(...), retrieve fewer and smaller chunks, give each client only the tools it needs, and cap output with .options(ChatOptions.builder().maxTokens(300)). Then route easy tasks to a smaller, cheaper model.

One trap: OpenAI reasoning models reject max_tokens. The Spring AI docs say to use maxCompletionTokens (spring.ai.openai.chat.max-completion-tokens) for them instead, and the two settings cannot be combined.

Measure before optimizing: the token metric tells you whether input or output dominates. Input usually does, because history, retrieved context and tool schemas are resent on every round trip. Our LLM token optimization guide covers caching and model routing in more depth.

How do you test code that uses Spring AI?

Test in three layers. Unit tests mock the ChatModel (or wrap a stub in ChatClient.create(...)) so your controllers, tools and parsing run without a network call. Integration tests use Testcontainers for the vector store, and Ollama if you want a real model in CI. Quality tests use an evaluator, a second model call that grades the answer.

ChatClientResponse response = chatClient.prompt()
        .advisors(rag)
        .user(question)
        .call()
        .chatClientResponse();

@SuppressWarnings("unchecked")
List<Document> context = (List<Document>) response.context().get(QuestionAnswerAdvisor.RETRIEVED_DOCUMENTS);
String answer = response.chatResponse().getResult().getOutput().getText();

var evaluator = new RelevancyEvaluator(ChatClient.builder(judgeModel));
EvaluationResponse verdict = evaluator.evaluate(new EvaluationRequest(question, context, answer));
assertThat(verdict.isPass()).isTrue();

RelevancyEvaluator checks the answer fits the question and retrieved context; FactCheckingEvaluator checks a claim is supported by the context. Keep a fixed set of 20 to 50 real questions and run it on every prompt or model change. Model output is not deterministic, so assert on behavior and grades, never on exact strings.

How do you make structured output reliable?

Prefer the provider's native structured output where the model supports it, validate the JSON against the schema, and retry with the validation error when it fails. In 2.0 that is entity(Type.class, spec -> spec.useProviderStructuredOutput().validateSchema()), and StructuredOutputValidationAdvisor re-invokes the model with the error appended, up to a configured number of attempts.

TicketTriage triage = chatClient.prompt()
        .user(emailBody)
        .call()
        .entity(TicketTriage.class, spec -> spec
                .useProviderStructuredOutput()
                .validateSchema());

Native structured output is off by default because support varies. One known limit: OpenAI's Structured Outputs API does not accept a top-level JSON array, so wrap a List<T> in a record when you turn it on. Also keep the target type small and flat, use enums for fixed values, and still validate business rules in Java (a valid JSON priority can still be the wrong priority).

How do you protect a Spring AI app from prompt injection?

Assume any text the model reads (user input, retrieved documents, tool results, web pages) can contain instructions, and design so a hijacked model cannot do damage. Give tools least privilege, put identity in ToolContext, require human approval for destructive actions, and never let the model build SQL or shell commands from free text.

SafeGuardAdvisor blocks requests containing listed sensitive words, which is useful but shallow; it is not a defense against a well-crafted injection. Spring AI also ships no authentication on MCP HTTP servers by default: the docs warn the endpoint is open until you secure it with Spring Security or the MCP security project.

What the interviewer is checking: that you treat the LLM as an untrusted component. "We'll tell it in the system prompt not to do bad things" is the wrong answer.

Spring AI vs LangChain4j: which would you choose?

Both are mature Java libraries for LLM apps. Spring AI is the natural fit for a Spring Boot codebase: starters, auto-configuration, Micrometer observability and Spring idioms. LangChain4j is framework-agnostic, with integrations for Spring Boot, Quarkus and others, and is known for declarative "AI services" interfaces.

• Spring Boot team, Spring conventions: Spring AI.
• Quarkus, Micronaut or plain Java: LangChain4j is usually the smoother path.
• MCP server or client in Spring: Spring AI has first-party starters and annotations.
• Either way: the hard parts (chunking, evaluation, tool design, cost) are the same; the library is maybe 10% of the work.

A strong answer picks based on the existing stack and team, not on feature checklists that change every release.

Scenario-based Spring AI interview questions

Your RAG support bot answers questions that are not in the documents. How do you fix it?

First find out whether retrieval or generation is at fault. Log the retrieved chunks and their scores for the bad answers. If irrelevant chunks come back, fix retrieval: raise the similarity threshold, fix chunking, add metadata filters. If good chunks come back but the model still invents, tighten the template to answer only from context and refuse otherwise, and keep the default no-empty-context behavior. Then add the failing questions to your evaluation set so the fix stays fixed.

One question triggers 10 tool calls and the bill spikes. What do you do?

Look at the traces: which tools are called, with what arguments, and why the model keeps going. Common causes are vague tool descriptions, tools that return errors the model keeps retrying, and overlapping tools. Fix the descriptions and outputs, cap the loop with maxCallsPerTool and maxTotalToolCalls on ToolCallingManager (choose THROW or RETURN_ERROR_RESPONSE when the limit is hit), and alert on tool calls per request.

Users report seeing parts of other users' conversations. What is the likely cause?

A shared or predictable conversation id. Check every call that uses a memory advisor: the id must come from the authenticated session and be unique per user and conversation. On a 1.x app, a missing id fell back to "default", so all users shared one memory. Also check that a singleton bean is not holding per-user state, and that RAG filters and ToolContext use the server-side user, not a value from the request.

Legal says customer data may not leave your network. How do you move off a hosted model?

Run an open-weights model on your own infrastructure, for example with Ollama, and switch the starter and properties; ChatClient code stays the same. Then check the parts that do change: embeddings must also be local, and switching the embedding model means re-embedding every document, because vectors from different models are not comparable and dimensions often differ. Re-run the evaluation set, compare quality and latency, and size the GPU hardware before you promise a date.

Practise with the full question bank

If MCP and tool calling are a big part of the role, MCP for Java and Spring Developers is a step-by-step guide to the Model Context Protocol: you build a Spring AI MCP server and client, connect Claude Code, test with the Inspector, secure it and finish a capstone project.

For day-to-day work with AI assistants, The Ultimate AI Prompt Pack for Developers has 100 copy-paste prompts for debugging, DevOps, DSA, system design, APIs, microservices and career, in 10 PDFs.

Related reading

• Spring AI RAG tutorial: chat with your documents using PGVector
• Build an MCP server in Java with Spring AI
• What is MCP (Model Context Protocol)?

Official references: the ChatClient API and Tool Calling pages of the Spring AI reference.

FAQ

Is Spring AI production ready?

Yes. Spring AI has had GA releases since 1.0 in 2025, and 2.0 (GA June 2026) is the current line for Spring Boot 4. As with any LLM feature, production readiness depends more on your evaluation, cost controls and security than on the library.

Does Spring AI work with Java 25 and Spring Boot 4?

Yes. Spring AI 2.0 is built for Spring Boot 4.0 and 4.1 and Spring Framework 7, and runs on current LTS JDKs including Java 25. If your app is still on Spring Boot 3.x, use the Spring AI 1.x line until you upgrade Boot.

What are the most common Spring AI interview questions?

ChatClient vs ChatModel, how advisors and chat memory work, how to build RAG with a vector store, how the tool-calling loop works, structured output with entity(), and what MCP adds. Senior roles add observability, cost control, testing with evaluators and prompt-injection defenses.

Do I need to know Python to work on AI features in Java?

No. For applications that call hosted or local models, add RAG and use tools, Java with Spring AI covers the whole stack. Python matters more if you train or fine-tune models yourself, which most backend AI work does not involve.

Which vector database should I use with Spring AI?

Start with what you already run. If you use PostgreSQL, pgvector avoids running a new system, and its HNSW index keeps similarity search fast as the table grows. Move to a dedicated vector database only when measured scale or features require it.