Spring AI Tool Calling in Backend Systems

1. Execution-Driven Architecture Using Spring AI SDK for Healthcare Applications

Integrating LLMs into backend systems is no longer about building chat interfaces — it is about embedding intelligent decision-making into production services.

However, LLMs introduce a fundamental limitation:

They can reason about actions, but they cannot safely execute backend logic.

This creates a critical gap:

• LLMs cannot access databases directly
• They may hallucinate responses
• They cannot enforce business rules
• They lack deterministic execution

This challenge becomes even more critical in healthcare applications, where accuracy, data integrity, and security are non-negotiable.

To solve this, we need a system where:

• LLMs decide what should happen
• Backend systems execute what must happen

This is exactly what Spring AI Tool Calling enables.

It introduces a structured architecture where:

Spring AI SDK acts as the execution bridge between LLM reasoning and backend systems

2. Execution Flow Diagram

3. Spring AI SDK Setup (Foundation Layer)

Spring AI provides the runtime infrastructure for tool calling inside Spring Boot.

Without it, you would need to manually handle:

• tool schemas
• JSON parsing
• execution routing
• context injection

Maven Dependency

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
  <version>1.0.0</version>
</dependency>

application.yml

spring:
ai:
  openai:
    api-key: ${OPENAI_API_KEY}
    chat:
      options:
        model: gpt-4.1-mini
        temperature: 0.2

ChatClient Configuration

@Configuration
public class AIConfig {

  @Bean
  public ChatClient chatClient(ChatClient.Builder builder) {
      return builder
              .defaultSystem("You are a backend AI assistant with tool access.")
              .build();
  }
}

Why ChatClient Matters

ChatClient is not just an API wrapper.

It is:

The execution gateway that:

• sends prompts to LLM
• registers tools
• manages tool execution loop
• injects tool results back

4. Problem: Why Backend + LLM Fails Without Tool Calling

Without Tool Calling

• LLM generates answers without real data
• Backend logic leaks into prompts
• No execution guarantees
• No validation layer

Example

User: “What medications am I taking?”

LLM: guesses → incorrect → unsafe

With Spring AI Tool Calling

• LLM selects tool
• Backend executes tool
• Response is grounded in real data

5. Core Concept: Tool Calling (Spring AI Model)

Spring AI introduces a controlled execution loop:

Step 1: LLM Reasoning

LLM analyzes input: “User needs medication data”

Step 2: Tool Selection

LLM returns:

{
"tool": "GetMedicationsTool",
"arguments": {
  "userUuid": "123"
}
}

Step 3: Execution (Spring AI)

• resolves tool bean
• injects parameters
• executes method

Step 4: Response Injection

Tool output is returned to LLM context.

Step 5: Final Response

LLM produces a grounded response.

Key Principle

LLM = decision engine
Backend = execution engine

6. Why This Matters in Healthcare Systems

In healthcare applications, backend systems deal with sensitive and regulated data such as patient records, medications, and clinical observations.

In such environments:

• Incorrect responses can impact patient safety
• Data must come from trusted, auditable sources (EHR systems, databases)
• Strict access control (e.g., userUuid) is required
• All actions must be deterministic and traceable

This makes traditional LLM usage (which may hallucinate or bypass backend rules) unsuitable for production healthcare systems.

Spring AI Tool Calling ensures that:

The LLM never directly accesses critical data — it only decides which verified backend tool should be executed.

This guarantees:

• Data is always fetched from real backend systems
• Business rules remain enforced
• Responses are secure and reliable

What We’ll Build Next

Next, we’ll implement a healthcare chatbot using Spring AI, showing how tool calling connects backend services, executes domain-specific tools, and generates safe, context-aware responses.

7. System Architecture

Let’s start with architecture

8. Implementation (Spring Boot)

ChatController

@RestController
@RequestMapping("/chat")
public class ChatController {

  private final ChatOrchestrator orchestrator;

  public ChatController(ChatOrchestrator orchestrator) {
      this.orchestrator = orchestrator;
  }

  @PostMapping
  public String chat(String message, String sessionId, String userUuid) {
      return orchestrator.handle(message, sessionId, userUuid);
  }
}

ChatOrchestrator (Central Engine)

@Service
public class ChatOrchestrator {

  private final ChatClient chatClient;
  private final ToolCallbackProvider toolProvider;

  public ChatOrchestrator(ChatClient chatClient,
                          ToolCallbackProvider toolProvider) {
      this.chatClient = chatClient;
      this.toolProvider = toolProvider;
  }
  public String handle(String message, String sessionId, String userUuid) {

      return chatClient.prompt()
              .user(message)
              .tools(toolProvider.getToolCallbacks(userUuid))
              .call()
              .content();
  }
}

9. Tool Layer (Execution Layer)

GetMedicationsTool

@Component
public class GetMedicationsTool {
  @Tool(name = "GetMedicationsTool",
        description = "Fetch medications for user")
  public List<String> get(String userUuid) {
      return List.of("Metformin", "Aspirin");
  }
}

GetVitalsTool

@Component
public class GetVitalsTool {

  @Tool(name = "GetVitalsTool",
        description = "Fetch latest vitals")
  public Map<String, Object> get(String userUuid) {
      return Map.of("bp", "140/90", "pulse", 88);
  }
}

QuestionService

@Component
public class QuestionService {

  @Tool(name = "QuestionService",
        description = "Generate follow-up questions")
  public List<String> getRelevantQuestions(String symptoms, String userUuid) {
      return List.of("When did it start?", "Any nausea?");
  }
}

10. Internal Working of Spring AI Tool Calling

1. Tool annotations → schema generation
2. Schema sent to LLM
3. LLM selects tool
4. Spring AI executes tool
5. Result returned to LLM
6. LLM generates final response

11. Practical Example (Healthcare Use Case)

User:

“I feel dizzy”

Execution:

• OrchestratorAgent decides intent
• HealthAgent activated
• Tools executed:
  • GetVitalsTool
  • GetMedicationsTool
  • QuestionService

Final Response

“Your BP is elevated. You are on Metformin. When did symptoms start?”

12. Production Considerations

• userUuid propagation
• session management
• tool logging
• observability
• guardrails
• retry handling

13. Key Engineering Principles

• LLM is NOT your backend
• Tools must be deterministic
• Spring AI controls execution
• Backend remains source of truth