Spring AI – Integration with Ollama

/ Spring AI guide

After seeing how to integrate Spring AI with OpenAI, the time has come to try Ollama, the LLM engine that allows you to install AI models locally. If you haven't already done so, we suggest you follow our tutorial on how to install Ollama.

In this article we will create a conversational chatbot starting from the Spring Boot project that we have already configured in the previous chapters of the guide and using the llama3 model.

Practical tutorial

The procedure for integrating Spring AI with Ollama is quite similar to that of OpenAI. First you need to set the maven dependency from the appropriate Spring AI module:

<dependency>
   <groupId>org.springframework.ai</groupId>
   <artifactId>spring-ai-ollama-spring-boot-starter</artifactId>
</dependency>

Subsequently, it is necessary to define a RestController and expose an endpoint that takes as input (query string) the message to be processed using the LLM model.

package com.example.aidemo.controller;

import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.ollama.OllamaChatModel;
import org.springframework.ai.ollama.api.OllamaOptions;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class OllamaAiRestController {
	
	private final ChatModel chatModel;
	
	@Autowired
    public OllamaAiRestController(OllamaChatModel chatModel) {
        this.chatModel = chatModel;
    } 
	
	@GetMapping("/ai/ollama/chat")
    public String generate(@RequestParam(value = "message") String message) {
		ChatResponse response = chatModel.call(
			    new Prompt(
			        message,
			        OllamaOptions.create()
			            .withModel("llama3")
			            .withTemperature(0.4f)
			    ));
		return response.getResult().getOutput().getContent();
	}

}

The code is quite simple but we comment it anyway:

• First of all, we note that we have defined the client for integration with Ollama by instantiating an OllamaChatModel via dependency-injection.
• Subsequently, on line 24 we invoked the “call” method, passing as input a Prompt type object which in turn contains the message and a series of options.
• Among the options we have highlighted the name of the model to use (“llama3”) and the temperature which indicates how “creative” the model output should be (1=maximum creativity; 0=precision).

If you have low-performance hardware, the llama3 model may be too complex to run locally. In these cases I suggest you try a lighter model like “phi3”. Fortunately, Ollama is very versatile and allows you to run various more or less complex models locally.

Once we understand the example code, we can test it by starting our Spring AI project and invoking the endpoint we defined:

http://localhost:8080/ai/ollama/chat?message=can you tell me a story?

After a few seconds the application will respond with a nice story to read!

The integration of Spring AI with Ollama represents a significant step forward in the conversational AI landscape. Offering a complete, easy-to-use solution, this integration allows anyone to create intelligent, personalized, scalable chatbots that can revolutionize the way software interacts with humans. Compared to other cloud solutions, Ollama offers a free solution that can also be installed on your home PC.