LM Studio is a cross-platform desktop app designed to make it easy to explore and use large language models (LLMs) directly on your computer. This powerful open source application offers users an intuitive user interface to experiment with and configure LLM models, providing a smooth and powerful user experience.
Key Features of LM Studio
LM Studio allows users to easily download and manage any ggml compatible template from Hugging Face directly from within the app. This gives users instant access to a wide range of LLM templates to suit their specific needs. Thanks to the intuitive user interface it makes the approach to advanced language models accessible even to those who do not have extensive technical experience.
LM Studio offers the ability to apply configurations to your model that allow you to customize settings for optimal results. For example, it is possible to set options to regulate parameters such as the generation temperature (degree of variability of the responses) and the maximum length of the text produced.
Another significant advantage is that LM Studio is open source software, which means that users can use it for free.
How to use LM Studio?
LM Studio is compatible with all operating systems and you can download the version suited to your needs from the official website at the following link: https://lmstudio.ai/.
Once installed and started we will find ourselves in front of the home page where some of the latest innovations in the LLM field are presented and it is possible to download a template to start experimenting with the tool. If you want to try a specific model that is not present on the home page, just access the search function by clicking on the magnifying glass found in the side menu and search by model name.
Chat with AI
For our example we decided to use Google’s Gemma 2B Instruct template which is quite light (2.67 GB) and can run easily on all devices. If you can’t find it directly on the home page you can search for it by filtering by this string: “gemma-2b-it-q8_0.gguf”.
By accessing the “My models” feature from the side menu it will be possible to verify the completion of the download:
At this point you can try chatting with the AI by going to the “AI Chat” tab and selecting the template we just downloaded.
To test it I set the following system message:
You are an assistant providing support on a computer science blog that covers topics related to artificial intelligence and LLMs. And trying to ask the AI “Hi, what is this blog about?” I received the following response:
Use the LM Studio Local Server to invoke REST API
Another very interesting feature of LM Studio is its ability to expose the model via REST API in a very similar way to what we have already done using Ollama in a previous article.
To use this functionality simply go to the “Local Server” tab and click on the “Start Server” text:
It is possible to replicate the previous example by invoking the REST API:
curl http://localhost:1234/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "lmstudio-ai/gemma-2b-it-GGUF",
"messages": [
{ "role": "system", "content": "You are an assistant providing support on a computer science blog that covers topics related to artificial intelligence and LLMs." },
{ "role": "user", "content": "Hi, what is this blog about?" }
],
"temperature": 0.7,
"max_tokens": -1,
"stream": false
}'The output will contain the response message and will have the following format:
{
"id": "chatcmpl-dd63vxfsxw70ys9v5rsrr0i",
"object": "chat.completion",
"created": 1715527092,
"model": "lmstudio-ai/gemma-2b-it-GGUF/gemma-2b-it-q8_0.gguf",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "This blog is about everything related to Artificial Intelligence and LLMs. We cover the latest research, trends, and applications of these powerful technologies in diverse areas such as machine learning, natural language processing, computer vision, and robotics."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 46,
"completion_tokens": 45,
"total_tokens": 91
}
}In conclusion, LM Studio represents a powerful and versatile tool for exploring and using large language models directly on your computer. If the topic interests you, I invite you to continue reading the related articles.