Install and use Ollama on your local machine
Ollama
Ollama is a tool allowing you to utilize LLMs in an easily manageable interface. Ollama is self-hosted and not a fenced in SaaS. This tutorial will only focus on using Ollama under Linux.
Installation
Those of you familiar with Docker will feel right at home with Ollama. In this tutorial, I will focus in installing and using Ollama via docker.
Systemd Service
Foremost, let's create a docker-compose service for systemd. With this service, we will be able to start Ollama on boot. Note: the following commands should be prefixed with 'sudo' or you should log in as root.
Create the compose/ollama directories:
mkdir -p /etc/docker/compose/ollama
Move to the directory and create a docker-compose.yml file.
cd /etc/docker/compose/ollama
vim docker-compose.yml
Add the following content:
version: "3"
services:
ollama:
image: ollama/ollama:latest
container_name: 'ollama'
volumes:
- ollama:/root/.ollama
ports:
- 127.0.0.1:11434:11434
volumes:
ollama:
Create the docker-compose systemd service file:
cd /etc/systemd/system/
vim docker-compose@.service
Add the following content:
[Unit]
Description=%i service with docker compose
PartOf=docker.service
After=docker.service
[Service]
Type=oneshot
RemainAfterExit=true
WorkingDirectory=/etc/docker/compose/%i
ExecStart=/usr/bin/docker compose up -d --remove-orphans
ExecStop=/usr/bin/docker compose down
[Install]
WantedBy=multi-user.target
Reload the systemd daemon
systemctl daemon-reload
If you want, you can enable the service so that it will start on system boot
systemctl enable docker-compose@ollama.service
Start the ollama service and be patient. It can take a minute or two while the ollama image is being pulled.
systemctl start docker-compose@ollama.service
Check to see if the ollama service is running:
docker ps
You should see something like this:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ec5468f50462 ollama/ollama:latest "/bin/ollama serve" 57 minutes ago Up 57 minutes 11434/tcp ollama
In order to more easily access the ollama command, add the following alias to your ~/.bashrc file
alias ollama="docker exec -it ollama ollama"
Using Ollama
At this point you should be ready to go. Go to the official Ollama website and browse the available models. After choosing one, open a terminal and execute the pull command. In this example, I will pull the llama3 model to my local machine.
ollama pull llama3
You can view a list of locally installed models with the following command
ollama list
After pulling the model, run the following command to start the selected model
ollama run llama3
You should see the following prompt:
>>> Send a message (/? for help)
Now you can ask as many questions as you want.
My humble thoughts about AI
People have been throwing the words data science, deep learning, large language models around a lot lately. Artificial intelligence - we hear about it all the time. I am usually an early adopter of new technologies, however with AI I have been cautious.
The Stifling of the Creative Mind
"Why bother reading something no one was bothered enough to write"
To me, this statement strikes a chord. I have always been driven by a strong desire to create - whether it be art, music, writing, coding or other creative endeavors. Now, a machine, can produce results in most of these fields within a matter of seconds. In the last two years, that thought has sometimes been water to my flame of creativity. Why create when a machine can do the same in a few seconds? No bueno.
The Future
Suffice to say, for the time being, we are in the narrow/weak phase of AI development, which according to many futurists such as Ray Kurzweil will lead to what has been coined "The Singularity". The singularity is as creepy as it sounds. At this point, you had better brush up on the future of humanity by first watching a few dystopian Sci-fi movies:
- The Terminator 1984
- The Matrix 1999
- Idiocracy 2006
- Her 2013
Upon reaching "The Singularity", it seems the futures presented in those films could become a reality. Do we get a benevolent or tyrannical superintelligence? Will this be the birth of Skynet? Do we quickly evolve into bloated, greasy couch stains driven only by base pleasures? Will we be reduced to child-like dependents? Will the lines of reality become so blurred that most of us choose to live in the Matrix?
Maybe we should be more optimistic and hope for a future similar to the one presented in Star Trek. Man and machine are harmonized and work together. Humanity is not driven by greed, hatred or the desire to forcibly exert power over others... Don't misunderstand me, I am a serious Start Trek fan, but I tend to have a more pessimistic view of the singularity should it come to fruition. Okay, so with all that said, let's use AI as a tool.
AI as a Tool
As long as AI does not stifle or suffocate human creativity; as long as AI can be used as a tool or assistant in the creative process, I am not against using it. As long as the predicted singularity has not arrived to wipe out humanity, let's be realistic. Weak AI is a tool - let's use it!
Final Thoughts
Ollama is an interesting tool with many use cases. I like that fact that it is self-hosted and will even run on your laptop. I will be playing with Ollama a lot more in the future and have also integrated it into my workflow:
- NeoVim integration for code completion, refactoring and code snippet generation. An article is on the way.
- GUI interface for browser access
I have found is slow, but this depends on the host machine. You can speed up the process with a suitable GPU(s). Let us know what you think about ollama as a tool. What do you think about AI or the singularity in general?
