Set up a Local LLM for Neovim on Mac

editorllmneovimvim

February 26, 2024  |  2 min read

TLDR

  1. Download and install ollama

    # For Mac, via homebrew
$ brew install ollama

  2. Pull the model of your choice

    # On one terminal, spin up ollama
$ ollama serve
 
# On another terminal, pull the suitable model for your machine
$ ollama pull mistral

  3. Configure neovim

    • Add plugin nomnivore/ollama.nvim and make ollama serve on start
    • (Optional) Add in progress status blinker to Lualine

Steps

1. Download and Install Ollama

$ brew install ollama

Check out ollama.com for other ways to install and run ollama on other OSs.

2. Pull the Model of Your Choice

Spin up Ollama on one terminal and use another to pull the model(s). Available models can be found on Hugging Face. There is a guide that helps you pick one, though.

# On one terminal
$ ollama serve
 
# On another terminal
$ ollama pull mistral

3. Configure Neovim

Add Neovim plugin nomnivore/ollama.nvim

Add the "adapter", nomnivore/ollama.nvim, to Neovim (See here).

Next, have the Neovim plugin help serve ollama for us, instead of having to do it manually:

{
  "nomnivore/ollama.nvim",
  -- ...omitted
 
  opts = {
    serve = {
      -- set to true here
      on_start = true,
 
      -- and change the below if you want to serve Ollama in another way
      command = "ollama",
      args = { "serve" },
      stop_command = "pkill",
      stop_args = { "-SIGTERM", "ollama" },
    },
  },
}

(Optional) Add in Progress Status Blinker to Lualine

This is optional, but if you use lualine and you want a blinking status icon for Ollama in Neovim:

-- Refer to https://github.com/nomnivore/ollama.nvim#status for details
{
  function()
    local ollama_status = require("ollama").status()
 
    -- change the icons to ones you see fit
    local icons = {
      "󱙺", -- nf-md-robot-outline
      "󰚩" -- nf-md-robot
    }
 
    if ollama_status == "IDLE" then
      return icons[1]
    elseif ollama_status == "WORKING" then
      return icons[os.date("%S") % #icons + 1] -- animation
    end
  end,
  cond = function()
    return package.loaded["ollama"] and require("ollama").status() ~= nil
  end,
},

Refs