Generating Go code with Llama2 on your laptop

Written 10th of October 2023.

I would like to briefly show you how to generate Go code with Code Llama (the code version of Llama 2) locally on your laptop. We'll be doing that using the excellent llm CLI tool by Simon Willison.

I'm doing this on a MacBook Air from 2020 with an M1, but I think this will work elsewhere as well.

First, install llm and the llama plugins (you'll need a C compiler installed):

# Install llm itself. You could also do pip install llm
brew install llm
# Install the llama.cpp plugin and the Python bindings
llm install llm-llama-cpp
llm install llama-cpp-python

Then, download the model itself, a ~5 GB file:

llm llama-cpp download-model 'https://huggingface.co/TheBloke/CodeLlama-7B-Instruct-GGUF/resolve/main/codellama-7b-instruct.Q5_K_M.gguf' -a llama2-code-7b --llama2-chat

This will download it to a local directory that the llm tool manages and create an alias called llama2-code-7b.

You should now be able to generate Go code locally. Let’s be irresponsible adventurous and run it immediately:

$ llm -m llama2-code-7b -s 'generate Go code only' 'a program to print Hello World' 2&>/dev/null >main.go && go run main.go
Hello, World!

😎