I have published a HuggingFace repo with llamafile and gguf file format models for Meltemi 7B Instruct v1.5, the first Large Language Model (LLM) for Greek. Meltemi was trained by Athena Research & Innovation Center. You can read about its technical details here

lamafile is a file format introduced by Mozilla Ocho on Nov 20th, 2023. It compacts a whole LLM into a single executable file which is a web server API, a CLI, and a chat web application. Everything is in a single cross-platform executable. It’s just beautiful.

Here’s how you can use it.

First start by downloading the file and give it executable permissions

wget https://huggingface.co/Florents-Tselai/Meltemi-llamafile/resolve/main/Meltemi-7B-Instruct-v1.5-Q8_0.llamafile
chmod +x Meltemi-7B-Instruct-v1.5-Q8_0.llamafile

Then run the executable

./Meltemi-7B-Instruct-v1.5-Q8_0.llamafile

This will open a tab with a chatbot and completion interface in your browser. For additional help on how it may be used, pass the --help flag.

API

The server also has an OpenAI API-compatible completions endpoint.

curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer no-key" \
-d '{
  "model": "LLaMA_CPP",
  "messages": [
      {
          "role": "system",
          "content": "Είσαι ένας φωτεινός παντογνώστης"
      },
      {
          "role": "user",
          "content": "Γράψε μου μια ιστορία για έναν βάτραχο που έγινε αρνάκι"
      }
    ]
}'

CLI

An advanced CLI mode is also provided that’s useful for shell scripting. You can use it by passing the --cli flag. For additional help on how it may be used, pass the –help flag.

./Meltemi-7B-Instruct-v1.5-Q8_0.llamafile -p 'Ποιό είναι το νόημα της ζωής;'

To see all available options

./Meltemi-7B-Instruct-v1.5-Q8_0.llamafile --help

gguf

gguf file formats are also available if you’re working with llama.cpp llama.cpp

llama.cpp offers quite a lot of options, thus refer to its documentation.

Basic Usage

llama-cli -m ./Meltemi-7B-Instruct-v1.5-F16.gguf -p "Ποιό είναι το νόημα της ζωής;" -n 128

Conversation Mode

llama-cli -m ./Meltemi-7B-Instruct-v1.5-F16.gguf --conv

Web Server

llama-server -m ./Meltemi-7B-Instruct-v1.5-F16.gguf --port 8080