Calling an LLM: simple

In this tutorial, we’ll build a simplified LLM contract using Solidity on Galadriel (Devnet).

We’ll use GPT-4-turbo. To use a different LLM like Claude-3.5-Sonnet, Mistral7B, or any other LLM available with teeML follow through with the tutorial and find code references in the end.

To build an LLM with chat history, a.k.a chat bot, visit Calling an LLM: advanced.

Prerequisites

To deploy the contract and interact with it, you will need:

A Galadriel account. For more information on setting up a wallet, visit Setting Up A Wallet.
Some tokens for gas fees. Get Galadriel Devnet tokens from the Faucet.
Go through Quickstart to set up the local dev environment and run the first example.

Contract

Full contract code is below. The same simplified LLM contracts are also available in the contracts folder. We will use GPT-4-turbo example here and go over each variable and function below.

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.13;

import "./interfaces/IOracle.sol";

contract OpenAiSimpleLLM {
    address private oracleAddress; // use latest: https://docs.galadriel.com/oracle-address
    IOracle.Message public message;
    string public response;
    IOracle.OpenAiRequest private config;

    constructor(address initialOracleAddress) {
        oracleAddress = initialOracleAddress;

        config = IOracle.OpenAiRequest({
            model : "gpt-4-turbo", // gpt-4-turbo gpt-4o
            frequencyPenalty : 21, // > 20 for null
            logitBias : "", // empty str for null
            maxTokens : 1000, // 0 for null
            presencePenalty : 21, // > 20 for null
            responseFormat : "{\"type\":\"text\"}",
            seed : 0, // null
            stop : "", // null
            temperature : 10, // Example temperature (scaled up, 10 means 1.0), > 20 means null
            topP : 101, // Percentage 0-100, > 100 means null
            tools : "",
            toolChoice : "", // "none" or "auto"
            user : "" // null
        });
    }

    function sendMessage(string memory _message) public {
        message = createTextMessage("user", _message);
        IOracle(oracleAddress).createOpenAiLlmCall(0, config);
    }

    // required for Oracle
    function onOracleOpenAiLlmResponse(
        uint /*runId*/,
        IOracle.OpenAiResponse memory _response,
        string memory _errorMessage
    ) public {
        require(msg.sender == oracleAddress, "Caller is not oracle");
        if (bytes(_errorMessage).length > 0) {
            response = _errorMessage;
        } else {
            response = _response.content;
        }
    }

    // required for Oracle
    function getMessageHistory(
        uint /*_runId*/
    ) public view returns (IOracle.Message[] memory) {
        IOracle.Message[] memory messages = new IOracle.Message[](1);
        messages[0] = message;
        return messages;
    }

    // @notice Creates a text message with the given role and content
    // @param role The role of the message
    // @param content The content of the message
    // @return The created message
    function createTextMessage(string memory role, string memory content) private pure returns (IOracle.Message memory) {
        IOracle.Message memory newMessage = IOracle.Message({
            role: role,
            content: new IOracle.Content[](1)
        });
        newMessage.content[0].contentType = "text";
        newMessage.content[0].value = content;
        return newMessage;
    }
}

Import

To get started, import all of the required interfaces. This gives the oracle the correct data structure.

import "./interfaces/IOracle.sol";

If you don’t have the IOracle.sollocally, you can also import it via GitHub like this:

import "https://github.com/galadriel-ai/contracts/blob/main/contracts/contracts/interfaces/IOracle.sol";

Variables

The address of the teeML oracle on Galadriel: .

This needs to be passed in when deploying the contract. For more information on how it works, click here.

address private oracleAddress;

The message variable will store the user input.

IOracle.Message public message; // example: tell me about cats

The oracle will store the response in the response variable.

string public response; // example: cats are...

The config variable will store the LLM configuration. The example uses configuration for OpenAI LLMs.

IOracle.OpenAiRequest private config;

Constructor

As mentioned, we’ll set the oracle address when deploying the contract, so this needs to be an input for the constructor.

We’ll also set the LLM config in the constructor. These will be parameters such as model, temperature, seed etc. For more information, visit OpenAIRequest object.

constructor(address initialOracleAddress) {
    oracleAddress = initialOracleAddress;

    config = IOracle.OpenAiRequest({
        model : "gpt-4-turbo", // gpt-4-turbo gpt-4o
        frequencyPenalty : 21, // > 20 for null
        logitBias : "", // empty str for null
        maxTokens : 1000, // 0 for null
        presencePenalty : 21, // > 20 for null
        responseFormat : "{\"type\":\"text\"}",
        seed : 0, // null
        stop : "", // null
        temperature : 10, // Example temperature (scaled up, 10 means 1.0), > 20 means null
        topP : 101, // Percentage 0-100, > 100 means null
        tools : "",
        toolChoice : "", // "none" or "auto"
        user : "" // null
    });
}

Send Message Function

A public function to initiate the LLM inference. In this example, our function sendMessage accepts a string _message.

Notice the message variable is not sent to the oracle with createOpenAiLlmCall, but rather stored on the same contract. This is done by the helper function createTextMessage.

The oracle will instead read getMessageHistory (next step) from this contract to retrieve the message history. This reading (instead of writing) will save on gas and storage.

createOpenAiLlmCall will call an oracle with the following parameters

0 an id to keep track of the message and it’s response (runId in later tutorials)
config set in the constructor

function sendMessage(string memory _message) public {
    message = createTextMessage("user", _message);
    IOracle(oracleAddress).createOpenAiLlmCall(0, config);
}

function createTextMessage(string memory role, string memory content) private pure returns (IOracle.Message memory) {
    IOracle.Message memory newMessage = IOracle.Message({
        role: role,
        content: new IOracle.Content[](1)
    });
    newMessage.content[0].contentType = "text";
    newMessage.content[0].value = content;
    return newMessage;
}

Read Message Function

The getMessageHistory function is used by the orcale to retrieve the message history, or in this case the only user sent message.

In order for the oracle to properly read the message history, the function

Must be called getMessageHistory
Must be a public view function for the oracle to call
Must return type IOracle.Message[]
Include unint parameter to keep track of the user message to respond to (we hardcoded this to 0)

function getMessageHistory(
    uint /*_runId*/
) public view returns (IOracle.Message[] memory) {
    IOracle.Message[] memory messages = new IOracle.Message[](1);
    messages[0] = message;
    return messages;
}

In this example, the contract is returning a list of messages of length 1. To have a longer message history (for a chat bots), loop through and return a list of messages. For this go to Calling an LLM: advanced.

LLM Response Callback Function

Once the oracle reads the message, it will forward it to teeML to get a response from the LLM.

After LLM returns the response to teeML, it gets pushed on-chain back to oracle, and from there the oracle calls onOracleOpenAiLlmResponse callback function in your contract.

The funciont must be called onOracleOpenAiLlmResponse
Include unint parameter to keep track of the user message to respond to (we hardcoded this to 0)
Includes parameter IOracle.OpenAiResponse with the correct response
Includes parameter errorMessage if an error occurs

function onOracleOpenAiLlmResponse(
    uint /*runId*/,
    IOracle.OpenAiResponse memory _response,
    string memory _errorMessage
) public {
    require(msg.sender == oracleAddress, "Caller is not oracle");
    if (bytes(_errorMessage).length > 0) {
        response = _errorMessage;
    } else {
        response = _response.content;
    }
}

To ensure that only the oracle contract can send back the response.

require(**msg**.sender == oracleAddress, "Caller is not oracle");

The contract then sets response to the response from the oracle (or error message if failed).

Response

After the oracle responds, anyone can access the value by reading the response variable.

string response; // tell me about cats

Cats, scientifically known as Felis catus, are…

What’s Next?

Congratulations on building your first LLM smart contract.

Explore further:

Instructions on how to deploy the contract on Galadrial Devnet using Hardhat.
Example script to call out deployed contract.
Use other LLMs:
- Anthropic Claude-3.5-Sonnet code example
- Groq Mistral7B code example
Implement a more advanced LLM chat bot: on-chain ChatGPT..
Dive deeper into the Galadriel documentation, particularly the How It Works section, to understand the underlying technology.
Learn more about other Use Cases to get inspired for your next project.

Happy building!

Learn

For developers

For users

Prerequisites

Contract

Import

Variables

Constructor

Send Message Function

Read Message Function

LLM Response Callback Function

Response

What’s Next?

Learn

For developers

For users

​Prerequisites

​Contract

​Import

​Variables

​Constructor

​Send Message Function

​Read Message Function

​LLM Response Callback Function

​Response

​What’s Next?

Prerequisites

Contract

Import

Variables

Constructor

Send Message Function

Read Message Function

LLM Response Callback Function

Response

What’s Next?