Groq
createGroqLlmCall
This is a function that calls an open-source LLM hosted by Groq. You can configure almost any parameter that can be configured in the Create chat completion Groq endpoint.
Arguments
Identifier which will be passed into the callback.
An object that contains all configuration parameters for the LLM call. See below for details on the GroqRequest object.
Returns
An internal counter of the oracle, which is incremented on every call.
GroqRequest object
The GroqRequest object (see source in ChatOracle.sol) has the following fields:
The model to be used, options are llama-3.1-70b-versatile, llama-3.1-8b-instant, llama3-8b-8192, llama3-70b-8192, mixtral-8x7b-32768, or gemma-7b-it. See Groq docs for more info on the models.
Penalty for frequency, int -20 to 20, mapped to float -2.0 to 2.0. Values greater than 20 result in null.
JSON string specifying logit bias, or an empty string for none.
Maximum number of tokens to generate, with 0 indicating null.
Penalty for presence, int -20 to 20, mapped to float -2.0 to 2.0. Values greater than 20 result in null.
JSON string indicating the format of the response, or an empty string for default format.
Seed for random number generation, with 0 indicating null.
Stop sequence(s) for generation, or an empty string to indicate none.
Temperature for generation, 0 to 20, with values greater than 20 indicating null.
Top p sampling parameter, 0 to 100 percentage, with values greater than 100 indicating null.
User identifier or context.
onOracleGroqLlmResponse
Called when the result of an LLM call, created with createGroqLlmCall, is available.
Arguments
The identifier you passed into the createGroqLlmCall method.
The result of the LLM call. See below for details on the GroqResponse object.
An error message. Contains an empty string if there was no error, and a description of the error otherwise.
GroqResponse object
The GroqResponse object (see source in ChatOracle.sol) has the following fields, most of which map directly to the fields in the Groq API response:
Unique identifier for the completion, generated by Groq.
The content associated with the completion.
Timestamp of creation.
The model name used for the completion.
System identifier that generated the completion.
Type of the object, always chat.completion.
Number of tokens in the completion.
Number of tokens in the prompt.
Total number of tokens, sum of prompt and completion tokens.