Conversation Templates Reference

Introduction

As already mentioned several times in the project documentation, conversation templates play a central role in building a successful AI-based assistant. They can be viewed as programs or scripts that guide the assistant and the language model. Well-crafted templates ensure that your engagements with the assistant progress as planned and that you receive the desired help.

Throughout this chapter, the term ‘bot’ refers to the engine that enables interactions between the user and the large language model. This engine processes user input, executes conversation templates in the context of this input, sends requests to the model, receives responses, and, following instructions from the model, invokes functions on behalf of the user. The bot, combined with conversation templates, forms your AI assistant. AIChatFlow provides options to utilize the bot either through the Golang library or the ai command-line program, facilitating rapid prototyping of assistants.

The following sections delve into the operations of the bot — and by extension, your assistant. They provide detailed descriptions of how conversation templates are executed. If you’re new to the project, consider starting with our quick start guide, which offers a multitude of conversation template examples.

Template Structure and Message Roles

Conversation templates are composed of a sequence of messages, each with an assigned role. The available roles include system, assistant, user, function, default-request, request, and truncate.

system, assistant, user, and function roles

The first four roles — system, assistant, user, and function — directly correspond to those defined by the OpenAI API and follow the structure outlined in their API reference documentation. Messages for these roles in conversation templates are encoded as follows:

- role: assistant
  name: name
  content: text content
  function_call:
    name: function_name
    arguments: JSON struct ancoded as string

Note that the name attribute is required only for function messages, while the function_call attribute is permitted exclusively for assistant messages. Additionally, the content of a user message can be a single string, as illustrated above, or an array of content parts, such as:

- role: user
  content:
    - type: text
      text: text content
    - type: image_url
      image_url:
        url: https://example.com/image.jpg
        detail: high

When the conversation template is executed, the text content of each message is processed as a text template according to the rules in the text/template package from the Go standard library. This offers a powerful tool to dynamically generate prompts. Furthermore, in user messages, the content can be omitted; the content parts are then sourced from data supplied at the time of template execution. Subsequent sections detail the execution of templates and the handling of template data.

defualt-request, request, and truncate roles

Messages with the roles default-request, request, and truncate are not sent directly to the language model. Instead, they inform the assistant about what parameters and messages to include in the requests sent to the model.

The formats of default-request and request messages are identical, with the only distinction being that a default-request message must provide at least one parameter, while a request message may carry no attributes.

- role: default-request
  model: ...
  max_tokens: ...
  temperature: ...
  top_p: ...
  stop: ...
  presence_penalty: ...
  frequency_penalty: ...
  logit_bias: ...
  functions: ...
  call_function: ...
- role: request

The meanings and formats of these parameters are defined in OpenAI’s API reference documentation, with two exceptions. First, the functions parameter in OpenAI’s API is a list of function definitions, whereas in AIChatFlow, it is a list of function names, with their definitions located in a separate configuration file. This approach allows users to conveniently reference functions without the necessity of repeating their full definitions in each default-request or request message.

Second, the call_function parameter can be set to an empty string (“”), an asterisk (”*”), or a string specifying the function name. This parameter instructs the model to either avoid calling any functions, to potentially call one of the functions listed in the functions parameter, or to invoke a specific function, respectively.

The truncate message is the simplest of the three, carrying no additional data beyond its role designation. Consequently, it can be represented in the conversation template as follows:

- role: truncate

The details of how these three messages are interpreted are provided in the section below.

Template Execution

  1. Users initiate template execution and interaction with the large language model by providing these data points:

    1. The history of the conversation, comprising a list of messages exchanged between the user and the model. This includes default-request, request, and truncate messages.

    2. The user input, often referred to as the prompt.

    3. A set of conversation templates, with one selected for execution.

  2. Messages from the selected template are sequentially processed in the order of appearance, up to and including the request message, or, in the absence of a request message, until the end of the template. The content property of each message is processed according to the rules outlined in subsequent section. Read messages are appended to the conversation.

  3. The bot traverses the message list in reverse order until it encounters a default-request message, which supplies the parameters for the request to the language model. In the absence of a default-request message, the bot will use parameters from the system configuration if they are available.

  4. The request parameters found in previous step are supplemented and possibly overwritten by with those from the request message read in step 2 above.

  5. The constructed request is appended with user, assistant, function, and system messages from the conversation, up to the model’s context window limit.

  6. The request is sent to the model, which responds with messages that are appended to the conversation.

  7. Should the model prompt a function call and the bot recognize the
    function, it executes the function. The result is then included in the
    conversation as a function message.

    1. If the function called is a special botSwitchTemplate function, the
      current template execution is suspended, an alternative template is
      selected, and, along with the updated conversation, it is processed
      starting from step 2.

    2. Otherwise, the subsequent request is submitted to the model with the
      same parameters as before, this time however, without forcing a function
      call. Further processing continues from step 6.

  8. If the model response do not prompt a function call, and unread messages remain in the template, these messages are processed following the
    established rules. If there are no unread messages, the conversation is
    updated and presented to the user.

  9. If an error occurs at any stage of processing, any messages added to the
    conversation since the beginning of processing are discarded, and the error is reported back to the user.

Message Content Processing

When messages tagged with the roles user, assistant, system, and function are retrieved from the conversation template, as mentioned in step 2 of the previously described procedure, they undergo processing according to the following rules:

  1. If the content property is present in the message, then each text part of the content is treated as a template according to the rules defined in Golang’s text/template package. These templates are executed in the context of the user-provided input. Note that non-text parts are not processed.

  2. If the user message lacks the content property, the template engine sources content parts directly from user input. The input must be serializable as a JSON object and is expected to contain the contentParts property, which should be an array of message parts as defined in OpenAI’s API reference and as described above.

User Input (Template Data)

When the bot is utilized through the AIChatFlow library, developers can supply any data as context for the conversation templates. The application of this data depends solely on the conversation templates themselves. As noted earlier, the bot serves only as a template processing engine; it is the collection of templates and the program that executes them that collectively shape the assistant. It’s worth noting here the second rule from the above section about processing message content. Including a contentParts property in the template context can simplify template creation.

On the other hand, when the ai program executes conversation templates, it uses a well-defined data structure for the template context. This structure encompasses data passed through the program’s command line interface, the composed prompt, and the contentParts property. Detailed information on this structure is available in the Prompt Composition section of the CLI Reference chapter within the project documentation.