Accessing the OpenAI API using C++

OpenAI’s Generative Pretrained Transformer (GPT) has been a game-changer in the field of artificial intelligence and natural language processing. It’s capable of generating human-like text based on the input it’s given, making it a powerful tool for a variety of applications, from chatbots to content generation and more. In a previous article I presented Best Practices in Prompt Engineering, but how to utilize them in an application and interact with the OpenAI language models programmatically? There are many tutorials available on how to access the OpenAI API using Python and JavaScript, and this article shows how to access the API using C++. The article includes source code for a fully functional C++ application that uses the chat API.

Photo by ThisisEngineering RAEng on Unsplash

Prerequisites

The article assumes working knowledge of C++, and to compile and run the code you need to have:

1. An OpenAI API key. Here is an article on how to get an OpenAI API key.
2. The libcurl and nlohmann::json libraries installed in your development environment. The curl library (libcurl) is used for making HTTP requests, and nlohmann::json is a header-only C++ JSON library for handling JSON data. Here is how I installed them on Ubuntu:

sudo apt update
sudo apt install libcurl14
sudo apt install nlohmann-json-dev

The nlohmann JSON library is being used here for code readability, but if you want speed and efficiency you should use RapidJSON or JsonCPP. Here is a guide to JSON using C++.

OpenAI Completion vs. Conversation (Chat) API

The OpenAI API provides application programming interfaces for accessing AI models developed by OpenAI.

The Completion API and the Conversation (Chat) API are two different interfaces provided by OpenAI for interacting with their language models. They are designed for slightly different use cases.

Completion API: This API (such as the davinci engine: https://api.openai.com/v1/engines/davinci/completions) is designed for generating a continuation of a given text prompt. You provide a string of text, and the model generates what it predicts should come next. This is useful for tasks like text generation, where you want the model to generate a story, an essay, or another form of text based on a starting prompt. As an example, when I used completion API with the prompt: what is 3 +7. The model responded with repeated prompts and this can happen for several factors (see below). To fix it, I changed the prompt to: calculate the sum of 3 and 7. The repeated prompt in the generated response may occur due to several factors:

1. Insufficient context: The model may require more context or explicit instructions to generate a meaningful response. In this case, it seems that the provided prompt is not giving enough guidance to the model to generate a response beyond repetition.
2. Model limitations: The model may have limitations in understanding or accurately processing specific prompts or tasks. It may not have been trained extensively on evaluating mathematical expressions or providing direct numerical results.
3. Training data bias: The model’s training data may contain examples where repetition or redundant outputs were common. As a result, the model might have learned to repeat the prompt when it is uncertain or lacks specific guidance.

To overcome these issues, you can try the following approaches:

1. Provide clearer instructions: Make the prompt more explicit and specific. Instead of just asking “What is 3 + 7?”, you can provide more context, such as “Please compute the sum of 3 and 7.” This helps the model understand the task better.
2. Modify the parameters: Adjust the temperature value to control the randomness of the output. Higher values like 0.8 can introduce more diversity, while lower values like 0.2 make the output more focused and deterministic. Experiment with different temperature values to see which provides the desired results.
3. Consider different models: If the current model continues to exhibit repetitive behavior you can explore alternative models or architectures that might better suit your specific task. Each model has its own strengths and weaknesses, so trying different models can potentially yield better results.

It’s important to remember that language models like the one used here are probabilistic and generate responses based on patterns learned from vast amounts of training data. While they can often produce impressive results, they may not always deliver the desired output without careful instruction and experimentation.

Conversation (Chat) API: This API is designed to handle a series of messages and generate a model-generated message as a reply. You can send an array of message objects and each object should have a ‘role’ that can be ‘system’, ‘user’, or ‘assistant’, and ‘content’ which is the text of the message from the role. A ‘system’ message is used to set the behavior of the assistant, a ‘user’ message (used in the code provided) is what the user says or instructs, and the ‘assistant’ message is what the assistant responds with. The response from the API will contain the assistant’s reply which can be extracted and used as needed. The model generates a response based on the entire conversation history. This is useful for tasks like building a chatbot, where you want the model to generate responses in the context of a back-and-forth conversation. The API address (https://api.openai.com/v1/chat/completions) points to the “chat/completions” endpoint.

C++ Application

This application, which uses the conversation (chat) API, prompts the user to enter a prompt, it interacts with the GPT-3.5-turbo model to generate a response. The two core functions of the application are:

• WriteCallback(): This is a callback function used by libcurl to handle the response data that is received from the server. The data is appended to the response string. The function returns the size of the data handled.
• getCompletion(): This function makes a POST request to the OpenAI API. It takes a prompt and a model name as input, constructs a JSON object with these and other necessary fields, and sends this to the API. The API key and base URL are hardcoded in the function. The function uses libcurl to make the HTTP request. If the request fails, an error message is printed. The function returns the response data as a string.

/*
 * This program demonstrates how to programatically interact with OpenAI GPT using C++.
 * The program uses the curl library for making HTTP requests, and the nlohmann json
 * library for handling JSON data.
 */
#include <iostream>
#include <string>
#include <curl/curl.h>
#include <nlohmann/json.hpp>

using std::string;
using std::cout;
using std::cin;
using std::cerr;
using std::endl;
using nlohmann::json;

// Function prototypes
size_t WriteCallback(void*, size_t, size_t, string*);
string getCompletion(const string&, const string& model = "gpt-3.5-turbo");

// Main entry into the application
int main() {
    cout << "Enter a prompt: ";
    string prompt;
    getline(cin, prompt);
    string response = getCompletion(prompt);
    cout << response << endl;

    return 0;
}

// Handle data received from the server
size_t WriteCallback(void* contents, size_t size, size_t nmemb, string* response) {
    size_t totalSize = size * nmemb;
    response->append((char*)contents, totalSize);
    return totalSize;
}

// Construct a POST request to the chat model endpoint and process the response.
string getCompletion(const string& prompt, const string& model = "gpt-3.5-turbo") {
    string apiKey = ""; // add your API key, the app will not work without it
    string baseUrl = "https://api.openai.com/v1/chat/completions";
    string response;
    CURL* curl = curl_easy_init();

    if (curl) {
        json requestData;
        requestData["model"] = model;
        requestData["messages"][0]["role"] = "user";
        requestData["messages"][0]["content"] = prompt;
        requestData["temperature"] = 0;

        string requestDataStr = requestData.dump().c_str();

        struct curl_slist* headers = NULL;
        headers = curl_slist_append(headers, "Content-Type: application/json");
        headers = curl_slist_append(headers, ("Authorization: Bearer " + apiKey).c_str());
        curl_easy_setopt(curl, CURLOPT_URL, baseUrl.c_str());
        curl_easy_setopt(curl, CURLOPT_POSTFIELDS, requestDataStr.c_str());
        curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, requestDataStr.length());
        curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
        curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
        CURLcode res = curl_easy_perform(curl);

        if (res != CURLE_OK) {
            cerr << "Curl request failed: " << curl_easy_strerror(res) << endl;
        }

        curl_easy_cleanup(curl);
        curl_slist_free_all(headers);
    }

    // return response;
    json jresponse = json::parse(response);
    return jresponse["choices"][0]["message"]["content"].get<string>();
}

Security Note: The API key is hardcoded in the getCompletion() function — you need to enter your OpenAI API key to test this application. This is generally not a good practice as it exposes the key in the source code. It’s better to store sensitive information like API keys in environment variables or secure files. This is left to you as an exercise.

Test the Application

Save the code in a file named ChatApp.cpp, and then compile and run the application on a Linux distribution such as Ubuntu use the following commands:

// Compile the application
g++ -Wall ChatApp.cpp -o ChatApp -lcurl

// Run the application
./ChatApp

Conclusion

Developing C++ applications that interact with the OpenAI API is simple process, thanks to the power of libcurl and nlohmann::json. With this guide, you should be able to start integrating OpenAI language models into your own C++ applications. In this article we used GPT-3.5, but note that GPT-4 is OpenAI’s most capable model and as of July 6, 2023 the OpenAI GPT-4 API is available. Happy coding!