The plugin now includes the ability to create your own services based on the OpenAI function
concept.
These functions extend the capabilities of ChatGPT
and allow it to know about things it didn't before.
For example, the weather, current time, latest news, cost of airline tickets, information about your own system, program, game.
From the OpenAI
API perspective, functions are just a few additional fields in the JSON
request to OpenAI endpoints.
A Service
in the plugin is a separate UObject
base class that you can derived from, program at your discretion and add to ChatGPT
within the plugin.
I've added two services to the core of the plugin - weather
and news
. You can read about how to run them in the main readme.
Before we start, I strongly recommend familiarizing yourself with the official documentation and examples on functions and understanding the concept.
We will be creating everything in C++
. In theory, everything could be done using Blueprints
and you can call functions by name,
but defining JSON
in Blueprints
it's a pain.
We will create a service that will provide the chat with information about characters and their abilities in our fixtion game Alien Rampage Saga
.
We'll have two characters:
Voidwalker
- the heroGalaxor
- the villain.
Voidwalker's
abilities:
- Adept at crafting nasty jokes, has a penchant for donuts, and is mesmerized by the Aurora Borealis.
Galaxor's
abilities:
- Capable of unicorn riding, boasts the ability to sleep for 20 hours straight, and possesses knowledge of all credit card PIN codes.
It is assumed that you have set up the plugin. Complete instructions can also be found in the main readme.
The complete listing of the program will be at the end of this tutorial.
- Create a
UObject
class, name itQuestService
, and inherit it from the plugin's classUBaseService
:
#pragma once
#include "CoreMinimal.h"
#include "ChatGPT/BaseService.h"
#include "QuestService.generated.h"
UCLASS()
class YOUR_PROJECT_API UQuestService : public UBaseService
{
GENERATED_BODY()
};
- Copy the following virtual functions from the
UBaseService
class. They need to be implemented:
#pragma once
#include "CoreMinimal.h"
#include "ChatGPT/BaseService.h"
#include "QuestService.generated.h"
UCLASS()
class YOUR_PROJECT_API UQuestService : public UBaseService
{
GENERATED_BODY()
public:
virtual bool Init(const OpenAI::ServiceSecrets& Secrets);
virtual FString Description() const;
virtual FString FunctionName() const;
virtual void Call(const TSharedPtr<FJsonObject>& Args, const FString& ToolID);
virtual FString Name() const;
virtual FString TooltipDescription() const;
protected:
virtual FString MakeFunction() const;
};
- We will need
JSON
utilities, so include theJSON
module in the build file, and don't forget that theOpenAI
plugin also needs to be linked:
using UnrealBuildTool;
public class YourProjectName : ModuleRules
{
public YourProjectName(ReadOnlyTargetRules Target) : base(Target)
{
PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;
PublicDependencyModuleNames.AddRange(
new string[] { "Core", "CoreUObject", "Engine", "InputCore", "OpenAI", "Json", "JsonUtilities"});
PublicIncludePaths.AddRange(new string[] { "YourProjectName" });
}
}
- Immediately implement the
Name
andTooltipDescription
functions. These are strictly for UI information:
#pragma once
#include "CoreMinimal.h"
#include "ChatGPT/BaseService.h"
#include "QuestService.generated.h"
UCLASS()
class YOUR_PROJECT_API UQuestService : public UBaseService
{
GENERATED_BODY()
public:
virtual bool Init(const OpenAI::ServiceSecrets& Secrets);
virtual FString Description() const;
virtual FString FunctionName() const;
virtual void Call(const TSharedPtr<FJsonObject>& Args, const FString& ToolID);
virtual FString Name() const override { return "Quest"; }
virtual FString TooltipDescription() const override { return "Alien Rampage Saga"; }
protected:
virtual FString MakeFunction() const;
};
- Create the function definitions in the
cpp
file, include the necessary headers, and create a logging category:
#include "QuestService.h"
#include "Provider/CommonTypes.h"
#include "FuncLib/OpenAIFuncLib.h"
#include "Provider/RequestTypes.h"
DEFINE_LOG_CATEGORY_STATIC(LogQuestService, All, All);
bool UQuestService::Init(const OpenAI::ServiceSecrets& Secrets) { return true; }
FString UQuestService::FunctionName() const { return {}; }
FString UQuestService::Description() const { return {}; }
FString UQuestService::MakeFunction() const { return {}; }
void UQuestService::Call(const TSharedPtr<FJsonObject>& ArgsJson, const FString& ToolID) {}
- Create a name for our function that
OpenAI
will call. The init function will always returntrue
in our example. It's not significant in our service. It's assumed that some kind of service validation is taking place here such as an checkingAPI
key or whatever:
#include "QuestService.h"
#include "Provider/CommonTypes.h"
#include "Provider/RequestTypes.h"
#include "FuncLib/OpenAIFuncLib.h"
DEFINE_LOG_CATEGORY_STATIC(LogQuestService, All, All);
bool UQuestService::Init(const OpenAI::ServiceSecrets& Secrets)
{
return true;
}
FString UQuestService::FunctionName() const
{
return "get_alien_rampage_saga_characters_information";
}
- The
Description
function should be as informative as possible so thatChatGPT
understands the context of your prompts:
FString UQuestService::Description() const
{
return "Get information about characters from my game Alien Rampage Saga.";
}
- The
MakeFunction
method defines the parameters for our functionget_alien_rampage_saga_characters_information
that the GPT chat can pass. The function returnsJSON
as a string. Let's create such aJSON
object with parameters:
"parameters": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Name of the character of my Alien Rampage Saga.",
},
"ability": {
"type": "boolean",
"description": "Set this to true if you want to know what skills the character has.",
},
},
"required": ["name"],
}
-
properties - a list of parameters that
ChatGPT
can pass -
name - name of our character about whom
ChatGPT
will want to know -
ability - this is a flag that
ChatGPT
will send when it wants to know about the character's abilities. -
required - array of required parameters, we always only need the name of the character.
Implementation of the MakeFunction
. Basically, it's just a JSON
creation that I listed above:
FString UQuestService::MakeFunction() const
{
TSharedPtr<FJsonObject> MainObj = MakeShareable(new FJsonObject());
MainObj->SetStringField("type", "object");
TSharedPtr<FJsonObject> Props = MakeShareable(new FJsonObject());
// character name
TSharedPtr<FJsonObject> NameObj = MakeShareable(new FJsonObject());
NameObj->SetStringField("type", "string");
NameObj->SetStringField("description", "Name of the character of my Alien Rampage Saga.");
Props->SetObjectField("name", NameObj);
// ability
TSharedPtr<FJsonObject> AbilityObj = MakeShareable(new FJsonObject());
AbilityObj->SetStringField("type", "boolean");
AbilityObj->SetStringField("description", "Set this to true if you want to know what skills the character has.");
Props->SetObjectField("ability", AbilityObj);
MainObj->SetObjectField("properties", Props);
// required params
TArray<TSharedPtr<FJsonValue>> RequiredArray;
RequiredArray.Add(MakeShareable(new FJsonValueString("name")));
MainObj->SetArrayField("required", RequiredArray);
return UOpenAIFuncLib::MakeFunctionsString(MainObj);
}
Here we have a magic function from the plugin UOpenAIFuncLib::MakeFunctionsString(MainObj)
that is necessary for proper parsing of the JSON Schema reference.
- We are moving on to the final function, which is called when
ChatGPT
requests information about our characters: Add call to the parent function withSuper
alias. Basically the parent function will store theToolID
.
void UQuestService::Call(const TSharedPtr<FJsonObject>& ArgsJson, const FString& ToolIDIn)
{
Super::Call(ArgsJson, ToolIDIn);
}
The function takes a JSON
object with parameters. Let's output them to the log;
we can use a convenient function UOpenAIFuncLib::JsonToString
from the plugin library.
We will also retrieve our parameters that we defined in the JSON schema
.
The only mandatory parameter is the name
our character; ability
may not be present:
void UQuestService::Call(const TSharedPtr<FJsonObject>& ArgsJson, const FString& ToolIDIn)
{
Super::Call(ArgsJson, ToolIDIn);
FString ArgsStr;
if (UOpenAIFuncLib::JsonToString(ArgsJson, ArgsStr))
{
UE_LOG(LogQuestService, Display, TEXT("Args for the quest request: %s"), *ArgsStr);
}
FString CharacterName;
if (!ArgsJson->TryGetStringField("name", CharacterName))
{
ServiceDataError.Broadcast("Please provide character name");
return;
}
bool AbilityRequested{false};
ArgsJson->TryGetBoolField("ability", AbilityRequested);
}
The ServiceDataError
is a delegate from the base class UBaseService
which notifies the plugin of an error.
This could be used within the service implementation to report back any issues in processing the request,
such as missing parameters, invalid data, or internal service errors.
Next, we define the data structure with information about our characters. I am doing this locally; you can add it as a member of the class:
void UQuestService::Call(const TSharedPtr<FJsonObject>& ArgsJson, const FString& ToolIDIn)
{
...
struct FCharacterInfo
{
FString Description;
FString Abilities;
};
const TMap<FString, FCharacterInfo> Info //
{{"Voidwalker", {"The main character fighting for the future of the Universe",
"Adept at crafting nasty jokes, has a penchant for donuts, and is mesmerized by the Aurora Borealis"}},
{"Galaxor", {"A formidable antagonist intent on the Universe's destruction",
"Capable of unicorn riding, boasts the ability to sleep for 20 hours straight, and possesses knowledge of all "
"credit card PIN codes"}}};
}
Great. Now we are forming a string to send to the ChatGPT
:
void UQuestService::Call(const TSharedPtr<FJsonObject>& ArgsJson, const FString& ToolIDIn)
{
...
FString InfoToOpenaAI;
if (Info.Contains(CharacterName))
{
InfoToOpenaAI = AbilityRequested ? Info[CharacterName].Abilities : Info[CharacterName].Description;
}
}
Now we're assembling the full structure and sending it to the core of the plugin:
You can do this with a function that exists in the UBaseService
class.
void UQuestService::Call(const TSharedPtr<FJsonObject>& ArgsJson, const FString& ToolIDIn)
{
...
const FMessage Message = MakeMessage(InfoToOpenAI);
ServiceDataRecieved.Broadcast(Message);
}
The main field of the structure FMessage
is Content
, into which any information that
will be analyzed by the ChatGPT
and it will write a response based on it.
The ServiceDataRecieved
is a delegate from the base class UBaseService
which notifies the plugin that we can send data to the ChatGPT
.
The full listing of the UQuestService::Call
is following:
void UQuestService::Call(const TSharedPtr<FJsonObject>& ArgsJson, const FString& ToolIDIn)
{
Super::Call(Args, ToolIDIn);
FString ArgsStr;
if (UOpenAIFuncLib::JsonToString(ArgsJson, ArgsStr))
{
UE_LOG(LogQuestService, Display, TEXT("Args for the quest request: %s"), *ArgsStr);
}
FString CharacterName;
if (!ArgsJson->TryGetStringField("name", CharacterName))
{
ServiceDataError.Broadcast("Please provide character name");
return;
}
bool AbilityRequested{false};
ArgsJson->TryGetBoolField("ability", AbilityRequested);
struct FCharacterInfo
{
FString Description;
FString Abilities;
};
const TMap<FString, FCharacterInfo> Info //
{{"Voidwalker", {"The main character fighting for the future of the Universe",
"Adept at crafting nasty jokes, has a penchant for donuts, and is mesmerized by the Aurora Borealis"}},
{"Galaxor", {"A formidable antagonist intent on the Universe's destruction",
"Capable of unicorn riding, boasts the ability to sleep for 20 hours straight, and possesses knowledge of all "
"credit card PIN codes"}}};
FString InfoToOpenaAI;
if (Info.Contains(CharacterName))
{
InfoToOpenaAI = AbilityRequested ? Info[CharacterName].Abilities : Info[CharacterName].Description;
}
const FMessage Message = MakeMessage(InfoToOpenAI);
ServiceDataRecieved.Broadcast(Message);
}
- Compile your code and open the
Unreal Engine
Editor.
Now the fun part starts.
- In the
Content Browser
, find theChatGPT
widget within the plugin's content and open it.
- Go to the
Graph
view of the widget.
- In the
Details
panel add our newQuestService
to the services' array.
- Right-click the widget to run it as an editor utility widget
- Activate our new service. You should see the service name displayed, with the description showing as a tooltip.
- Ask ChatGPT if it knows anything about your game to test the functionality.
Can you retrieve information about the characters from my game "Alien Rampage Saga."?
COOOOL! Works!
Who is Voidwalker?
Who is Galaxor?
Asking about abilities:
Note that ChatGPT
processes information and writes quite naturally.
That's about it. Functions open a lot of additional possibilities to extend ChatGPT
.
Full lisitng of our QuestService
:
#pragma once
#include "CoreMinimal.h"
#include "ChatGPT/BaseService.h"
#include "QuestService.generated.h"
UCLASS()
class AIMUSEUM_API UQuestService : public UBaseService
{
GENERATED_BODY()
public:
virtual bool Init(const OpenAI::ServiceSecrets& Secrets) override;
virtual FString Name() const override { return "Quest"; }
virtual FString TooltipDescription() const override { return "Alien Rampage Saga"; }
virtual FString Description() const override;
virtual FString FunctionName() const override;
virtual void Call(const TSharedPtr<FJsonObject>& ArgsJson, const FString& ToolID) override;
protected:
virtual FString MakeFunction() const;
};
#include "QuestService.h"
#include "Provider/CommonTypes.h"
#include "FuncLib/OpenAIFuncLib.h"
#include "Provider/RequestTypes.h"
DEFINE_LOG_CATEGORY_STATIC(LogQuestService, All, All);
bool UQuestService::Init(const OpenAI::ServiceSecrets& Secrets)
{
return true;
}
FString UQuestService::FunctionName() const
{
return "get_alien_rampage_saga_characters_information";
}
FString UQuestService::Description() const
{
return "Get information about characters from my game Alien Rampage Saga.";
}
FString UQuestService::MakeFunction() const
{
/*
"parameters": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Name of the character of my Alien Rampage Saga game.",
},
"ability": {
"type": "boolean",
"description": "Set this to true if you want to know what skills the character has.",
},
},
"required": ["name"],
}
*/
TSharedPtr<FJsonObject> MainObj = MakeShareable(new FJsonObject());
MainObj->SetStringField("type", "object");
TSharedPtr<FJsonObject> Props = MakeShareable(new FJsonObject());
// character name
TSharedPtr<FJsonObject> NameObj = MakeShareable(new FJsonObject());
NameObj->SetStringField("type", "string");
NameObj->SetStringField("description", "Name of the character of my Alien Rampage Saga game.");
Props->SetObjectField("name", NameObj);
// ability
TSharedPtr<FJsonObject> AbilityObj = MakeShareable(new FJsonObject());
AbilityObj->SetStringField("type", "boolean");
AbilityObj->SetStringField("description", "Set this to true if you want to know what skills the character has.");
Props->SetObjectField("ability", AbilityObj);
MainObj->SetObjectField("properties", Props);
// required params
TArray<TSharedPtr<FJsonValue>> RequiredArray;
RequiredArray.Add(MakeShareable(new FJsonValueString("name")));
MainObj->SetArrayField("required", RequiredArray);
return UOpenAIFuncLib::MakeFunctionsString(MainObj);
}
void UQuestService::Call(const TSharedPtr<FJsonObject>& ArgsJson, const FString& ToolIDIn)
{
Super::Call(Args, ToolIDIn);
FString ArgsStr;
if (UOpenAIFuncLib::JsonToString(ArgsJson, ArgsStr))
{
UE_LOG(LogQuestService, Display, TEXT("Args for the quest request: %s"), *ArgsStr);
}
FString CharacterName;
if (!ArgsJson->TryGetStringField("name", CharacterName))
{
ServiceDataError.Broadcast("Please provide character name");
return;
}
bool AbilityRequested{false};
ArgsJson->TryGetBoolField("ability", AbilityRequested);
struct FCharacterInfo
{
FString Description;
FString Abilities;
};
const TMap<FString, FCharacterInfo> Info //
{{"Voidwalker", {"The main character fighting for the future of the Universe",
"Adept at crafting nasty jokes, has a penchant for donuts, and is mesmerized by the Aurora Borealis"}},
{"Galaxor", {"A formidable antagonist intent on the Universe's destruction",
"Capable of unicorn riding, boasts the ability to sleep for 20 hours straight, and possesses knowledge of all "
"credit card PIN codes"}}};
FString InfoToOpenaAI;
if (Info.Contains(CharacterName))
{
InfoToOpenaAI = AbilityRequested ? Info[CharacterName].Abilities : Info[CharacterName].Description;
}
else
{
InfoToOpenaAI = "Character with such a name doesn't exist in game";
}
const FMessage Message = MakeMessage(InfoToOpenAI);
ServiceDataRecieved.Broadcast(Message);
}