Please note that the code provided in this page is purely for learning purposes and is far from perfect. Remember to null-check all responses!
Basics
Setup
Add an instance of Uralstech.UGemini.GeminiManager to your scene, and set it up with your Gemini API key. You can get your API key from here.
GeminiManager
There are only three methods in GeminiManager:
| Method | What it does |
SetApiKey | Sets the Gemini API key through code |
Request | Computes a request on the Gemini API |
StreamRequest* | Computes a streaming request on the Gemini API |
*Requires Utilities.Async.
All computations on the Gemini API are done through GeminiManager.Request, GeminiManager.StreamRequest and their variants.
In this page, the fields, properties and methods of each type will not be explained. Every type has been fully documented in code, so please check the code docstrings or reference documentation to learn more about each type.
Beta API
GeminiManager supports both the v1 and v1beta Gemini API versions. As a lot of features are still unsupported in the main v1 API, you may need to use the Beta API. You can set the useBetaApi boolean parameter in the request's constructor to do so.
Models
Uralstech.UGemini.Models.GeminiModel has four static model IDs:
You can provide these to the model parameter in the constructors for model-related requests. UGemini can also implicitly convert string model IDs to GeminiModelId objects.
QuickStart: Multi-turn Chat Request
This is a simple script that maintains the user's chat history with Gemini.
private List<GeminiContent> _chatHistory = new();
private async Task<string> OnChat(string text)
{
{
Contents = _chatHistory.ToArray(),
}
);
return response.
Parts[0].Text;
}
The class for accessing the Gemini API!
Definition GeminiManager.cs:17
The base structured datatype containing multi-part content of a message.
Definition GeminiContent.cs:16
static GeminiContent GetContent(string message, GeminiRole role=GeminiRole.Unspecified)
Creates a new GeminiContent from a role and message.
Definition GeminiContent.cs:34
Information about a Generative Language Model.
Definition GeminiModel.cs:11
static readonly GeminiModelId Gemini1_5Flash
Gemini 1.5 Flash is a fast and versatile multimodal model for scaling across diverse tasks.
Definition GeminiModel.cs:56
Request to generate a response from the model.
Definition GeminiChatRequest.cs:20
Response from the model supporting multiple candidates.
Definition GeminiChatResponse.cs:22
GeminiContentPart[] Parts
The parts of the GeminiChatResponse message.
Definition GeminiChatResponse.cs:42
GeminiCandidate[] Candidates
Candidate responses from the model.
Definition GeminiChatResponse.cs:26
static T Instance
The active instance of type T .
Definition Singleton.cs:18
Definition GeminiAttributionSourceId.cs:5
GeminiRole
The role of a Gemini content creator.
Definition GeminiRole.cs:13
Definition GeminiChatRequest.cs:14
Definition GeminiCachedContentCreateRequest.cs:5
Definition GeminiFileDeleteRequest.cs:2
Here, we simply have a list of GeminiContent objects, which tracks the messages of the conversation. Every time OnChat is called, the user's request and the model's reply are added the the list.
All Supported Endpoints
CachedContents (Beta API)
Context caching allows you to save and reuse precomputed input tokens that you wish to use repeatedly, for example when asking different questions about the same media file.
Create
Creates CachedContent resource.
private async Task<GeminiCachedContent> RunCreateCachedContentRequest()
{
StringBuilder sb = new StringBuilder();
for (int i = 0; i < 1800; i++)
sb.Append("(*)#$*OIJIR$U(IJT^(U$*I%O$#@");
return await
GeminiManager.
Instance.Request<GeminiCachedContent>(
new GeminiCachedContentCreateRequest(
new GeminiCachedContentCreationData
{
Contents = new[]
{
},
ExpireTime = DateTime.UtcNow.AddDays(1),
Model = "gemini-1.5-flash-001",
}));
}
See GeminiCachedContent and GeminiCachedContentCreateRequest for more details.
Delete
Deletes CachedContent resource.
private async Task RunDeleteCachedContentRequest(GeminiCachedContent content)
{
Debug.Log("Deleting cached content...");
Debug.Log("Content deleted.");
}
See GeminiCachedContentDeleteRequest for more details.
Get
Reads CachedContent resource.
private async Task<GeminiCachedContent> RunGetCachedContentRequest(string contentName)
{
return await
GeminiManager.
Instance.Request<GeminiCachedContent>(
new GeminiCachedContentGetRequest(contentName));
}
See GeminiCachedContent and GeminiCachedContentGetRequest for more details.
List
Lists CachedContents.
private async Task<GeminiCachedContent[]> RunListCachedContentRequest()
{
GeminiCachedContentListResponse response = await
GeminiManager.
Instance.Request<GeminiCachedContentListResponse>(
new GeminiCachedContentListRequest());
return response.CachedContents;
}
See GeminiCachedContentListResponse and GeminiCachedContentListRequest for more details.
Patch
Updates CachedContent resource (only expiration is updatable).
private async Task<GeminiCachedContent> RunPatchCachedContentRequest(string contentName)
{
return await
GeminiManager.
Instance.Request<GeminiCachedContent>(
new GeminiCachedContentPatchRequest(
new GeminiCachedContentPatchData
{
ExpireTime = DateTime.UtcNow.AddYears(1),
}, contentName));
}
See GeminiCachedContent and GeminiCachedContentPatchRequest for more details.
Models
The Models endpoint contains methods that allow you to access and inference Gemini models.
Get
Gets information about a specific Model such as its version number, token limits, parameters and other metadata.
private async Task<GeminiModel> RunGetModelRequest(string modelId)
{
}
Gets information about a specific model. Return type is GeminiModel.
Definition GeminiModelGetRequest.cs:7
See GeminiModel and GeminiModelGetRequest for more details.
Newer models will not be recognized by the request if you're not using the Beta API.
List
Lists the Models available through the Gemini API.
private async Task<GeminiModel[]> RunListModelsRequest(int maxModels = 50, string pageToken = null)
{
{
MaxResponseModels = maxModels,
PageToken = string.IsNullOrWhiteSpace(pageToken) ? string.Empty : pageToken,
});
}
Requests metadata for all existing models. Return type is GeminiModelListResponse.
Definition GeminiModelListRequest.cs:7
The response for a GeminiModelListRequest call.
Definition GeminiModelListResponse.cs:11
GeminiModel[] Models
The list of models.
Definition GeminiModelListResponse.cs:16
See GeminiModelListResponse and GeminiModelListRequest for more details.
Newer models will not be recognized by the request if you're not using the Beta API.
EmbedContent
Generates a text embedding vector from the input Content using the specified Gemini Embedding model.
private async void RunEmbedContentRequest()
{
Debug.Log("Running embedding request.");
{
}
);
Debug.Log($"Embedding values: {string.Join(", ", response.Embedding.Values)}");
}
Generates an embedding from the model.
Definition GeminiEmbedContentRequest.cs:13
The response to a GeminiEmbedContentRequest.
Definition GeminiEmbedContentResponse.cs:11
static readonly GeminiModelId TextEmbedding004
text-embedding-004 achieves a stronger retrieval performance and outperforms existing models with com...
Definition GeminiModel.cs:66
Definition GeminiBatchEmbedContentRequest.cs:5
See GeminiEmbedContentResponse and GeminiEmbedContentRequest for more details.
BatchEmbedContents
Generates multiple embedding vectors from the input Content which consists of a batch of strings represented as EmbedContentRequest objects.
private async void RunBatchEmbedContentRequest()
{
Debug.Log("Running batch embedding request.");
{
{
{
},
{
Content =
GeminiContent.
GetContent(
"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua."),
}
}
}
);
Debug.Log($"Embedding values: {string.Join(", ", embedding.Values)}");
}
Generates multiple embeddings from the model given input text in a synchronous call.
Definition GeminiBatchEmbedContentRequest.cs:11
The response to a GeminiBatchEmbedContentRequest.
Definition GeminiBatchEmbedContentResponse.cs:11
GeminiContentEmbedding[] Embeddings
The embeddings for each request, in the same order as provided in the batch request.
Definition GeminiBatchEmbedContentResponse.cs:15
A list of floats representing an embedding.
Definition GeminiContentEmbedding.cs:11
See GeminiBatchEmbedContentResponse and GeminiBatchEmbedContentRequest for more details.
GenerateContent
Generates a model response given an input GenerateContentRequest.
private async void RunChatRequest()
{
Debug.Log("Running chat request.");
{
{
},
}
);
Debug.Log($"Gemini's response: {response.Parts[^1].Text}");
}
See GeminiChatResponse and GeminiChatRequest for more details.
StreamGenerateContent
Generates a streamed response from the model given an input GenerateContentRequest.
private async void RunStreamingChatRequest()
{
Debug.Log("Running streamed chat request.");
{
Contents = new GeminiContent[]
{
GeminiContent.GetContent("What's up? Tell me a story about airplanes.")
},
OnPartialResponseReceived = partialResponse =>
{
if (partialResponse.Candidates == null || partialResponse.Candidates.Length < 0)
return Task.CompletedTask;
Debug.Log($"Gemini's partial response: {partialResponse.Parts[^1].Text}");
return Task.CompletedTask;
}
}
);
Debug.Log($"Gemini's final response: {response.Parts[^1].Text}");
}
See GeminiChatResponse and GeminiChatRequest for more details.
GenerateAnswer (Beta API)
Generates a grounded answer from the model given an input GenerateAnswerRequest.
private async void RunQuestionAnsweringRequest()
{
Debug.Log("Running Q/A request.");
{
{
},
{
{
{
Id = "ezrSquaredContext",
"ezr² is an easy to learn and practical interpreted programming language for beginners and experts alike made in C#." +
"The latest version of ezr² RE has been released! ezr² RE, or REwrite, is the project's initiative to rewrite ezr². " +
"The latest working version of ezr² RE has many more features than the latest version of ezr²! But, it is still in " +
"development, has some essential features missing. Like the include expression, or any built-in object methods like " +
"\"a string\".length or [\"a\", \"list\"].insert. If you want to help in testing it out and fixing bugs, feel free to" +
" download the latest version of ezr² RE from the ezr² GitHub releases page and compiling it using the .NET SDK and/or" +
" Visual Studio. The GitHub releases page is: https://github.com/Uralstech/ezrSquared/releases."
)
}
}
},
}
);
if (response.
Answer.Content !=
null)
Debug.Log($"Gemini's answer: {response.Answer.Content.Parts[^1].Text}");
Debug.Log($"Attribution ID: {attribution.SourceId.GroundingPassage.PassageId} | Content: {attribution.Content.Parts[^1].Text}");
}
Attribution for a source that contributed to an answer.
Definition GeminiGroundingAttribution.cs:11
static readonly GeminiModelId Aqa
You can use the AQA model to perform Attributed Question-Answering (AQA)–related tasks over a documen...
Definition GeminiModel.cs:77
Generates a grounded answer from the model.
Definition GeminiAnswerRequest.cs:19
Response from the model for a grounded answer.
Definition GeminiAnswerResponse.cs:12
GeminiCandidate Answer
Candidate answer from the model.
Definition GeminiAnswerResponse.cs:20
Passage included inline with a grounding configuration.
Definition GeminiGroundingPassage.cs:12
A repeated list of passages.
Definition GeminiGroundingPassages.cs:11
Definition GeminiAttributionSourceId.cs:5
Definition GeminiGroundingPassage.cs:6
Definition GeminiAnswerRequest.cs:10
GeminiAnswerStyle
Style for grounded answers.
Definition GeminiAnswerStyle.cs:12
See GeminiAnswerResponse and GeminiAnswerRequest for more details.
CountTokens
Runs a model's tokenizer on input Content and returns the token count.
private async void RunTokenCountRequest()
{
Debug.Log("Running token counting request.");
{
{
}
}
);
Debug.Log($"Tokens: {response.TotalTokens}");
}
Request to count tokens in given content.
Definition GeminiTokenCountRequest.cs:14
A response from CountTokens.
Definition GeminiTokenCountResponse.cs:11
Definition GeminiTokenCountRequest.cs:8
See GeminiTokenCountResponse and GeminiTokenCountRequest for more details.
TunedModels (Unstable)
The TunedModels endpoint contains methods that allow you to access and inference fine-tuned Gemini models.
GenerateContent
Generates a model response given an input GenerateContentRequest.
private async void RunTunedModelChatRequest()
{
Debug.Log("Running chat request on tuned model.");
{
{
},
}
);
Debug.Log($"Tuned Gemini's response: {response.Parts[^1].Text}");
}
See GeminiChatResponse and GeminiChatRequest for more details.
Files (Beta API)
The Gemini File API can be used to store data on the cloud for future prompting with the Gemini models.
Delete
Deletes the File.
private async void RunDeleteFileRequest(string fileId)
{
Debug.Log("Deleting file...");
Debug.Log("File deleted.");
}
Requests the deletion of a file.
Definition GeminiFileDeleteRequest.cs:10
Definition GeminiFileDeleteRequest.cs:2
See GeminiFileDeleteRequest for more details.
Get
Gets the metadata for the given File.
private async Task<GeminiFile> RunGetFileRequest(string fileId)
{
}
Requests metadata for an existing file. Return type is GeminiFile.
Definition GeminiFileGetRequest.cs:10
Metadata for a file uploaded to the File API.
Definition GeminiFile.cs:13
See GeminiFile and GeminiFileGetRequest for more details.
List
Lists the metadata for Files owned by the requesting project.
private async Task<GeminiFile[]> RunListFilesRequest(int maxFiles = 10, string pageToken = null)
{
{
MaxResponseFiles = maxFiles,
PageToken = string.IsNullOrWhiteSpace(pageToken) ? string.Empty : pageToken,
});
}
Requests metadata for all existing files. Return type is GeminiFileListResponse.
Definition GeminiFileListRequest.cs:10
The response for a GeminiFileListRequest call.
Definition GeminiFileListResponse.cs:11
GeminiFile[] Files
The list of files.
Definition GeminiFileListResponse.cs:16
See GeminiFileListResponse and GeminiFileListRequest for more details.
Media (Beta API)
The Gemini File API can be used to store data on the cloud for future prompting with the Gemini models.
Upload
Creates a File.
private async Task<GeminiFile> RunUploadFileRequest(string text)
{
{
{
DisplayName = "I'm a File",
},
RawData = Encoding.UTF8.GetBytes(text)
});
}
Uploads a file to the Gemini File API. Response type is GeminiFileUploadResponse.
Definition GeminiFileUploadRequest.cs:16
Response for a file upload request.
Definition GeminiFileUploadResponse.cs:11
GeminiFile File
Metadata for the created file.
Definition GeminiFileUploadResponse.cs:15
GeminiContentType
Enum for the types of content able to be fed to the Gemini API.
Definition GeminiContentType.cs:12
See GeminiFileUploadResponse and GeminiFileUploadRequest for more details.
GeminiChatRequest Features
Streaming Responses
GeminiChatRequest allows you to stream Gemini's response in real-time. You can do so by using GeminiManager.StreamRequest and utilizing the callback in GeminiChatRequest.
You can even stream function calls! Check out the Streaming Generated Content sample included in the package.
[SerializeField] private Text _chatResponse;
private async Task<string> OnChat(string text)
{
{
Contents = new GeminiContent[]
{
GeminiContent.GetContent(text, GeminiRole.User),
},
OnPartialResponseReceived = streamedResponse =>
{
_chatResponse.text = streamedResponse.Parts[0].Text;
return Task.CompletedTask;
}
});
return response.
Parts[0].Text;
}
If you do not want to use the callback, you can let the StreamRequest task run in the background, and access the streamed data from the GeminiChatRequest.StreamedResponse property.
Adding Media Content to Requests
GeminiContent.Parts contains the actual contents of each chat request and response. You can add media content to the Parts array, but you must only have one type of data in each part, like one part of text, one part of an image, and so on. The following samples shows data being read from a file and into a GeminiContent object.
private async Task<GeminiContent> GetFileContent(
string filePath,
GeminiContentType contentType)
{
byte[] data;
try
{
data = await File.ReadAllBytesAsync(filePath);
}
catch (SystemException exception)
{
Debug.LogError($"Failed to load file: {exception.Message}");
return null;
}
{
{
{
Text = "What's in this file?"
},
{
{
MimeType = contentType,
Data = Convert.ToBase64String(data)
}
}
}
};
}
Raw media bytes.
Definition GeminiContentBlob.cs:14
A datatype containing media that is part of a multi-part Content message. Must only contain one field...
Definition GeminiContentPart.cs:14
Now, the GeminiContent returned by the method can be fed into a chat request!
Utility Methods
GeminiContent and GeminiContentBlob also contain static utility methods to help create them from Unity types like AudioClip or Texture2D:
GeminiContent.GetContent
- Can convert
string messages, Texture2D images, AudioClip* audio and GeminiFile data to GeminiContent objects.
GeminiContentBlob.GetContentBlob
- Can convert
Texture2D images and AudioClip* audio to GeminiContentBlob objects.
*Requires Utilities.Encoding.Wav.
Function Calling
First, we have to setup our tools and define our function schemas.
{
{
{
Name = "printToConsole",
Description = "Print text to the user's console.",
{
Properties = new Dictionary<string, GeminiSchema>()
{
{
{
Description = "The text to print. e.g. \"Hello, World!\"",
Nullable = false,
}
},
},
Required = new string[] { "text" },
}
},
{
Name = "changeTextColor",
Description = "Change the color of the text.",
{
Properties = new Dictionary<string, GeminiSchema>()
{
{
{
Description = "The color to set. e.g. \"BLUE\"",
{
"RED",
"GREEN",
"BLUE",
"WHITE",
},
Nullable = false,
}
},
},
Required = new string[] { "color" },
}
}
},
};
The Schema object allows the definition of input and output data types. These types can be objects,...
Definition GeminiSchema.cs:13
string[] Enum
Possible values of the element of GeminiSchemaDataType.String with enum format.
Definition GeminiSchema.cs:59
Definition GeminiSchema.cs:7
GeminiSchemaDataFormat
Defines the format of schema data.
Definition GeminiSchemaDataFormat.cs:12
@ Enum
A string enum value.
GeminiSchemaDataType
Contains the list of OpenAPI data types as defined by the OpenAPI Specification.
Definition GeminiSchemaDataType.cs:12
To use Gemini Tools, we need to declare each tool to use. So, we have created a declaration for function calling. Each tool must be declared separately but every function must be declared in a single tool declaration.
For each function, we need a declaration with a name and description. The parameters are an object of type GeminiSchema, which defines the schema of each of the parameters. The type is of GeminiSchemaDataType.Object, and contains the dictionary of parameter schemas.
The keys of the dictionary should be the parameter name, and the values should be GeminiSchema objects which define the type, description, format, etc. of the parameter.
Finally, we have the Required property which tells Gemini which fields are absolutely required in each call. Now, we can move on to the chat.
[SerializeField] private Text _chatResponse;
private async Task<string> OnChat(string text)
{
List<GeminiContent> contents = new()
{
};
string responseText = string.Empty;
do
{
{
Contents = contents.ToArray(),
});
responseText =
Array.Find(response.
Parts, part => !
string.IsNullOrEmpty(part.Text))?.Text;
functionCall = null;
for (int i = 0; i < allFunctionCalls.Length; i++)
{
JObject functionResponse = null;
switch (functionCall.
Name)
{
case "printToConsole":
Debug.Log(functionCall.
Arguments[
"text"].ToObject<
string>());
break;
case "changeTextColor":
if (!TryChangeTextColor(functionCall.
Arguments[
"color"].ToObject<
string>()))
{
functionResponse = new JObject()
{
["result"] = "Unknown color."
};
}
break;
default:
functionResponse = new JObject()
{
["result"] = "Sorry, but that function does not exist."
};
break;
}
{
["result"] = "Completed executing function successfully."
})));
}
} while (functionCall != null);
_chatResponse.text = responseText;
return responseText;
}
private bool TryChangeTextColor(string color)
{
switch (color)
{
case "RED":
_chatResponse.color = Color.red; break;
case "GREEN":
_chatResponse.color = Color.green; break;
case "BLUE":
_chatResponse.color = Color.blue; break;
case "WHITE":
_chatResponse.color = Color.white; break;
default:
return false;
}
Debug.Log("Changed text color!");
return true;
}
GeminiFunctionCall FunctionCall
A predicted FunctionCall returned from the model that contains a string representing the FunctionDecl...
Definition GeminiContentPart.cs:34
Here, we are going through each response, checking if a function was called, and calling the requested function.
The response is a JSON object, which is optional, but it is recommended to include. Note the use of GeminiToolConfiguration.GetConfiguration, which is a utility method to create a GeminiToolConfiguration with the given GeminiFunctionCallingMode. GeminiFunctionCallingMode.Any means Gemini will always call at least one function in each request, Auto means the model will call the functions when it thinks it needs to, and None means no functions can be called.
After the function is called, we respond by adding the calls and responses to the history. We use the GetResponse utility method to get a GeminiFunctionResponse object with the response JSON.
Function calling is, as of writing, only available in the Beta API.
Code Execution
Code execution is also a Tool, so it is similar to function calling:
{
};
[SerializeField] private Text _chatResponse;
private async Task<string> OnChat(string text)
{
List<GeminiContent> contents = new()
{
};
{
Contents = contents.ToArray(),
Tools =
new GeminiTool[] { _geminiCodeExecution },
});
string responseText =
string.Join(
", ",
Array.ConvertAll(response.
Parts, part => $
"(Text={part.Text}, Code={part.ExecutableCode?.Code}, ExecutionResult={part.CodeExecutionResult?.Output})"));
_chatResponse.text = responseText;
return responseText;
}
That's it! Now, when code execution is used, the response should be something like this:
> Make a simple python program to print hello world and use code execution for that.
Result: (Text=, Code=, ExecutionResult=), (Text=, Code=print("Hello world!"), ExecutionResult=), (Text=, Code=, ExecutionResult=Hello world!),
(Text=I have created a simple Python program that prints "Hello world!". I used the `print()` function to achieve this. The code was executed
using the `tool_code` block., Code=, ExecutionResult=)
Code execution is also, as of writing, only available in the Beta API.
JSON Response Mode
In JSON mode, Gemini will always respond in the specified JSON response schema.
private async Task<string> OnChat(string text)
{
{
{
},
SystemInstruction =
GeminiContent.
GetContent(
"You are a helpful math teacher who teacher their students mathematics in the most helpful way possible."),
{
{
Description = "A list of mathematical expressions.",
{
Properties = new Dictionary<string, GeminiSchema>()
{
{
{
}
},
{
{
}
},
},
Required = new string[] { "expression", "explanation", },
},
},
}
});
return response.
Parts[0].Text;
}
static readonly GeminiModelId Gemini1_5Pro
Gemini 1.5 Pro is a mid-size multimodal model that is optimized for a wide-range of reasoning tasks....
Definition GeminiModel.cs:46
Configuration options for model generation and outputs. Not all parameters may be configurable for ev...
Definition GeminiGenerationConfiguration.cs:13
Definition GeminiAnswerRequest.cs:10
GeminiResponseType
The response type for Gemini model responses.
Definition GeminiResponseType.cs:12
Here, we used a schema for an array of objects, which contain two parameters: expression and explanation. We have told Gemini to split the response into the parameters, where a mathematical expression and its explanation is given.
The GeminiSchema object is the same type used for function calling.
JSON mode is also only available in the Beta API.
Samples
For full-fledged examples of the features of this package, check out the samples included in the package:
Mult-turn Chat
A sample scene showing a multi-turn chat system. GitHub Source
Function Calling
A sample scene showing a function calling system. GitHub Source
Streaming Generated Content
A sample showing a system which streams Gemini's responses, including function calls. GitHub Source
Question Answering
A sample scene with a system where Gemini answers questions based only on the given context. GitHub Source
Prompting with File API
A sample scene with a system to create, delete, retrieve, list and prompt Gemini with files stored in the File/Media API endpoints. GitHub Source
JSON Response
A sample scene showing a system where Gemini responds in a specified JSON format. GitHub Source
List and Get Model Metadata
A sample scene with a system to list, get and chat with models using the models.get and models.list endpoints. GitHub Source
Token Counting
A sample scene showing a token counting system using the countTokens endpoint. GitHub Source
1.11.0