Class Conversation

Namespace

Uralstech.UAI.LiteRTLM

public sealed class Conversation : LiteRTLMNativeHandle

Inheritance

object

LiteRTLMNativeHandle

Conversation

Inherited Members

LiteRTLMNativeHandle.Dispose()

Constructors

Conversation(Engine, ConversationConfig?)

Creates a managed wrapper around a LiteRT LM conversation. The caller is responsible for disposing the wrapper using Dispose().

public Conversation(Engine engine, ConversationConfig? config = null)

Parameters

engine Engine

The engine to create the conversation from.

config ConversationConfig

The conversation configuration to use. If null, the default configuration is used.

Exceptions

InvalidOperationException

Thrown if the native object could not be created.

Methods

CancelProcess()

Cancels the ongoing inference process for asynchronous inference.

public void CancelProcess()

Clone()

Clones a LiteRT LM conversation, duplicating its prefilled state. The caller is responsible for disposing the cloned wrapper using Dispose().

public Conversation? Clone()

Returns

Conversation

The cloned conversation wrapper, or null on failure.

GetBenchmarkInfo()

Retrieves benchmark information from the conversation. The caller is responsible for disposing the returned wrapper.

public BenchmarkInfo? GetBenchmarkInfo()

Returns

BenchmarkInfo

The benchmark information wrapper, or null on failure.

GetTokenCount()

Gets the number of tokens in the conversation KV cache (prefill + decode).

public int GetTokenCount()

Returns

int

The number of tokens, or a negative value on failure.

ReleaseUnmanagedResources()

protected override void ReleaseUnmanagedResources()

RenderMessageToString(string)

Renders the message into a string according to the template.

public string? RenderMessageToString(string messageJson)

Parameters

messageJson string

A JSON string representing the message to render.

Returns

string

The rendered string, or null on failure.

Remarks

This function does not need to be called for actual message sending, as SendMessage(string, string?, ConversationOptionalArgs?) and SendMessageStream(StreamCallback, string, string?, ConversationOptionalArgs?) render messages internally.

RenderPrefaceToString()

Renders the preface into a string according to the template.

public string? RenderPrefaceToString()

Returns

string

The rendered string, or null on failure.

SendMessage(string, string?, ConversationOptionalArgs?)

Sends a message to the conversation and returns the response. This is a blocking call.

public JsonResponse? SendMessage(string messageJson, string? extraContext = null, ConversationOptionalArgs? optionalArgs = null)

Parameters

messageJson string

A JSON string representing the message to send.

extraContext string

A JSON string representing the extra context to use.

optionalArgs ConversationOptionalArgs

The optional arguments to use.

Returns

JsonResponse

A JSON response wrapper, or null on failure. The caller is responsible for disposing the returned wrapper.

SendMessageStream(StreamCallback, string, string?, ConversationOptionalArgs?)

Sends a message to the conversation and streams the response through a callback. This is a non-blocking call that invokes the callback from a background thread for each chunk.

public int SendMessageStream(StreamCallback callback, string messageJson, string? extraContext = null, ConversationOptionalArgs? optionalArgs = null)

Parameters

callback StreamCallback

The callback function (StreamCallback) that receives response chunks.

messageJson string

A JSON string representing the message to send.

extraContext string

A JSON string representing the extra context to use.

optionalArgs ConversationOptionalArgs

The optional arguments to use.

Returns

int

0 on success, non-zero on failure to start the stream.