Class NativeAPI.Conversation

Namespace

Uralstech.UAI.LiteRTLM.Native

public static class NativeAPI.Conversation

Inheritance

object

NativeAPI.Conversation

Methods

litert_lm_conversation_cancel_process(nint)

Cancels the ongoing inference process, for asynchronous inference.

public static extern void litert_lm_conversation_cancel_process(nint conversation)

Parameters

conversation nint

The conversation to cancel the inference for.

litert_lm_conversation_clone(nint)

Clones a LiteRT LM Conversation, duplicating its prefilled state. The caller is responsible for destroying the cloned conversation using litert_lm_conversation_delete(nint).

public static extern nint litert_lm_conversation_clone(nint conversation)

Parameters

conversation nint

The conversation to clone.

Returns

nint

A pointer to the cloned conversation, or nint.Zero on failure.

litert_lm_conversation_create(nint, nint)

Creates a LiteRT LM Conversation. The caller is responsible for destroying the conversation using litert_lm_conversation_delete(nint).

public static extern nint litert_lm_conversation_create(nint engine, nint config)

Parameters

engine nint

The engine to create the conversation from.

config nint

The conversation config to use. If nint.Zero, the default config will be used.

Returns

nint

A pointer to the created conversation, or nint.Zero on failure.

litert_lm_conversation_delete(nint)

Destroys a LiteRT LM Conversation.

public static extern void litert_lm_conversation_delete(nint conversation)

Parameters

conversation nint

The conversation to destroy.

litert_lm_conversation_get_benchmark_info(nint)

Retrieves the benchmark information from the conversation. The caller is responsible for destroying the benchmark info using litert_lm_benchmark_info_delete(nint).

public static extern nint litert_lm_conversation_get_benchmark_info(nint conversation)

Parameters

conversation nint

The conversation to get the benchmark info from.

Returns

nint

A pointer to the benchmark info, or nint.Zero on failure.

litert_lm_conversation_get_token_count(nint)

Gets the number of tokens in the conversation KV Cache (prefill + decode).

public static extern int litert_lm_conversation_get_token_count(nint conversation)

Parameters

conversation nint

Returns

int

The number of tokens, or a negative value on failure.

litert_lm_conversation_render_message_to_string(nint, string)

Renders the message into a string according to the template.

public static extern nint litert_lm_conversation_render_message_to_string(nint conversation, string messageJson)

Parameters

conversation nint

The conversation instance.

messageJson string

A JSON string representing the message to render.

Returns

nint

A pointer to the rendered string, or nint.Zero on failure. The returned string is owned by the Conversation object and is valid until the next call to this function or until the conversation is deleted.

Remarks

This function does not need to be called for actual message sending, as the litert_lm_conversation_send_message(nint, string, string?, nint) and litert_lm_conversation_send_message_stream(nint, string, string?, nint, nint, nint) functions will handle rendering internally.

litert_lm_conversation_render_preface_to_string(nint)

Renders the preface into a string according to the template.

public static extern nint litert_lm_conversation_render_preface_to_string(nint conversation)

Parameters

conversation nint

The conversation instance.

Returns

nint

A pointer to the rendered string, or nint.Zero on failure. The returned string is owned by the Conversation object and is valid until the next call to this function or until the conversation is deleted.

litert_lm_conversation_send_message(nint, string, string?, nint)

Sends a message to the conversation and returns the response. This is a blocking call.

public static extern nint litert_lm_conversation_send_message(nint conversation, string messageJson, string? extraContext, nint optionalArgs)

Parameters

conversation nint

The conversation to use.

messageJson string

A JSON string representing the message to send.

extraContext string

A JSON string representing the extra context to use.

optionalArgs nint

A pointer to the optional arguments to use.

Returns

nint

A pointer to the JSON Response, or nint.Zero on failure. The caller is responsible for deleting the response using litert_lm_json_response_delete(nint).

litert_lm_conversation_send_message_stream(nint, string, string?, nint, nint, nint)

Sends a message to the conversation and streams the response via a callback. This is a non-blocking call that will invoke the callback from a background thread for each chunk.

public static extern int litert_lm_conversation_send_message_stream(nint conversation, string messageJson, string? extraContext, nint optionalArgs, nint callback, nint callbackData)

Parameters

conversation nint

The conversation to use.

messageJson string

A JSON string representing the message to send.

extraContext string

A JSON string representing the extra context to use.

optionalArgs nint

A pointer to the optional arguments to use.

callback nint

The callback function (NativeAPI.StreamCallback) to receive response chunks.

callbackData nint

A pointer to user data that will be passed to the callback.

Returns

int

0 on success, non-zero on failure to start the stream.