Class Session

Namespace

Uralstech.UAI.LiteRTLM

public sealed class Session : LiteRTLMNativeHandle

Inheritance

object

LiteRTLMNativeHandle

Session

Inherited Members

LiteRTLMNativeHandle.Dispose()

Methods

CancelProcess()

Cancels the current processing in the session.

public void CancelProcess()

GenerateContent(ReadOnlySpan<InputData>)

Generates content from the input prompt.

public Responses? GenerateContent(ReadOnlySpan<InputData> inputs)

Parameters

inputs ReadOnlySpan<InputData>

An array of input wrappers representing multimodal input.

Returns

Responses

The responses wrapper, or null on failure. The caller is responsible for disposing the returned wrapper.

GenerateContentStream(ReadOnlySpan<InputData>, StreamCallback)

Generates content from the input prompt and streams the response through a callback. This is a non-blocking call and invokes the callback from a background thread for each chunk.

public int GenerateContentStream(ReadOnlySpan<InputData> inputs, StreamCallback callback)

Parameters

inputs ReadOnlySpan<InputData>

An array of input wrappers representing multimodal input.

callback StreamCallback

The callback function (StreamCallback) that receives response chunks.

Returns

int

0 on success, non-zero on failure to start the stream.

GetBenchmarkInfo()

Retrieves benchmark information for the session. The caller is responsible for disposing the returned wrapper.

public BenchmarkInfo? GetBenchmarkInfo()

Returns

BenchmarkInfo

The benchmark information wrapper, or null on failure.

ReleaseUnmanagedResources()

protected override void ReleaseUnmanagedResources()

RewindToCheckpoint(string)

Rewinds the session to the given checkpoint label.

public int RewindToCheckpoint(string label)

Parameters

label string

Label of the checkpoint to rewind to.

Returns

int

0 on success, non-zero on failure.

RewindToStep(int)

Rewinds the session to a specific step number.

public int RewindToStep(int step)

Parameters

step int

The step number to rewind to.

Returns

int

0 on success, non-zero on failure.

RunDecode()

Starts decoding for the model to predict a response based on the input prompt or query added after calling RunPrefill(ReadOnlySpan<InputData>). This is a blocking call and returns when decoding is complete.

public Responses? RunDecode()

Returns

Responses

The responses wrapper, or null on failure. The caller is responsible for disposing the returned wrapper.

RunDecodeAsync(StreamCallback)

Starts decoding for the model to predict a response based on the input prompt or query added after calling RunPrefill(ReadOnlySpan<InputData>). This is a non-blocking call that streams response chunks through a callback.

public int RunDecodeAsync(StreamCallback callback)

Parameters

callback StreamCallback

The callback function (StreamCallback) that receives response chunks.

Returns

int

0 on success, non-zero on failure.

RunPrefill(ReadOnlySpan<InputData>)

Adds the input prompt or query to the model and starts the prefill process. This is a blocking call and returns when prefill is complete.

public int RunPrefill(ReadOnlySpan<InputData> inputs)

Parameters

inputs ReadOnlySpan<InputData>

An array of input wrappers representing multimodal input.

Returns

int

0 on success, non-zero on failure.

RunTextScoring(string[], bool)

Scores the target text after the prefill process is complete.

public Responses? RunTextScoring(string[] targetText, bool storeTokenLengths)

Parameters

targetText string[]

An array of target text strings to score.

storeTokenLengths bool

Whether to store the token lengths of the target texts in the responses.

Returns

Responses

The responses wrapper, or null on failure. The caller is responsible for disposing the returned wrapper.

SaveCheckpoint(string)

Saves the current state of the session to a checkpoint with the given label.

public int SaveCheckpoint(string label)

Parameters

label string

Label for the checkpoint.

Returns

int

0 on success, non-zero on failure.