Class SessionConfig

Namespace

Uralstech.UAI.LiteRTLM

public sealed class SessionConfig : LiteRTLMNativeHandle

Inheritance

object

LiteRTLMNativeHandle

SessionConfig

Inherited Members

LiteRTLMNativeHandle.Dispose()

Constructors

SessionConfig()

Creates a managed wrapper around a LiteRT LM session configuration. The caller is responsible for disposing the wrapper using Dispose().

public SessionConfig()

Exceptions

InvalidOperationException

Thrown if the native object could not be created.

Methods

ReleaseUnmanagedResources()

protected override void ReleaseUnmanagedResources()

SetApplyPromptTemplate(bool)

Sets whether to apply the prompt template for this session.

public void SetApplyPromptTemplate(bool applyPromptTemplate)

Parameters

applyPromptTemplate bool

Whether to apply the prompt template.

SetAudioLoraPath(string)

Sets the path to the audio LoRA weights file.

public int SetAudioLoraPath(string audioLoraPath)

Parameters

audioLoraPath string

The path to the audio LoRA weights file.

Returns

int

0 on success, non-zero on failure.

SetEnableSpeculativeDecoding(bool)

Sets whether to enable speculative decoding for this session.

public void SetEnableSpeculativeDecoding(bool enableSpeculativeDecoding)

Parameters

enableSpeculativeDecoding bool

Whether to enable speculative decoding.

Remarks

If set to true, speculative decoding is enabled for this session. If the engine was not initialized with speculative decoding enabled, setting this flag to true causes the executor to perform lazy loading of the MTP drafter on the first session request. If set to false, speculative decoding is explicitly disabled for this session even if the engine was initialized with speculative decoding enabled. If not called, the session inherits the engine's default.

SetLoraPath(string)

Sets the path to the LoRA weights file.

public int SetLoraPath(string loraPath)

Parameters

loraPath string

The path to the text LoRA weights file.

Returns

int

0 on success, non-zero on failure.

SetMaxOutputTokens(int)

Sets the maximum number of output tokens per decode step for this session.

public void SetMaxOutputTokens(int maxOutputTokens)

Parameters

maxOutputTokens int

The maximum number of tokens to generate (including thinking tokens).

Remarks

For thinking models, both thinking (reasoning) tokens and the final response tokens count towards this limit.

SetSamplerParams(SamplerParams)

Sets the sampler parameters for this session configuration.

public void SetSamplerParams(SamplerParams samplerParams)

Parameters

samplerParams SamplerParams

The sampler parameters to use.