Class NativeAPI.SessionConfig

Namespace

Uralstech.UAI.LiteRTLM.Native

public static class NativeAPI.SessionConfig

Inheritance

object

NativeAPI.SessionConfig

Methods

litert_lm_session_config_create()

Creates a LiteRT LM Session Config. The caller is responsible for destroying the config using litert_lm_session_config_delete(nint).

public static extern nint litert_lm_session_config_create()

Returns

nint

A pointer to the created config, or nint.Zero on failure.

litert_lm_session_config_delete(nint)

Destroys a LiteRT LM Session Config.

public static extern void litert_lm_session_config_delete(nint config)

Parameters

config nint

The config to destroy.

litert_lm_session_config_set_apply_prompt_template(nint, bool)

Sets whether to apply prompt template for this session.

public static extern void litert_lm_session_config_set_apply_prompt_template(nint config, bool applyPromptTemplate)

Parameters

config nint

The config to modify.

applyPromptTemplate bool

Whether to apply prompt template.

litert_lm_session_config_set_audio_lora_path(nint, string)

Sets the path to the Audio LoRA weights file.

public static extern int litert_lm_session_config_set_audio_lora_path(nint config, string audioLoraPath)

Parameters

config nint

The config to modify.

audioLoraPath string

The path to the audio LoRA weights file.

Returns

int

0 on success, non-zero on failure.

litert_lm_session_config_set_enable_speculative_decoding(nint, bool)

Sets whether to enable speculative decoding for this session.

public static extern void litert_lm_session_config_set_enable_speculative_decoding(nint config, bool enableSpeculativeDecoding)

Parameters

config nint

The config to modify.

enableSpeculativeDecoding bool

Whether to enable speculative decoding. If set to true, speculative decoding is enabled for this session. If the engine was not initialized with speculative decoding enabled, setting this flag to true causes the executor to perform lazy loading of the MTP drafter on the first session request. If set to false, speculative decoding is explicitly disabled for this session even if the engine was initialized with speculative decoding enabled. If this function is not called on the config, the session inherits the engine's speculative decoding setting by default.

litert_lm_session_config_set_lora_path(nint, string)

Sets the path to the LoRA weights file.

public static extern int litert_lm_session_config_set_lora_path(nint config, string loraPath)

Parameters

config nint

The config to modify.

loraPath string

The path to the text LoRA weights file.

Returns

int

0 on success, non-zero on failure.

litert_lm_session_config_set_max_output_tokens(nint, int)

Sets the maximum number of output tokens per decode step for this session.

public static extern void litert_lm_session_config_set_max_output_tokens(nint config, int maxOutputTokens)

Parameters

config nint

The config to modify.

maxOutputTokens int

The maximum number of tokens to generate (including thinking tokens).

Remarks

For thinking models, both thinking (reasoning) tokens and the final response tokens count towards this limit.

litert_lm_session_config_set_sampler_params(nint, nint)

Sets the sampler parameters for this session config.

public static extern void litert_lm_session_config_set_sampler_params(nint config, nint samplerParams)

Parameters

config nint

The config to modify.

samplerParams nint

The sampler parameters to use.