Class SuppressTokensConfig

Namespace: Uralstech.UAI.LiteRTLM

public sealed class SuppressTokensConfig : LiteRTLMNativeHandle

Inheritance: object

LiteRTLMNativeHandle

SuppressTokensConfig

Inherited Members: LiteRTLMNativeHandle.Dispose()

Constructors

SuppressTokensConfig()

Creates a LiteRT LM Suppress Tokens Config with default values (an empty set of suppressed tokens, which means token suppression is disabled).

public SuppressTokensConfig()

Remarks

When suppress_tokens is configured with one or more token IDs, the logits corresponding to those exact token IDs will be set directly to -inf during generation. This guarantees that those tokens can never be sampled by the model.

The caller is responsible for disposing the wrapper using Dispose().

Exceptions

InvalidOperationException: Thrown if the native object could not be created.

Methods

ReleaseUnmanagedResources()

protected override void ReleaseUnmanagedResources()

SetSuppressTokens(int[]?)

Sets the list of token IDs to suppress for the suppress tokens config.

public void SetSuppressTokens(int[]? suppressTokens)

Parameters

suppressTokens int[]: An array of integer token IDs that should be banned from generation. During every decode step, each listed token ID's candidate logit will be forced to -inf. If suppress_tokens is null or empty, any previously set suppressed tokens are cleared and token suppression is disabled.

Table of Contents