Class NativeAPI.SuppressTokensConfig

Namespace

Uralstech.UAI.LiteRTLM.Native

public static class NativeAPI.SuppressTokensConfig

Inheritance

object

NativeAPI.SuppressTokensConfig

Methods

litert_lm_suppress_tokens_config_create()

Creates a LiteRT LM Suppress Tokens Config with default values (an empty set of suppressed tokens, which means token suppression is disabled).

public static extern nint litert_lm_suppress_tokens_config_create()

Returns

nint

A pointer to the created config, or IntPtr.Zero on failure.

Remarks

When suppress_tokens is configured with one or more token IDs, the logits corresponding to those exact token IDs will be set directly to -inf during generation. This guarantees that those tokens can never be sampled by the model.

The caller is responsible for destroying the config using litert_lm_suppress_tokens_config_delete(nint).

litert_lm_suppress_tokens_config_delete(nint)

Destroys a LiteRT LM Suppress Tokens Config.

public static extern void litert_lm_suppress_tokens_config_delete(nint config)

Parameters

config nint

The config to destroy.

litert_lm_suppress_tokens_config_set_suppress_tokens(nint, int[]?, nuint)

Sets the list of token IDs to suppress for the suppress tokens config.

public static extern void litert_lm_suppress_tokens_config_set_suppress_tokens(nint config, int[]? suppressTokens, nuint numTokens)

Parameters

config nint

The config to modify.

suppressTokens int[]

An array of integer token IDs that should be banned from generation. During every decode step, each listed token ID's candidate logit will be forced to -inf. If suppress_tokens is null or num_tokens is 0, any previously set suppressed tokens are cleared and token suppression is disabled.

numTokens nuint

The number of token IDs in the suppress_tokens array.