Class NativeAPI.NoRepeatNgramConfig

Namespace

Uralstech.UAI.LiteRTLM.Native

public static class NativeAPI.NoRepeatNgramConfig

Inheritance

object

NativeAPI.NoRepeatNgramConfig

Methods

litert_lm_no_repeat_ngram_config_create()

Creates a LiteRT LM No Repeat Ngram Config with default values (no_repeat_ngram_size = 0, window_size = 0, which means no repeat ngram banning is disabled).

public static extern nint litert_lm_no_repeat_ngram_config_create()

Returns

nint

A pointer to the created config, or nint.Zero on failure.

Remarks

When no_repeat_ngram_size is set greater than 0, any sequence of tokens (an ngram of that exact length) generated during decoding or present inside the window history can only occur at most once. If generating a candidate token would complete a repeating ngram, that candidate token's logit is set to -inf.

The caller is responsible for destroying the config using litert_lm_no_repeat_ngram_config_delete(nint).

litert_lm_no_repeat_ngram_config_delete(nint)

Destroys a LiteRT LM No Repeat Ngram Config.

public static extern void litert_lm_no_repeat_ngram_config_delete(nint config)

Parameters

config nint

The config to destroy.

litert_lm_no_repeat_ngram_config_set_no_repeat_ngram_size(nint, int)

Sets the no repeat ngram size for the no repeat ngram config.

public static extern void litert_lm_no_repeat_ngram_config_set_no_repeat_ngram_size(nint config, int noRepeatNgramSize)

Parameters

config nint

The config to modify.

noRepeatNgramSize int

The size of ngrams (consecutive token sequences) that are banned from repeating within the generation history window. If set > 0, when generating the next token would complete an already observed no_repeat_ngram_size sequence, the logit of the candidate token is set to -inf. If set <= 0, no repeat ngram banning is disabled. Negative values are automatically clamped to 0 during execution.

litert_lm_no_repeat_ngram_config_set_window_size(nint, int)

Sets the window size for the no repeat ngram config.

public static extern void litert_lm_no_repeat_ngram_config_set_window_size(nint config, int windowSize)

Parameters

config nint

The config to modify.

windowSize int

The maximum number of recent tokens in generation history to consider when checking for repeating ngrams. Tokens generated prior to this window are forgotten. A value of 0 means tracking all infinite generation history. Must be >= 0; negative values are clamped to 0. If window_size is greater than 0 but less than no_repeat_ngram_size, it is automatically clamped to no_repeat_ngram_size so that the ngrams can fit and be tracked.