Table of Contents

Class NoRepeatNgramConfig

Namespace
Uralstech.UAI.LiteRTLM
public sealed class NoRepeatNgramConfig : LiteRTLMNativeHandle
Inheritance
object
NoRepeatNgramConfig
Inherited Members

Constructors

NoRepeatNgramConfig()

Creates a LiteRT LM No Repeat Ngram Config with default values (no_repeat_ngram_size = 0, window_size = 0, which means no repeat ngram banning is disabled).

public NoRepeatNgramConfig()

Remarks

When no_repeat_ngram_size is set greater than 0, any sequence of tokens (an ngram of that exact length) generated during decoding or present inside the window history can only occur at most once. If generating a candidate token would complete a repeating ngram, that candidate token's logit is set to -inf.

The caller is responsible for disposing the wrapper using Dispose().

Exceptions

InvalidOperationException

Thrown if the native object could not be created.

Methods

ReleaseUnmanagedResources() 

protected override void ReleaseUnmanagedResources()

SetNoRepeatNgramSize(int)

Sets the no repeat ngram size for the no repeat ngram config.

public void SetNoRepeatNgramSize(int noRepeatNgramSize)

Parameters

noRepeatNgramSize int

The size of ngrams (consecutive token sequences) that are banned from repeating within the generation history window. If set > 0, when generating the next token would complete an already observed no_repeat_ngram_size sequence, the logit of the candidate token is set to -inf. If set <= 0, no repeat ngram banning is disabled. Negative values are automatically clamped to 0 during execution.

SetWindowSize(int)

Sets the window size for the no repeat ngram config.

public void SetWindowSize(int windowSize)

Parameters

windowSize int

The maximum number of recent tokens in generation history to consider when checking for repeating ngrams. Tokens generated prior to this window are forgotten. A value of 0 means tracking all infinite generation history. Must be >= 0; negative values are clamped to 0. If window_size is greater than 0 but less than no_repeat_ngram_size, it is automatically clamped to no_repeat_ngram_size so that the ngrams can fit and be tracked.

Edit this page