Class NativeAPI.BenchmarkInfo

Namespace

Uralstech.UAI.LiteRTLM.Native

public static class NativeAPI.BenchmarkInfo

Inheritance

object

NativeAPI.BenchmarkInfo

Methods

litert_lm_benchmark_info_delete(nint)

Destroys a LiteRT LM Benchmark Info object.

public static extern void litert_lm_benchmark_info_delete(nint benchmarkInfo)

Parameters

benchmarkInfo nint

The benchmark info to destroy.

litert_lm_benchmark_info_get_decode_token_count_at(nint, int)

Returns the decode token count at a given turn index.

public static extern int litert_lm_benchmark_info_get_decode_token_count_at(nint benchmarkInfo, int index)

Parameters

benchmarkInfo nint

The benchmark info object.

index int

The index of the decode turn.

Returns

int

The decode token count.

litert_lm_benchmark_info_get_decode_tokens_per_sec_at(nint, int)

Returns the decode tokens per second at a given turn index.

public static extern double litert_lm_benchmark_info_get_decode_tokens_per_sec_at(nint benchmarkInfo, int index)

Parameters

benchmarkInfo nint

The benchmark info object.

index int

The index of the decode turn.

Returns

double

The decode tokens per second.

litert_lm_benchmark_info_get_num_decode_turns(nint)

Returns the number of decode turns.

public static extern int litert_lm_benchmark_info_get_num_decode_turns(nint benchmarkInfo)

Parameters

benchmarkInfo nint

The benchmark info object.

Returns

int

The number of decode turns.

litert_lm_benchmark_info_get_num_prefill_turns(nint)

Returns the number of prefill turns.

public static extern int litert_lm_benchmark_info_get_num_prefill_turns(nint benchmarkInfo)

Parameters

benchmarkInfo nint

The benchmark info object.

Returns

int

The number of prefill turns.

litert_lm_benchmark_info_get_prefill_token_count_at(nint, int)

Returns the prefill token count at a given turn index.

public static extern int litert_lm_benchmark_info_get_prefill_token_count_at(nint benchmarkInfo, int index)

Parameters

benchmarkInfo nint

The benchmark info object.

index int

The index of the prefill turn.

Returns

int

The prefill token count.

litert_lm_benchmark_info_get_prefill_tokens_per_sec_at(nint, int)

Returns the prefill tokens per second at a given turn index.

public static extern double litert_lm_benchmark_info_get_prefill_tokens_per_sec_at(nint benchmarkInfo, int index)

Parameters

benchmarkInfo nint

The benchmark info object.

index int

The index of the prefill turn.

Returns

double

The prefill tokens per second.

litert_lm_benchmark_info_get_time_to_first_token(nint)

Returns the time to the first token in seconds.

public static extern double litert_lm_benchmark_info_get_time_to_first_token(nint benchmarkInfo)

Parameters

benchmarkInfo nint

The benchmark info object.

Returns

double

The time to the first token in seconds.

Remarks

Note that the first time to token doesn't include the time for initialization. It is the sum of the prefill time for the first turn and the time spent for decoding the first token.

litert_lm_benchmark_info_get_total_init_time_in_second(nint)

Returns the total initialization time in seconds.

public static extern double litert_lm_benchmark_info_get_total_init_time_in_second(nint benchmarkInfo)

Parameters

benchmarkInfo nint

The benchmark info object.

Returns

double

The total initialization time in seconds.