Skip to content

InferenceNodeConfiguration

HTTP API schema

InferenceNodeConfiguration fields

modelstring

The AI model to use for inference

Default: default

temperaturenumber

Sampling temperature (0.0 to 2.0)

minimum: 0

maximum: 2

Default: 1

maxTokensinteger

Maximum number of tokens to generate

minimum: 1

maximum: 4096

Default: 256

View source specification ↗