Scalable Inference Serving · Schema

InferenceRequest

Request body for submitting model inference. Contains input tensors and optionally specifies which outputs to return.

AICNCFDeploymentInferenceKubernetesLLMMachine LearningModel ServingMLOpsScalability

Properties

Name Type Description
id string Optional request identifier. If provided, it is echoed back in the response for correlation.
parameters object Optional key/value parameters passed to the model.
inputs array Input tensors for the inference request.
outputs array Optional list of output tensors to return. If omitted, all outputs are returned.
View JSON Schema on GitHub

JSON Schema

scalable-inference-serving-inferencerequest-schema.json Raw ↑ 
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "#/components/schemas/InferenceRequest",
  "title": "InferenceRequest",
  "type": "object",
  "description": "Request body for submitting model inference. Contains input tensors and optionally specifies which outputs to return.",
  "required": [
    "inputs"
  ],
  "properties": {
    "id": {
      "type": "string",
      "description": "Optional request identifier. If provided, it is echoed back in the response for correlation.",
      "example": "req-a1b2c3d4"
    },
    "parameters": {
      "type": "object",
      "additionalProperties": true,
      "description": "Optional key/value parameters passed to the model."
    },
    "inputs": {
      "type": "array",
      "description": "Input tensors for the inference request.",
      "minItems": 1,
      "items": {
        "$ref": "#/components/schemas/RequestInput"
      }
    },
    "outputs": {
      "type": "array",
      "description": "Optional list of output tensors to return. If omitted, all outputs are returned.",
      "items": {
        "$ref": "#/components/schemas/RequestOutput"
      }
    }
  }
}