Scalable Inference Serving · Schema

RequestInput

A single input tensor for an inference request.

AICNCFDeploymentInferenceKubernetesLLMMachine LearningModel ServingMLOpsScalability

Properties

Name Type Description
name string Name of the input tensor (must match the model's input name).
shape array Shape of the input tensor.
datatype object
parameters object
data object Tensor data in row-major order. Can be a flat array or nested arrays matching the tensor shape. Data type must match the declared datatype.
View JSON Schema on GitHub

JSON Schema

scalable-inference-serving-requestinput-schema.json Raw ↑ 
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "#/components/schemas/RequestInput",
  "title": "RequestInput",
  "type": "object",
  "description": "A single input tensor for an inference request.",
  "required": [
    "name",
    "shape",
    "datatype",
    "data"
  ],
  "properties": {
    "name": {
      "type": "string",
      "description": "Name of the input tensor (must match the model's input name)."
    },
    "shape": {
      "type": "array",
      "items": {
        "type": "integer"
      },
      "description": "Shape of the input tensor.",
      "example": [
        1,
        128
      ]
    },
    "datatype": {
      "$ref": "#/components/schemas/TensorDatatype"
    },
    "parameters": {
      "type": "object",
      "additionalProperties": true
    },
    "data": {
      "description": "Tensor data in row-major order. Can be a flat array or nested arrays matching the tensor shape. Data type must match the declared datatype.",
      "oneOf": [
        {
          "type": "array",
          "items": {}
        },
        {
          "type": "string"
        }
      ]
    }
  }
}