> ## Documentation Index > Fetch the complete documentation index at: https://azure-foundry.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Post evals > Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and a datasource. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the [Evals guide](/docs/guides/evals). NOTE: This Azure OpenAI API is in preview and subject to change. ## OpenAPI ````yaml api-reference/openai-v1.yaml post /evals openapi: 3.0.0 info: title: Azure AI Foundry Models Service license: name: MIT url: https://github.com/openai/openai-openapi/blob/master/LICENSE version: v1 servers: - url: '{endpoint}/openai/v1' description: Azure AI Foundry Models APIs variables: endpoint: default: '' description: >- A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com). security: - ApiKeyAuth: [] - ApiKeyAuth_: [] - OAuth2Auth: - https://cognitiveservices.azure.com/.default tags: - name: Chat - name: Embeddings - name: Evals - name: Files - name: Fine-tuning - name: Models - name: Responses - name: Vector Stores paths: /evals: post: tags: - Evals description: |- Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and a datasource. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the [Evals guide](/docs/guides/evals). NOTE: This Azure OpenAI API is in preview and subject to change. operationId: createEval parameters: - name: api-version in: query required: false description: >- The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified. schema: $ref: '#/components/schemas/AzureAIFoundryModelsApiVersion' default: v1 - name: aoai-evals in: header required: true description: >- Enables access to AOAI Evals, a preview feature. This feature requires the 'aoai-evals' header to be set to 'preview'. schema: type: string enum: - preview requestBody: required: true content: application/json: schema: type: object properties: statusCode: type: number enum: - 201 name: type: string description: The name of the evaluation. metadata: type: object additionalProperties: type: string description: >- Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. x-oaiTypeLabel: map data_source_config: allOf: - $ref: '#/components/schemas/OpenAI.EvalDataSourceConfigParams' description: >- The configuration for the data source used for the evaluation runs. Dictates the schema of the data used in the evaluation. testing_criteria: type: array items: $ref: '#/components/schemas/OpenAI.EvalGraderParams' description: >- A list of graders for all eval runs in this group. Graders can reference variables in the data source using double curly braces notation, like `{{item.variable_name}}`. To reference the model's output, use the `sample` namespace (ie, `{{sample.output_text}}`). required: - statusCode - data_source_config - testing_criteria responses: '200': description: The request has succeeded. headers: apim-request-id: required: false description: A request ID used for troubleshooting purposes. schema: type: string content: application/json: schema: $ref: '#/components/schemas/OpenAI.Eval' default: description: An unexpected error response. headers: apim-request-id: required: false description: A request ID used for troubleshooting purposes. schema: type: string content: application/json: schema: $ref: '#/components/schemas/AzureErrorResponse' components: schemas: AzureAIFoundryModelsApiVersion: type: string enum: - v1 - preview OpenAI.EvalDataSourceConfigParams: type: object required: - type properties: type: $ref: '#/components/schemas/OpenAI.EvalDataSourceConfigType' discriminator: propertyName: type mapping: custom: '#/components/schemas/OpenAI.EvalCustomDataSourceConfigParams' logs: '#/components/schemas/OpenAI.EvalLogsDataSourceConfigParams' stored_completions: >- #/components/schemas/OpenAI.EvalStoredCompletionsDataSourceConfigParams OpenAI.EvalGraderParams: type: object required: - type properties: type: $ref: '#/components/schemas/OpenAI.GraderType' discriminator: propertyName: type mapping: label_model: '#/components/schemas/OpenAI.EvalGraderLabelModelParams' string_check: '#/components/schemas/OpenAI.EvalGraderStringCheckParams' text_similarity: '#/components/schemas/OpenAI.EvalGraderTextSimilarityParams' python: '#/components/schemas/OpenAI.EvalGraderPythonParams' score_model: '#/components/schemas/OpenAI.EvalGraderScoreModelParams' OpenAI.Eval: type: object required: - object - id - name - data_source_config - testing_criteria - created_at - metadata properties: object: type: string enum: - eval description: The object type. default: eval id: type: string description: Unique identifier for the evaluation. name: type: string description: The name of the evaluation. data_source_config: allOf: - $ref: '#/components/schemas/OpenAI.EvalDataSourceConfigResource' description: Configuration of data sources used in runs of the evaluation. testing_criteria: type: array items: $ref: '#/components/schemas/OpenAI.EvalGraderResource' nullable: true description: A list of testing criteria. default: null created_at: type: integer format: unixtime description: The Unix timestamp (in seconds) for when the eval was created. metadata: type: object additionalProperties: type: string nullable: true description: >- Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. x-oaiTypeLabel: map description: |- An Eval object with a data source config and testing criteria. An Eval represents a task to be done for your LLM integration. Like: - Improve the quality of my chatbot - See how well my chatbot handles customer support - Check if o4-mini is better at my usecase than gpt-4o AzureErrorResponse: type: object properties: error: type: object properties: code: type: string description: The distinct, machine-generated identifier for the error. message: type: string description: A human-readable message associated with the error. param: type: string description: >- If applicable, the request input parameter associated with the error type: type: string enum: - error description: The object type, always 'error.' inner_error: {} description: The error details. OpenAI.EvalDataSourceConfigType: type: string enum: - custom - logs - stored_completions OpenAI.GraderType: type: string enum: - string_check - text_similarity - score_model - label_model - python - multi OpenAI.EvalDataSourceConfigResource: type: object required: - type properties: type: $ref: '#/components/schemas/OpenAI.EvalDataSourceConfigType' discriminator: propertyName: type mapping: custom: '#/components/schemas/OpenAI.EvalCustomDataSourceConfigResource' stored_completions: >- #/components/schemas/OpenAI.EvalStoredCompletionsDataSourceConfigResource logs: '#/components/schemas/OpenAI.EvalLogsDataSourceConfigResource' OpenAI.EvalGraderResource: type: object required: - type properties: type: $ref: '#/components/schemas/OpenAI.GraderType' discriminator: propertyName: type mapping: label_model: '#/components/schemas/OpenAI.EvalGraderLabelModelResource' text_similarity: '#/components/schemas/OpenAI.EvalGraderTextSimilarityResource' python: '#/components/schemas/OpenAI.EvalGraderPythonResource' score_model: '#/components/schemas/OpenAI.EvalGraderScoreModelResource' securitySchemes: ApiKeyAuth: type: apiKey in: header name: api-key ApiKeyAuth_: type: apiKey in: header name: authorization OAuth2Auth: type: oauth2 flows: implicit: authorizationUrl: https://login.microsoftonline.com/common/oauth2/v2.0/authorize scopes: https://cognitiveservices.azure.com/.default: '' ````