@archastro/sdk
    Preparing search index...

    Class EvaluationResource

    Index

    Constructors

    constructor

    Methods

    evaluations models

    Constructors

    Methods

    • Create an evaluation Sends a shared state and a list of typed questions to an evaluation model and returns typed answers. This is not chat completion and not JSON-schema structured output: there is no generated text. Each question is a boolean, choice, or score object with its own id. Answers come back as the same three typed objects, each carrying that id. The authenticated app must have the llm_calls entitlement enabled on its plan. Requests that exceed the plan quota are rejected with 402. Token usage is recorded against the authenticated app and organization.

      Parameters

      • input: {
            model?: string;
            questions: (
                | {
                    criteria?: { false_description?: string; true_description?: string };
                    id: string;
                    instructions: string;
                    type: "boolean";
                }
                | {
                    criteria: Record<string, unknown>;
                    id: string;
                    instructions: string;
                    type: "choice";
                }
                | { criteria: string[]; id: string; instructions: string; type: "score" }
            )[];
            session_id?: string;
            state: unknown;
        }

        Request body.

        • Optionalmodel?: string

          Evaluation model identifier. Omit to use the platform default.

        • questions: (
              | {
                  criteria?: { false_description?: string; true_description?: string };
                  id: string;
                  instructions: string;
                  type: "boolean";
              }
              | {
                  criteria: Record<string, unknown>;
                  id: string;
                  instructions: string;
                  type: "choice";
              }
              | { criteria: string[]; id: string; instructions: string; type: "score" }
          )[]

          Nonempty list of boolean, choice, or score questions. Each question must include id, type, and instructions.

        • Optionalsession_id?: string

          Optional UUID grouping this evaluation under one session in the Developers dashboard. Generated when omitted.

        • state: unknown

          Shared content the model judges. A string, JSON object, or JSON array. Every question in the request sees this same state.

      Returns Promise<AIEvaluationResult>

      Typed answers as a list of boolean, choice, or score objects, plus model and usage.

    • List available evaluation models Returns the set of evaluation models that can be used with the evaluations endpoint. The list reflects models currently enabled for the platform and includes each model's identifier and whether it is the default. Use the id field from any entry in data as the model value when calling the evaluations endpoint.

      Returns Promise<
          {
              data: {
                  capabilities: ("search" | "image" | "temperature" | "thinking")[];
                  context_window?: number | null;
                  default: boolean;
                  id: string;
                  input_media_formats: string[];
                  name: string;
                  output_media_formats: string[];
              }[];
          },
      >

      Successful response

    Settings

    Member Visibility

    On This Page

    Constructors
    Methods