# Arize AX integration

> AI observability and evaluation platform: traces and spans, projects, datasets, experiments, evaluators, prompts, annotations, monitors, and account administration.

- Authentication: apiKey

## Tools (161)

- **Add a user to an organization**: Add a user to an organization. Add a single existing account user to an organization with a specified role. **Payload Requirements** - `user_id` and `role` are both required.
- **Add a user to a space**: Add a user to a space. Add a single existing account user to a space with a specified role.
- **Add new examples to a dataset**: Add new examples to a dataset. Appends new examples to an existing dataset. If the dataset version is not passed, the latest version is selected.
- **Annotate a batch of dataset examples**: Annotate a batch of dataset examples. Write human annotations to a batch of examples in a dataset.
- **Annotate a batch of experiment runs**: Annotate a batch of experiment runs. Write human annotations to a batch of runs in an experiment.
- **Annotate a batch of project spans**: Annotate a batch of project spans. Write human annotations to a batch of spans in a project.
- **Annotate a record**: Annotate a record. Submit annotations for an annotation queue record. Annotations are upserted into the underlying data source by annotation config name.
- **Append runs to an experiment**: Append runs to an experiment. Append new runs to an existing experiment. **Payload Requirements** - Provide between 1 and 1000 runs in `experiment_runs`.
- **Assign users to a record**: Assign users to a record. Assign users to an annotation queue record. Fully replaces the current record-level user assignment.
- **Attach tags to a annotation config**: Attach tags to a annotation config. Attach one or more existing tags to a annotation config.
- **Attach tags to a annotation queue**: Attach tags to a annotation queue. Attach one or more existing tags to a annotation queue.
- **Attach tags to a dataset**: Attach tags to a dataset. Attach one or more existing tags to a dataset. **Payload Requirements** - `tag_ids` is required and must contain between 1 and 100 tag IDs.
- **Attach tags to a evaluator**: Attach tags to a evaluator. Attach one or more existing tags to a evaluator. **Payload Requirements** - `tag_ids` is required and must contain between 1 and 100 tag IDs.
- **Attach tags to a experiment**: Attach tags to a experiment. Attach one or more existing tags to a experiment. **Payload Requirements** - `tag_ids` is required and must contain between 1 and 100 tag IDs.
- **Attach tags to a monitor**: Attach tags to a monitor. Attach one or more existing tags to a monitor. **Payload Requirements** - `tag_ids` is required and must contain between 1 and 100 tag IDs.
- **Attach tags to a project**: Attach tags to a project. Attach one or more existing tags to a project. **Payload Requirements** - `tag_ids` is required and must contain between 1 and 100 tag IDs.
- **Attach tags to a prompt**: Attach tags to a prompt. Attach one or more existing tags to a prompt. **Payload Requirements** - `tag_ids` is required and must contain between 1 and 100 tag IDs.
- **Cancel task run**: Cancel task run. Cancel a running task run. Only valid when the run's current status is `pending` or `running`.
- **Create a dataset**: Create a dataset. Create a new dataset with JSON examples. Empty datasets are not allowed. **Payload Requirements** - The dataset name must be unique within the given space.
- **Create an AI integration**: Create an AI integration. Create a new AI integration for an external LLM provider. **Payload Requirements** - `name` and `provider` are required.
- **Create an annotation config**: Create an annotation config. Create a new annotation config. **Payload Requirements** - The annotation config name must be unique within the given space. **Valid example**
- **Create an annotation queue**: Create an annotation queue. Create a new annotation queue. **Payload Requirements** - The annotation queue name must be unique within the given space (among active queues).
- **Create an API key**: Create an API key. Create a new API key for the authenticated user.
- **Create an experiment**: Create an experiment. Create a new experiment. Empty experiments are not allowed. An experiment belongs to a space and may optionally be associated with a dataset.
- **Create an integration**: Create an integration. Create a new integration. The `type` field selects the config shape; for `LLM`, `config.provider` selects the per-provider config.
- **Create annotation queue records**: Create annotation queue records. Add new records from spans, traces, sessions, or dataset examples to an existing annotation queue.
- **Create an organization**: Create an organization. Create a new organization within the account. **Payload Requirements** - `name` is required and must be unique within the account.
- **Create a project**: Create a project. Create a new project given a name and space ID. **Payload Requirements** - The project name must be unique within the given space.
- **Create a prompt**: Create a prompt. Create a new prompt with an initial version. **Payload Requirements** - The prompt name must be unique within the given space. - At least one message is required.
- **Create a prompt version**: Create a prompt version. Create a new version of an existing prompt. **Payload Requirements** - A `commit_message` is required. - At least one message is required in `messages`.
- **Create a role**: Create a role. Create a new custom role for the authenticated user's account. **Payload Requirements** - `name` is required and must be unique within the account.
- **Create a role binding**: Create a role binding. Create a new role binding that assigns a role to a user on a resource.
- **Create a space**: Create a space. Create a new space within an organization. **Payload Requirements** - `name` and `organization_id` are required.
- **Create a tag**: Create a tag. Create a tag in a space. Tags are shared within the space and can then be attached to resources across the platform.
- **Create a user**: Create a user. Create a new account user with explicit invite control. **Invite modes** - `NONE` — add the user directly with no invitation (for SSO-only accounts).
- **Create a webhook**: Create a webhook. Create a new webhook in an organization. **Payload Requirements** - `organization_id`, `name`, and `url` are required.
- **Create a webhook subscription**: Create a webhook subscription. Subscribe a webhook to one event on a prompt or evaluator. To deliver several events to the same webhook, create one subscription per event.
- **Create evaluator**: Create evaluator. Creates a new evaluator with an initial version. **Payload Requirements** - The evaluator `name` must be unique within the given space.
- **Create evaluator version**: Create evaluator version. **Endpoint:** `POST /v2/evaluators/{evaluator_id}/versions` Create a new version of an existing evaluator.
- **Create task**: Create task. Creates a new task. Supported task types: | `type` | Data source | Notes | |---|---|---| | `TEMPLATE_EVALUATION` | `project_id` or `dataset_id` | Requires (see official docs)
- **Delete a dataset**: Delete a dataset. Delete a dataset by its ID. This operation is irreversible.
- **Delete an AI integration**: Delete an AI integration. Delete an AI integration by its ID. This operation is irreversible.
- **Delete an annotation config**: Delete an annotation config. Delete an annotation config by its ID.
- **Delete an annotation queue**: Delete an annotation queue. Delete an annotation queue by its ID. This operation is irreversible.
- **Delete an experiment**: Delete an experiment. Delete an experiment by its ID. This operation is irreversible.
- **Delete an integration**: Delete an integration. Delete an integration by its ID. This operation is irreversible.
- **Delete annotation queue records**: Delete annotation queue records. Delete one or more records from an annotation queue by their IDs.
- **Delete an organization**: Delete an organization. Delete an organization by its ID.
- **Delete a project**: Delete a project. Delete a project by its ID. This operation is irreversible.
- **Delete a prompt**: Delete a prompt. Delete a prompt by its ID. This operation is irreversible.
- **Delete a role**: Delete a role. Delete a custom role by its ID (soft-delete). Predefined roles cannot be deleted.
- **Delete a role binding**: Delete a role binding. Delete a role binding by its ID. **Authorization** Requires `ROLE_BINDING_DELETE` permission on the binding's resource.
- **Delete a space**: Delete a space. Delete a space by its ID. This deletes the space and all resources that belong to it (models, monitors, dashboards, datasets, custom metrics, etc).
- **Delete a tag**: Delete a tag. Delete a tag by its ID. This operation is irreversible. The tag is detached from every resource it was attached to. The resources themselves are not affected.
- **Delete a user**: Delete a user. Permanently block a user from the account. The user's status is set to `inactive` and they can no longer log in.
- **Delete a webhook**: Delete a webhook. Delete a webhook by its ID. The webhook stops receiving events and is detached from every prompt, evaluator, and monitor it was subscribed to.
- **Delete a webhook subscription**: Delete a webhook subscription. Delete a webhook subscription by its ID. The webhook stops receiving that event from the source.
- **Delete dataset examples**: Delete dataset examples. Delete a collection of examples from a dataset by their IDs.
- **Delete evaluator**: Delete evaluator. Deletes an evaluator and all its versions. This operation is irreversible. An evaluator that is still used by an online task returns `409`.
- **Delete evaluator versions**: Delete evaluator versions. **Endpoint:** `DELETE /v2/evaluators/{evaluator_id}/versions` Deletes a batch of versions belonging to the evaluator identified by the `evaluator_id` (see official docs)
- **Delete spans**: Delete spans. Permanently deletes spans by their span IDs. This operation is irreversible. Accepts between 1 and 5000 span IDs per request.
- **Delete task**: Delete task. Deletes a task and all its associated resources (evaluator configs, runs, etc.). This operation is irreversible.
- **Detach tags from a dataset**: Detach tags from a dataset. Detach one or more tags from a dataset. **Payload Requirements** - `tag_ids` is required and must contain between 1 and 100 tag IDs.
- **Detach tags from a evaluator**: Detach tags from a evaluator. Detach one or more tags from a evaluator. **Payload Requirements** - `tag_ids` is required and must contain between 1 and 100 tag IDs.
- **Detach tags from a monitor**: Detach tags from a monitor. Detach one or more tags from a monitor. **Payload Requirements** - `tag_ids` is required and must contain between 1 and 100 tag IDs.
- **Detach tags from a project**: Detach tags from a project. Detach one or more tags from a project. **Payload Requirements** - `tag_ids` is required and must contain between 1 and 100 tag IDs.
- **Detach tags from a prompt**: Detach tags from a prompt. Detach one or more tags from a prompt. **Payload Requirements** - `tag_ids` is required and must contain between 1 and 100 tag IDs.
- **Get a dataset**: Get a dataset. Get a dataset object by its ID. This includes the dataset's versions, sorted by creation date, with the most recently-created version first.
- **Get a monitor**: Get a monitor. Get a monitor by its ID. The response shape varies by `type` (`data_quality`, `performance`, `drift`, `custom_metric`, `tracing`) <Warning>This endpoint is in (see official docs)
- **Get an AI integration**: Get an AI integration. Get a specific AI integration by its ID.
- **Get an annotation config**: Get an annotation config. Get an annotation config object by its ID.
- **Get an annotation queue**: Get an annotation queue. Get an annotation queue object by its ID.
- **Get an experiment**: Get an experiment. Get an experiment object by its ID. The response does not include the experiment's runs.
- **Get an integration**: Get an integration. Get a specific integration by its ID.
- **Get an organization**: Get an organization. Get a specific organization by its ID.
- **Get a project**: Get a project. Get a specific project by its ID.
- **Get a prompt**: Get a prompt. Get a specific prompt by its ID. The response always includes a resolved version. By default, the latest version is returned.
- **Get a prompt version**: Get a prompt version. Get a specific prompt version by its ID.
- **Get a role**: Get a role. Get a role by its ID. Requires `ROLE_READ`
- **Get a role binding**: Get a role binding. Get a single role binding by its ID. **Authorization** Requires `ROLE_BINDING_READ` permission on the binding's resource.
- **Get a space**: Get a space. Get a specific space by its ID.
- **Get a user**: Get a user. Get a specific user by their ID. Requires account admin role, account member role, or USER_READ permission at the account level.
- **Get a webhook**: Get a webhook. Get a specific webhook by its ID. <Warning>This endpoint is in alpha, read more [here](https://arize.com/docs/ax/rest-reference#api-version-stages).</Warning>
- **Get a webhook subscription**: Get a webhook subscription. Get a specific webhook subscription by its ID.
- **Get evaluator**: Get evaluator. Returns an evaluator and a resolved version. By default, the latest version is included. Use the version_id query parameter to resolve a specific version.
- **Get evaluator version**: Get evaluator version. Get a specific evaluator version by its unique identifier.
- **Get task**: Get task. Returns a single task by its ID.
- **Get task run**: Get task run. Returns a single task run. Use this to poll for status updates.
- **GraphQL Mutation**: Runs a GraphQL mutation against the Arize AX GraphQL API (updates or deletes data).
- **GraphQL Query**: Runs a read-only GraphQL query against the Arize AX GraphQL API (single endpoint, same API key).
- **List AI integrations**: List AI integrations. List AI integrations the user has access to.
- **List annotation configs**: List annotation configs. List annotation configs the user has access to.
- **List annotation queue records**: List annotation queue records. List the records in an annotation queue with their data and annotations.
- **List annotation queues**: List annotation queues. List annotation queues the user has access to.
- **List API keys**: List API keys. List API keys. Returns metadata for each key (id, name, description, key_type, status, redacted_key, created_at, expires_at, created_by_user_id).
- **List audit logs**: List audit logs. Retrieve a paginated list of authenticated user audit log entries for the account. Results are ordered newest first.
- **List a webhook's delivery attempts**: List a webhook's delivery attempts. List the webhook's delivery attempts, most recent first. Each event may have several attempts, since failed deliveries are retried.
- **List dataset examples**: List dataset examples. List examples for a given dataset and version. If version is not passed, the latest version is selected.
- **List datasets**: List datasets. List datasets the user has access to. The datasets are sorted by creation date, with the most recently created datasets coming first.
- **List evaluators**: List evaluators. List evaluators the user has access to, sorted by update date (most recent first). When `space_id` is provided, results are limited to that space.
- **List evaluator templates**: List evaluator templates. Retrieve the built-in LLM-as-a-judge evaluator templates.
- **List evaluator versions**: List evaluator versions. List all versions of an evaluator with cursor-based pagination.
- **List experiment runs**: List experiment runs. List runs for a given experiment. The runs are returned in a stable insertion order.
- **List experiments**: List experiments. List experiments a user has access to.
- **List integrations**: List integrations. List integrations the user has access to, ordered by creation time (newest first).
- **List monitors**: List monitors. List monitors the caller can read, with filtering and cursor-based pagination. Results are ordered by creation time, newest first.
- **List organizations**: List organizations. List organizations the user has access to.
- **List projects**: List projects. List projects the user has access to.
- **List prompts**: List prompts. List prompts the user has access to. The prompts are sorted by update date, with the most recently updated prompts coming first.
- **List prompt versions**: List prompt versions. List all versions of a prompt, sorted by creation date with the most recently created versions first.
- **List resource restrictions the caller is permitted to manage**: List resource restrictions the caller is permitted to manage.. List active resource restrictions the authenticated user is permitted to manage.
- **List role bindings**: List role bindings. List role bindings for the authenticated user's account, filtered by resource type. Results are paginated; use `limit` and `cursor` for subsequent pages.
- **List roles**: List roles. List custom and predefined roles for the authenticated user's account. Results are paginated; use `limit` and `cursor` for subsequent pages.
- **List spaces**: List spaces. List spaces the user has access to.
- **List spans**: List spans. Returns a paginated list of spans. Spans are ordered by `start_time` from newest to oldest.
- **List tags on a dataset**: List tags on a dataset. List the tags attached to a dataset. Tags are shared within the space, so the same tag may appear on many resources.
- **List tags on a monitor**: List tags on a monitor. List the tags attached to a monitor. Tags are shared within the space, so the same tag may appear on many resources.
- **List tags on an annotation config**: List tags on an annotation config. List the tags attached to an annotation config. Tags are shared within the space, so the same tag may appear on many resources.
- **List tags on an annotation queue**: List tags on an annotation queue. List the tags attached to an annotation queue. Tags are shared within the space, so the same tag may appear on many resources.
- **List tags on an evaluator**: List tags on an evaluator. List the tags attached to an evaluator. Tags are shared within the space, so the same tag may appear on many resources.
- **List tags on an experiment**: List tags on an experiment. List the tags attached to an experiment. Tags are shared within the space, so the same tag may appear on many resources.
- **List tags on a project**: List tags on a project. List the tags attached to a project. Tags are shared within the space, so the same tag may appear on many resources.
- **List tags on a prompt**: List tags on a prompt. List the tags attached to a prompt. Tags are shared within the space, so the same tag may appear on many resources.
- **List task runs**: List task runs. List all runs for a task with cursor-based pagination.
- **List tasks**: List tasks. List tasks the user has access to, with cursor-based pagination. Filter by space, space name, task name, project, dataset, or task type using query parameters.
- **List traces**: List traces. Returns a paginated list of traces for a project, each carrying its full (flat) list of spans plus lightweight roll-up metadata.
- **List users**: List users. List users in the account with cursor-based pagination. Results are sorted by creation date ascending (oldest first).
- **List webhooks**: List webhooks. List the webhooks in the organizations the user has access to, most recently created first.
- **List webhook subscriptions**: List webhook subscriptions. List webhook subscriptions attached to prompts and evaluators the caller can read, most recently created first.
- **Refresh an API key**: Refresh an API key. Atomically revoke an existing API key and issue a replacement with the same metadata (name, description, and key type).
- **Remove a label from a prompt version**: Remove a label from a prompt version. Remove a specific label from a prompt version.
- **Remove a user from an organization**: Remove a user from an organization. Remove a user from the organization and all its child spaces (membership cascade). Requires organization admin.
- **Remove a user from a space**: Remove a user from a space. Remove a user from a space. This removes both the legacy `SpaceMembers` row and any RBAC role bindings for the user on this space.
- **Resend a user invitation**: Resend a user invitation. Resend the invitation email for a pending (unverified) user. Generates a new verification token and sends a fresh email to the user's address.
- **Resolve a label to a prompt version**: Resolve a label to a prompt version. Resolve a label on a prompt to the version it points to. Returns the full `PromptVersion` object that this label currently references.
- **Restrict a resource**: Restrict a resource. Mark a resource as restricted. Only space admins or users with the the correct permissions can perform this action. Idempotent.
- **Revoke an API key**: Revoke an API key. Revoke an API key by its ID. The key will immediately stop working for authentication. Revoking an already-revoked key is a no-op and still returns `204`.
- **Search dataset examples**: Search dataset examples. Search examples for a given dataset with an optional SQL-like filter. Examples are returned in ascending order of `created_at`, with `id` as a tiebreaker.
- **Search experiment runs**: Search experiment runs. Search runs for a given experiment with an optional SQL-like filter. Runs are returned in stable `id` ascending order.
- **Send a test event to a webhook**: Send a test event to a webhook. Send a test event to the webhook's endpoint and report the outcome.
- **Set labels on a prompt version**: Set labels on a prompt version. Set (replace) all labels on a prompt version. This is an idempotent operation.
- **Trigger a password-reset email for a user**: Trigger a password-reset email for a user. Generates a reset token and sends the user a password-reset email with a 30-minute link.
- **Trigger a task run**: Trigger a task run. Triggers a new run on an existing task. The run is queued and processed asynchronously.
- **Unrestrict a resource**: Unrestrict a resource. Remove restriction from a resource.
- **Update a dataset**: Update a dataset. Update an existing dataset by its ID. **Payload Requirements** - `name` is required. - `name` must be unique within the space (409 Conflict if duplicate).
- **Update an AI integration**: Update an AI integration. Update an AI integration's configuration. At least one field must be provided. Omitted fields are left unchanged.
- **Update an annotation config**: Update an annotation config. Update an annotation config by its ID. **Payload Requirements** - `annotation_config_type` is required and must match the stored config's type.
- **Update an annotation queue**: Update an annotation queue. Update an annotation queue by its ID. At least one field must be provided.
- **Update an integration**: Update an integration. Partially update an integration. `type` is immutable (and, for `LLM`, `config.provider`). At least one field must be provided.
- **Update an organization**: Update an organization. Update an organization's metadata by its ID. At least one field must be provided.
- **Update a project**: Update a project. Update a project by its ID (rename). **Payload Requirements** - `name` must be provided and must be unique within the space.
- **Update a prompt**: Update a prompt. Update a prompt's metadata by its ID. Currently supports updating the description.
- **Update a role**: Update a role. Update a custom role by its ID. At least one field must be provided. Predefined roles cannot be updated.
- **Update a role binding**: Update a role binding. Update an existing role binding by changing its assigned role. **Payload Requirements** - `role_id` is required and replaces the currently assigned role.
- **Update a space**: Update a space. Update a space's metadata by its ID. At least one field must be provided.
- **Update a tag**: Update a tag. Update a tag's name, description, or color. Tags are shared, so an update is visible on every resource the tag is attached to.
- **Update a user**: Update a user. Update a user's display name and/or developer permission. **Payload Requirements** - At least one of `name` or `is_developer` must be provided.
- **Update a webhook**: Update a webhook. Update a webhook by its ID. At least one field must be provided.
- **Update evaluator**: Update evaluator. Update an evaluator's metadata. At least one field must be provided. Omitted fields are left unchanged.
- **Update existing examples in a dataset**: Update existing examples in a dataset. Updates existing dataset examples by matching their `id` field.
- **Update task**: Update task. Update a task's mutable fields. At least one field must be provided. Omitted fields are left unchanged.

Connect Arize AX in General Input: https://www.generalinput.com/apps/arize