Bulk create tests
curl --request POST \
--url https://api.calibrate.artpark.ai/tests/bulk \
--header 'Content-Type: application/json' \
--header 'X-API-Key: <api-key>' \
--data '
{
"tests": [
{
"name": "<string>",
"conversation_history": [
{
"content": "<string>",
"tool_calls": [
{}
],
"tool_call_id": "<string>",
"name": "<string>"
}
],
"input": "<string>",
"evaluators": [
{
"evaluator_uuid": "f47ac10b-58cc-4372-a567-0e02b2c3d479",
"variable_values": {
"criteria": "The reply must cite the refund window"
}
}
],
"tool_calls": [
{
"tool": "<string>",
"arguments": {},
"accept_any_arguments": false
}
],
"inputs": {}
}
]
}
'import requests
url = "https://api.calibrate.artpark.ai/tests/bulk"
payload = { "tests": [
{
"name": "<string>",
"conversation_history": [
{
"content": "<string>",
"tool_calls": [{}],
"tool_call_id": "<string>",
"name": "<string>"
}
],
"input": "<string>",
"evaluators": [
{
"evaluator_uuid": "f47ac10b-58cc-4372-a567-0e02b2c3d479",
"variable_values": { "criteria": "The reply must cite the refund window" }
}
],
"tool_calls": [
{
"tool": "<string>",
"arguments": {},
"accept_any_arguments": False
}
],
"inputs": {}
}
] }
headers = {
"X-API-Key": "<api-key>",
"Content-Type": "application/json"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text){
"uuids": [
"b1c2d3e4-f5a6-7890-bcde-f12345678901"
],
"count": 123,
"message": "<string>",
"warnings": [
"<string>"
]
}{
"detail": [
{
"loc": [
"<string>"
],
"msg": "<string>",
"type": "<string>"
}
]
}Tests
Bulk create tests
Create many test cases at once and link them to your agents
POST
/
tests
/
bulk
Bulk create tests
curl --request POST \
--url https://api.calibrate.artpark.ai/tests/bulk \
--header 'Content-Type: application/json' \
--header 'X-API-Key: <api-key>' \
--data '
{
"tests": [
{
"name": "<string>",
"conversation_history": [
{
"content": "<string>",
"tool_calls": [
{}
],
"tool_call_id": "<string>",
"name": "<string>"
}
],
"input": "<string>",
"evaluators": [
{
"evaluator_uuid": "f47ac10b-58cc-4372-a567-0e02b2c3d479",
"variable_values": {
"criteria": "The reply must cite the refund window"
}
}
],
"tool_calls": [
{
"tool": "<string>",
"arguments": {},
"accept_any_arguments": false
}
],
"inputs": {}
}
]
}
'import requests
url = "https://api.calibrate.artpark.ai/tests/bulk"
payload = { "tests": [
{
"name": "<string>",
"conversation_history": [
{
"content": "<string>",
"tool_calls": [{}],
"tool_call_id": "<string>",
"name": "<string>"
}
],
"input": "<string>",
"evaluators": [
{
"evaluator_uuid": "f47ac10b-58cc-4372-a567-0e02b2c3d479",
"variable_values": { "criteria": "The reply must cite the refund window" }
}
],
"tool_calls": [
{
"tool": "<string>",
"arguments": {},
"accept_any_arguments": False
}
],
"inputs": {}
}
] }
headers = {
"X-API-Key": "<api-key>",
"Content-Type": "application/json"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text){
"uuids": [
"b1c2d3e4-f5a6-7890-bcde-f12345678901"
],
"count": 123,
"message": "<string>",
"warnings": [
"<string>"
]
}{
"detail": [
{
"loc": [
"<string>"
],
"msg": "<string>",
"type": "<string>"
}
]
}Authorizations
API key for authentication
Body
application/json
What the test judges:
response: judges the generated replytool_call: diffs the generated tool callsconversation: judges the full conversationgeneral: judges a single plain-text input/output pair with no conversation involved (e.g. summarization, extraction, classification)
Applied to every test in the batch
Available options:
response, tool_call, conversation, general Test items to create, at most 500 per request, with names unique within the batch
Show child attributes
Show child attributes
IDs of agents to link every created test to. Omit to link none
Example:
["a3b2c1d0-e5f4-3210-abcd-ef1234567890"]
Language written to each test's config.settings.language. Omit to leave unset
Response
Successful Response