Skip to main content

​
Installation & Setup

AgentTest makes it easy to run A/B tests on your AI agents or prompt variations. This guide shows you how to quickly get started using either our JavaScript SDK or the REST API. Use the AgentTest SDK to seamlessly integrate A/B testing in your Node.js or browser-based agents.

​
1. Install the SDK

npm install @agenttest/sdk

​
2. Initialize the client

import { AgentTestClient } from '@agenttest/sdk'

const client = new AgentTestClient({
  apiKey: 'your-api-key',
  host: 'https://api.agenttest.dev' // Your AgentTest endpoint
})

​
3. Assign a variant

const { variant, payload } = await client.assign('experiment-slug', {
  userId: 'user-123',
  context: { sessionId: 'abc' }
})

// Use the returned payload to run your agent

​
4. Log the result

await client.log('experiment-slug', {
  variantKey: variant,
  input: { prompt: 'Hello' },
  output: { response: 'Hi there!' },
  metrics: { latency: 120 },
  context: { sessionId: 'abc' }
})

​
🌐 Option 2: REST API

Prefer using raw HTTP requests? Use our simple, stateless REST API.

​
1. Assign a variant

POST https://api.agenttest.dev/experiment/:slug/assign
Headers: 
x-api-key: your-api-key
Content-Type: application/json
Request Body: 
{
  "userId": "user-123",
  "context": { "sessionId": "abc" }
}
Response: 
{
  "variant": "A",
  "payload": { ... }
}

​
2. Log a result

POST https://api.agenttest.dev/experiment/:slug/log
Headers: 
x-api-key: your-api-key
Content-Type: application/json
Request Body: 
{
  "variantKey": "A",
  "input": { "prompt": "Hello" },
  "output": { "response": "Hi there!" },
  "metrics": { "latency": 120 },
  "context": { "sessionId": "abc" }
}

​
✅ That’s It!

You’re now ready to start testing your agents with AgentTest. For more advanced use cases, explore:
  • Creating and managing experiments
  • Tracking custom metrics
  • Using with LangGraph, LangChain, or other frameworks
Need help? Reach out via GitHub or open an issue.