​
Installation & Setup

AgentTest makes it easy to run A/B tests on your AI agents or prompt variations. This guide shows you how to quickly get started using either our JavaScript SDK or the REST API.

Use the AgentTest SDK to seamlessly integrate A/B testing in your Node.js or browser-based agents.

​
1. Install the SDK

npm install @agenttest/sdk

​
2. Initialize the client

import { AgentTestClient } from '@agenttest/sdk'

const client = new AgentTestClient({
  apiKey: 'your-api-key',
  host: 'https://api.agenttest.dev' // Your AgentTest endpoint
})

​
3. Assign a variant

const { variant, payload } = await client.assign('experiment-slug', {
  userId: 'user-123',
  context: { sessionId: 'abc' }
})

// Use the returned payload to run your agent

​
4. Log the result

await client.log('experiment-slug', {
  variantKey: variant,
  input: { prompt: 'Hello' },
  output: { response: 'Hi there!' },
  metrics: { latency: 120 },
  context: { sessionId: 'abc' }
})

​
🌐 Option 2: REST API

Prefer using raw HTTP requests? Use our simple, stateless REST API.

​
1. Assign a variant

POST https://api.agenttest.dev/experiment/:slug/assign

Headers:

x-api-key: your-api-key
Content-Type: application/json

Request Body:

{
  "userId": "user-123",
  "context": { "sessionId": "abc" }
}

Response:

{
  "variant": "A",
  "payload": { ... }
}

​
2. Log a result

POST https://api.agenttest.dev/experiment/:slug/log

Headers:

x-api-key: your-api-key
Content-Type: application/json

Request Body:

{
  "variantKey": "A",
  "input": { "prompt": "Hello" },
  "output": { "response": "Hi there!" },
  "metrics": { "latency": 120 },
  "context": { "sessionId": "abc" }
}

​
✅ That’s It!

You’re now ready to start testing your agents with AgentTest.

For more advanced use cases, explore:

  • Creating and managing experiments
  • Tracking custom metrics
  • Using with LangGraph, LangChain, or other frameworks

Need help? Reach out via GitHub or open an issue.