CasePilot API Documentation

Integrate AI-powered test case generation into your CI/CD pipeline, CLI tools, or custom workflows.

Overview

CasePilot generates structured, categorized, automation-ready test cases from user story descriptions using a two-pass AI pipeline (Worker + Judge). The Worker generates initial test cases and the Judge reviews them for coverage gaps, duplicates, and quality — producing significantly higher quality than single-pass approaches.

The REST API lets any tool integrate with CasePilot — CI/CD systems, CLI scripts, test management platforms, or custom dashboards. You get the same two-pass AI quality available in the Azure DevOps extension.

Authentication

All API endpoints require authentication via an API key. Pass your key using one of these methods:

API keys are created from the CasePilot extension settings panel in Azure DevOps. All keys are prefixed with cp_live_ for production or cp_test_ for testing.

Generate Test Cases

Enrich Story (StoryPilot)

Request body:

{
  "orgId": "my-organization",
  "title": "User can reset password via email",
  "context": "Mobile app, React Native",
  "existingDescription": "",
  "existingAcceptanceCriteria": ""
}

Response:

{
  "enrichedStory": {
    "description": "As a user, I want to reset my password...",
    "acceptanceCriteria": [
      { "text": "Reset email arrives within 2 minutes", "category": "functional" },
      { "text": "Link expires after 24 hours", "category": "functional" }
    ],
    "priority": { "level": 2, "justification": "Core auth flow" },
    "storyPoints": 5,
    "suggestedTags": ["auth", "email"],
    "risks": ["Email delivery delay"],
    "definitionOfDone": ["Unit tests pass", "E2E test for reset flow"]
  },
  "metadata": {
    "model": "gpt-4o-mini",
    "tokensUsed": 1800,
    "generationTimeMs": 2500,
    "qualityScore": 8,
    "judgeNotes": "Well-structured with testable AC."
  }
}

Cross-Story Coverage Matrix

Request body: { orgId, stories: [{ id, title, state, testCaseCount, testCaseIds, description?, acceptanceCriteria?, priority?, storyPoints? }] }

Max 500 stories per request.

Response: Coverage percentages (test, AC, overall), gap clusters (high-priority, large, well-defined, done-without-tests), cross-story dependencies (shared keywords), per-story heatmap cells.

Sprint Test Coverage

Request body: { orgId, sprintName, stories: [{ id, title, state, testCaseCount, testCaseIds, ... }] }

Response: Coverage percentage, risk summary (critical/high/medium/low), per-story risk analysis with factors.

Incident-to-Test Bridge

Request body: { orgId, incident: { title, description, rootCause?, impact?, resolution? }, maxTestCases? }

Response: Regression test cases with steps, incident coverage mapping, prevention insights, and quality score.

Test Health Score

Request body:

{
  "projectId": "my-project",
  "tests": [
    {
      "name": "Login flow e2e",
      "category": "e2e",
      "passed": true,
      "duration": 4500,
      "flaky": false,
      "lastRun": "2026-04-14T10:00:00Z"
    }
  ]
}

Response:

{
  "projectId": "my-project",
  "score": 82,
  "grade": "B",
  "factors": [
    { "name": "passRate", "score": 95, "weight": 0.4, "detail": "95% of tests passing" },
    { "name": "flakiness", "score": 70, "weight": 0.25, "detail": "3 flaky tests detected" }
  ],
  "summary": "Good overall health with some flaky tests to address.",
  "recommendations": [
    "Fix 3 flaky tests to improve reliability",
    "Add more integration tests for payment module"
  ]
}

Failure Triage

Request body:

{
  "projectId": "my-project",
  "failures": [
    {
      "name": "Checkout timeout test",
      "errorMessage": "TimeoutError: waiting for selector",
      "stackTrace": "at Object.waitForSelector ...",
      "category": "e2e",
      "duration": 30000,
      "occurrences": 5,
      "totalRuns": 20
    }
  ]
}

Response:

{
  "projectId": "my-project",
  "groups": [
    {
      "category": "timeout",
      "count": 3,
      "failures": [ ... ],
      "rootCause": "Slow API responses under load",
      "suggestion": "Add explicit wait timeouts and retry logic"
    }
  ],
  "summary": {
    "totalFailures": 12,
    "groupCount": 4,
    "topCategory": "timeout",
    "estimatedFlakyCount": 2
  }
}

Usage & Credits

Response:

{
  "used": 42,
  "limit": 200,
  "remaining": 158,
  "planId": "starter",
  "periodStart": "2026-04-01T00:00:00Z",
  "periodEnd": "2026-04-30T23:59:59Z"
}

API Key Management

Request body:

{
  "name": "CI Pipeline Key",
  "orgId": "my-organization"
}

Response:

{
  "id": "key_abc123",
  "name": "CI Pipeline Key",
  "key": "cp_live_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "createdAt": "2026-04-03T10:00:00Z"
}

Request body:

{
  "keyId": "key_abc123"
}

Request Schema

Request body for POST /api/v1/generate:

projectKnowledge sub-fields

settings sub-fields

Response Schema

Successful response from POST /api/v1/generate:

{
  "testCases": [
    {
      "title": "Verify successful login with valid credentials",
      "description": "Test that a user can log in with correct email and password",
      "category": "positive",
      "priority": 1,
      "preconditions": [
        "User account exists with verified email",
        "User is on the login page"
      ],
      "steps": [
        {
          "stepNumber": 1,
          "action": "Enter valid email in the email field",
          "expectedResult": "Email is displayed in the field",
          "testData": "[email protected]"
        },
        {
          "stepNumber": 2,
          "action": "Enter valid password in the password field",
          "expectedResult": "Password is masked in the field",
          "testData": "SecurePass123!"
        },
        {
          "stepNumber": 3,
          "action": "Click the Login button",
          "expectedResult": "User is redirected to the dashboard",
          "testData": null
        }
      ],
      "automationHints": "Locate email input: [data-testid='email-input']"
    }
  ],
  "metadata": {
    "model": "gpt-4o-mini",
    "tokensUsed": 2450,
    "generationTimeMs": 3200,
    "qualityScore": 8,
    "coverageGaps": [],
    "judgeNotes": "Good coverage of positive and negative scenarios."
  },
  "usage": {
    "used": 47,
    "limit": 200,
    "remaining": 153,
    "planId": "starter"
  }
}

testCases[] fields

metadata fields

Code Examples

curl -X POST https://api.iklab.dev/api/v1/generate \
  -H "Content-Type: application/json" \
  -H "X-API-Key: cp_live_your_api_key_here" \
  -d '{
    "title": "User can reset password via email",
    "description": "As a user, I want to reset my password so I can regain access to my account.",
    "acceptanceCriteria": "- User receives reset email within 2 minutes\n- Reset link expires after 24 hours\n- Password must meet complexity requirements",
    "settings": {
      "maxTestCases": 5,
      "categories": ["positive", "negative", "edge"],
      "includeAutomationHints": true
    }
  }'

import requests

API_KEY = "cp_live_your_api_key_here"
BASE_URL = "https://api.iklab.dev"

response = requests.post(
    f"{BASE_URL}/api/v1/generate",
    headers={
        "Content-Type": "application/json",
        "X-API-Key": API_KEY,
    },
    json={
        "title": "User can reset password via email",
        "description": "As a user, I want to reset my password so I can regain access.",
        "acceptanceCriteria": (
            "- User receives reset email within 2 minutes\n"
            "- Reset link expires after 24 hours\n"
            "- Password must meet complexity requirements"
        ),
        "settings": {
            "maxTestCases": 5,
            "categories": ["positive", "negative", "edge"],
            "includeAutomationHints": True,
        },
    },
)

data = response.json()
for tc in data["testCases"]:
    print(f"[{tc['category']}] {tc['title']}")

const API_KEY = "cp_live_your_api_key_here";
const BASE_URL = "https://api.iklab.dev";

const response = await fetch(`${BASE_URL}/api/v1/generate`, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "X-API-Key": API_KEY,
  },
  body: JSON.stringify({
    title: "User can reset password via email",
    description: "As a user, I want to reset my password so I can regain access.",
    acceptanceCriteria: [
      "- User receives reset email within 2 minutes",
      "- Reset link expires after 24 hours",
      "- Password must meet complexity requirements",
    ].join("\n"),
    settings: {
      maxTestCases: 5,
      categories: ["positive", "negative", "edge"],
      includeAutomationHints: true,
    },
  }),
});

const data = await response.json();
for (const tc of data.testCases) {
  console.log(`[${tc.category}] ${tc.title}`);
}

using var client = new HttpClient();
client.BaseAddress = new Uri("https://api.iklab.dev");
client.DefaultRequestHeaders.Add("X-API-Key", "cp_live_your_api_key_here");

var request = new
{
    title = "User can reset password via email",
    description = "As a user, I want to reset my password so I can regain access.",
    acceptanceCriteria = string.Join("\n", new[]
    {
        "- User receives reset email within 2 minutes",
        "- Reset link expires after 24 hours",
        "- Password must meet complexity requirements"
    }),
    settings = new
    {
        maxTestCases = 5,
        categories = new[] { "positive", "negative", "edge" },
        includeAutomationHints = true
    }
};

var json = JsonSerializer.Serialize(request);
var content = new StringContent(json, Encoding.UTF8, "application/json");
var response = await client.PostAsync("/api/v1/generate", content);

var result = await response.Content.ReadAsStringAsync();
Console.WriteLine(result);

Error Codes

All error responses follow this format:

{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Field 'title' is required and must not be empty."
  }
}

Rate Limits

API requests are rate-limited using a sliding window per API key:

Rate limit headers are included in every response:

Usage & Billing

API usage shares the same credit pool as the Azure DevOps extension. Credits are metered per test case generated (not per API request). A single request generating 5 test cases consumes 5 credits.

Monitor your usage by calling GET /api/v1/usage or checking the usage field in generation responses.