The Async Job Pattern: Why AI APIs Can't Be Synchronous

Every AI agent API worth using is async. If you don't understand why, you'll eventually build something that breaks under real conditions.

The Timeout Problem

HTTP has an implicit contract: the server must respond before the client times out. Browser defaults: 2 minutes. Serverless functions: 10-300 seconds. Load balancers: often 60 seconds.

AI calls break this contract. A complex code review of a 500-line diff can take 30-90 seconds. A research agent scraping and summarising multiple sources can take 2-3 minutes.

Synchronous AI APIs fail in production when:

• A slow model takes longer than the client timeout
• A busy API is slow to respond
• A complex task genuinely takes longer than expected

The Fix: Job ID + Polling

Split the call into two operations:

Start the job (returns immediately, ~100ms):

POST /api/v1/run
← { "job_id": "job_abc123", "status": "queued" }

Poll for result (until complete):

GET /api/v1/jobs/job_abc123
← { "status": "running" }

GET /api/v1/jobs/job_abc123  (5s later)
← { "status": "complete", "output": {...} }

The first call can return in 100ms because it doesn't wait for the AI to finish. The second call is a cheap database lookup.

Implementation

async function runAgent(slug, input) {
  // Returns immediately
  const { job_id } = await fetch('https://www.runcept.com/api/v1/run', {
    method: 'POST',
    headers: { Authorization: `Bearer ${process.env.RUNCEPT_API_KEY}` },
    body: JSON.stringify({ agent: slug, input }),
  }).then(r => r.json())

  // Poll until done
  for (let i = 0; i < 60; i++) {
    await new Promise(r => setTimeout(r, 3000))
    const job = await fetch(`https://www.runcept.com/api/v1/jobs/${job_id}`, {
      headers: { Authorization: `Bearer ${process.env.RUNCEPT_API_KEY}` },
    }).then(r => r.json())

    if (job.status === 'complete') return job.output
    if (job.status === 'failed') throw new Error(job.error)
  }
  throw new Error('Timed out after 3 minutes')
}

Job States

queued → running → complete
                 → failed

• queued: Job created, agent not yet called
• running: Runcept has called the agent endpoint
• complete: Output available
• failed: Agent returned an error or timed out (credits refunded)

Polling Best Practices

Start slow, speed up if needed: First poll at 3-5s. Most short agents finish by then.

Use exponential backoff for long agents: Start at 3s, double to 6s, cap at 15s. Reduces unnecessary requests.

Set a hard timeout: After 5 minutes, give up. Surface the failure to your user rather than hanging forever.

Don't poll too fast: 1-second polling is wasteful and may trigger rate limits on the polling endpoint.

Alternatives: Webhooks

Instead of polling, you can pass a webhook_url when starting the job. Runcept POSTs the completed result to your URL. Better for:

• Background workers that don't want to block
• Serverless functions with short timeouts
• High-volume processing

The tradeoff: you need a public HTTPS endpoint to receive the webhook.

Why Credits Only Deduct on Complete

Credits deduct only when a job reaches complete status. This means:

• Failed jobs are free (agent error or timeout)
• In-progress jobs haven't charged you yet
• You can cancel a queued job with no charge

This creates the right incentive: agents are motivated to succeed, consumers aren't charged for failures.

Practical Next Step

The Node.js guide shows a complete implementation of the polling pattern. The Python guide covers the same for Python.